From Observation to Questionnaires

Introduction

Data collection is an indispensable part of research. Data collection refers to the process of gathering and measuring information to answer the research questions, test hypotheses, and evaluate the outcomes (Saunders et al. 2015). Data collection primarily aims at capturing quality evidence which then translates to rich data analysis thus allowing a researcher to answer research questions in a credible and convincing way (Oh 2014). There are various data collection methods including questionnaires, interviews, observation, document analysis, focus group discussions, case studies, critical incidents, diaries, and portfolios among others (Quinlan et al. 2019). For students who are pursuing their studies in fields like social sciences, business, or healthcare, the requirement for data analysis dissertation help becomes very essential. This portfolio discusses the concepts of observation, questionnaires, and mixed methods approaches to data collection. The portfolio incorporates two tasks: task one and task two. Task one outlines the key purpose of observation and its advantages and disadvantages, the dimensions that distinguish different approaches to observation, ways of minimising observer bias, and the ethics of using concealed observation as a research method. Task two focuses on online questionnaires and their advantages and disadvantages, the principles of wording and their importance in designing questionnaires, the relationship between mixed methods of data collection and reliability and validity, as well as the role of mixed methods in overcoming built-in biases of individual data collection methods. The portfolio also has a conclusion that restates thesis statement and ties up the paper.

Task 1

Key purpose of observation and its advantages and disadvantages

Observation as a data collection method can be defined as the process of watching, recording, analysing, and interpreting actions, behaviour or events (Saunders et al. 2015). The key purpose of observe is to systematically observe, record, describe, analyse, and interpret people’s behaviour (Jamshed 2014). Used as a data collection method, Suen and Ary (2014) argue that observation should help the researcher to describe a situation in a way that produces a written photograph. In the same vein, Khan (2014) states that observation should allow a researcher to describe an activity, the place it took place, and the persons present in the activity.

A key advantage of observation as a data collection method is that its helps a researcher identify and guide relationships with the participants. According to, Sutton and Austin (2015) through observation a researcher is able to learn how people interact in a given setting and how things are organised in that setting. On the other hand, Khan (2014) notes that observation is advantageous as it allows a researcher to learn what is important to people in a particular setting. The nature of data collected through the observational approach is rich and detailed thus offering in-depth answers to the research questions (Jamshed 2014). Through observation, a research is able to record information on human behaviour directly without relying on the retrospective account of the participants (Suen and Ary 2014). Another advantage of observation as a data collection method is that a researcher might be able to discover what the participants take for granted thus consider unimportant (Khan 2014). Further, through observation, a researcher can be able to gather information on the behaviour and environment of people that cannot speak for themselves making the results more generalizable (Jamshed 2014).

Observation also has its limitations. First, people are likely to consciously or unconsciously change their behaviour when they are under observation resulting in inaccurate data (Khan 2014). Second, the behaviour or environment of interest might be inaccessible making observation impossible or very difficult (Saunders et al. 2015). For example, it is very difficult to observe human sexual behaviour and different forms of deviance. Third, observations are always filtered through the researcher’s interpretive lens making it impossible or very hard for observation to provide direct representations of the reality (Suen and Ary 2014). Finally, the contribution and quality of the observations are determined by a researcher’s ability to accurately describe what was observed (Suen and Ary 2014).

Four dimensions that distinguish various approaches to observation

Business researchers have used different approaches of observation, which can be differentiated using four key dimensions that characterise that way in which observation is conducted. These key four dimensions are control, whether the researcher is a member of the group under observation or not, structure, and concealment of observation (Sekaran and Bougie 2016). Under the control dimension, the observation can be either controlled or uncontrolled. Under the membership dimension, observation can be either participant or non-participant. Under structure, observation can be either structured or unstructured. Under concealment, observation can be either concealed or unconcealed.

Controlled observation is conducted in an artificial setting while uncontrolled observation is conducted in a natural setting (Sekaran and Bougie 2016). In the artificial setting, a researcher exposes the study participants to certain manipulated conditions in order to identify behaviour discrepancies given a natural setting and the artificial setting. For example, a researcher could subject some participants to autocratic leadership and observe their behaviour to establish the effect of autocratic leadership on job satisfaction. In participant observation, a researcher watches the situation or events from inside through taking part in the group to be observed. As such, the researcher freely interacts with the study participants and takes part in different group activities so that he/she studies the behaviour of the group as a member of that group. On the contrary, in non-participant observation a researcher studies the behaviour of a group from a distance without participating in any of the group activities (Saunders et al. 2015). Nonetheless,) note that purely non-participant observation is very difficult to attain in that a researcher cannot penetrate to the core of the study without interacting with the participants. In structured observation, a researcher specifies in good detail what will be observed and the measurements through which it will be recorded. Contrary, a researcher in unstructured observation monitors all aspects of behaviour that seem relevant to the research topic and research objectives (Jamshed 2014). Concealment relates to whether the study participants are told that they are being investigates or not. In concealed observation, the researcher does not tell the study participants that they are being observed while in unconcealed observation, the researcher notifies the study participants that they are under investigation (Sekaran and Bougie 2016). Concealed observation has been praised in that it helps a researcher conduct more accurate information in that the participants lack the awareness that they are being observed resulting to higher validity of the study results (Sekaran and Bougie 2016). On the contrary, unconcealed observation is more likely to upset the authenticity of the behaviour of the persons under study resulting in validity issues (Sekaran and Bougie 2016).

Of the key four dimensions distinguishing different types of observation, participant observation and structured observation have attracted a lot of attention thus considered the two most important approaches to observation. Participant observation is qualitative in nature and seeks to discover the meanings that people attach to their actions while structured observation is quantitative and seeks to establish the frequency of human actions (Sekaran and Bougie 2016). Saunders et al. (2015) identify four categories of roles that a participant observer can adopt namely complete participant, complete observer, observer as participant, and participant as observer (see figure 1).

The complete participant roles sees a researcher as one who is attempting to fully become a member of the group under research. However, the researcher does not reveal his/her true purpose to the study participants (Saunder et al. 2015). The complete observer role also does not allow a researcher to reveal the true purpose to the research group but unlike complete participant the researcher does not engage in the activities of the group (Saunder et al. 2015). The observer as participant role allows a researcher to attend the functions of the study participants just to observe without taking part in those activities. In the words of Jorgensen (2015), a researcher is like a spectator when the observer as participant role is adopted but his/her purpose remains known to all the persons of interest. In the participant as observer role, a researcher reveals his/her purpose as a researcher such the both the researcher and the research subjects are aware of the fieldwork nature of the relationship (Spradley 2016). Participant observation is unstructured and unsystematic but on the contrary, structured observation has a high degree of predetermined structure and is systematic (Saunder et al. 2015).

Ways of minimising observer bias

Observer bias is any type of systematic discrepancy from the truth during the process of collecting data through observation. According to Holman et al. (2015), observer bias is any type of detection bias and significantly affects observational studies. Observer bias considerably affects the validity of research findings which underscores the need for minimising observer bias. There are several ways of minimising observer bias, which this section of the paper focuses on.

First, observer bias can be minimised by ensuring that observer are well trained prior to conducting the study. According to Traniello and Bakker (2015), some researchers are not even aware of researcher bias, its loopholes and its effect of study results thus they unconsciously get trapped in. In the same vein, Jorgensen (2015) states that understanding methodological concerns of collecting and recording observational data is pivotal to interpreting the validity of the inferences and results drawn by the researcher. Ostrov and Hart (2014) state that training observational researchers lowers their susceptibility to researcher bias in that they consciously avoid bias loopholes. Likewise, Sedgwick (2015) notes that training helps a researcher to understand the methodological concerns involved in observational data recording is integral to promoting reliability of results drawn from observational studies. Training should also aim at helping observers understand how to record findings and identify potential conflicts before recordings starts while clearly defining the tools, methods, and time frames for collecting data (Sedgwick 2015). Training could also be used to help observers recognise their habits and prejudice in order to promote the accuracy of observational study results.

A second way of minimising observer bias is concealing contextual information through conducting blinded and double-blinded studies. By concealing contextual information, a researcher remains blind to the purpose and expected outcome, thus avoids recording and interpreting data in a way that supports his/her expectations (Holman et al. 2015). Similarly, Couret et al. (2016) write that when an observer is unaware of the context in which data is to be collected, he/she is more likely to remain objective when collecting data through observation, which improves the accuracy of data collecting through minimising human bias. On the other hand, double-blinded studies conceal treatment assignments for both the observer and the study participants which is more advocated for when the study participants are humans (Kahan et al. 2015). This means that both the researcher/observer and the study participants are not informed of the aim of the research as well as the expected outcome such that observer and population perceptions and anticipations do not affect the validity and reliability of the study findings.

A third way of minimising bias in observational studies is for researchers to analyse datasets independently and to obtain inter-observer reliability of behavioural coding. Analysing subsets or entire data sets independently ensures that subjectivity from one researcher is identifies by another and a more objective approach to the aspect adopted thus improving the validity and reliability of the results (Malone et al. 2014). Independent analysis also helps observational researchers to identify prejudice and habits of each other and correct them to enhance the reliability of the results. Obtaining inter-observer reliability helps individual researchers to recognise their weaknesses in data collection and analysis, which builds their expertise for future studies (Vet 2014).

Ethics of using concealed observation

Concealed observation as a method of data collection has been heavily criticised on ethical grounds and treated as one of the principal issues in some studies (Walters and Godbold 2014). Informed by research ethics, some sources advise students not to use concealed observation as a method of data collection:

“Some of the more famous participant observation studies (such as Whyte’s study of street corner gangs) collected rich data because they did not disclose to the observed that their behaviour was being researched. Nowadays, such studies are viewed as ethically untenable and we are no longer able to conduct such research” (Alston and Bowles 2019, p. 196).

The above case is an example that research is becoming increasingly regulated by ethical considerations and concealed observation is becoming increasingly problematic to plan and execute. Indeed, institutions of higher learning require students to submit a form of ethical considerations for approval prior to data collection for studies using primary studies. The ethical consideration form should include information on how informed consent will be sought, the purpose of the study as well as issues of briefing and debriefing, which shows that concealed observation is increasingly difficult to adopt as a data collection method in today’s research. Globally, covert research is seen as intrinsically illegitimate, which further hinders the adoption of concealed observation as a method of data collection. In his words, Roberts (2015) states that concealment of the purposes of a study form or covert observation of identifiable participants are no longer considered ethical as they ate contrary to the principle respecting persons given such as approach to research does not give study participants free and fully informed consent. Similarly, Calvey (2017) perceives concealed observation as a data collection method as deception since the researcher does not disclose the research to the study subjects with an aim of ensuring they do not change behaviour while under observation. As such, a researcher using the concealed observation data collection method collects data without the subject’s knowledge that the research is taking place.

Concealed observation as a data collection method is also associated with violation of subject’s privacy. According to Sekaran and Bougie (2016), a researcher using the concealed observation data collection approach could just attend an event like a trial or watch what people are doing and then publish this information when the study participants did not even know they were being investigated. A result of concealed observation could be for example wiring unauthorised biography which violates privacy in that the subject was not informed, data was collected without the consent of the subject and the data was published without the subject’s consent. From a different perspective, Spicker (2011) writes that concealed observation as a data collection method infringes moral rights of research participants. These moral rights are said to be the particular rights specific to individuals and general rights which apply to everyone. Particular rights depend on the relationship between a researcher and the research subjects and given that in concealed observation there is no such a relationship, this approach to research is morally wrong (Spicker 2011). Similarly, Johnson (2014) states that covert researchers lack integrity in that they do not consider the rights of the study subjects. For example, concealed observation researchers work with methods that limit the applicability of confidentiality thus are not considerate of the rights of their subjects (Johnson 2014). General rights on the other hand are held by all persons within a society and a researcher is primarily expected to uphold beneficence which means avoiding harm and respecting persons. Concealed observation as a data collection method hinders that ability of subjects to knowingly take part in the research thus liable to harm the subjects if they discovered they are being investigated or their information has been published without their knowledge and consent. Therefore, concealed observation as a data collection method is ethically wrong thus researchers should consider other dimensions of observational approaches such as unconcealed observation.

Task 2

Mechanisms of administering online questionnaires

Internet diffusion and technological development have significantly changes methods of data collection with electronic tools and the World Wide Web presenting new challenges for researchers and providing new ways for researchers to connect with people and collect information. One of the most important instruments of data collection in the digital era is online questionnaires. This section considers different mechanisms through which online questionnaires can be administered.

Online questionnaires are administered using different mechanisms including websites, emails, popups, and app mechanisms. Website questionnaires are directly inserted on the website of the researcher which is often the website of the organisation conducting the study (Saunders et al. 2015). Similarly, Minto et al. (2017) note that website questionnaires are designed as a web page with a URL on which one could click and follow to answer the questions. Online questionnaires administered via organisational websites are a common way through which organisations connect with and collect data from their customers. In email questionnaires, the researcher send the questionnaire as an email message. In this case, the researcher identifies the study sample, gathers their email addresses, and them emails the questionnaire for filling and resubmission (Sekaran and Bougie 2016). On the other hand, pop up questionnaires appear in a window on the computer screen (Saunders et al. 2015). When a researcher uses pop up questionnaires, he/she does not have to create an actual list of study subjects given that the computer software triggers an invitation to participate at random to any computer user. Further, app questionnaires are designed as a Smart phone application which can be downloaded and the questionnaire completed (Minto et al. 2017). Researchers also use personal computers to administer electronic questionnaires where a specific software program is designed specifically for the survey (Minto et al. 2017).

Online questionnaires are dominantly used by organisations seeking to gain a deeper understanding of consumer preferences and opinions. A core advantage of using online questionnaires is the ability to gain access to persons and groups it would be difficult to access using traditional data collection methods. According to Minto et al. (2017), collecting data using online questionnaires does not require a researcher to go through gatekeepers yet gatekeepers are known to limit the accessibility of key sources of information such as executive management. However, using online questionnaires such as emails, a research can directly contact an executive member of an organisation and collect rich content. A second advantage of online questionnaires is that they enable a researcher to collect data from a wide geographical area. Saunders et al. (2015) note that through online questionnaires, a researcher does not have to travel from one respondent to the others thus he/she is able to reach many respondents over wide geographical area. For example, using the website mechanism, a researcher can send the URL to various respondents who complete the questionnaire at their convenience (Sekaran and Bougie 2016). In addition, online questionnaires are cost efficient: the automatic processing saves the researcher time, energy, and other costs, making it cheaper to collect data using online questionnaires. In the same vein, Sekaran and Bougie (2016) state that online questionnaires offer several advantages such as cost reduction, faster collection and analysis of data, personalised design for different targets, flexibility, lack of influence of researcher’s presence, functionality, increased convenience given that respondents complete the questionnaires where and when they prefer, and usability.

Online questionnaires also have some disadvantages. For example, website and app questionnaires allow self-selection making it difficult for the researcher to establish control over who respondents to the questions (Sekaran and Bougie 2016). In addition, some forms on online questionnaires such as app questionnaires have very low response rate limiting the generalizability of the findings (Minto et al. 2017). In agreement, Saunders et al. (2015) state that the return rate for email questionnaires is typically low with an average of 30% response rate. Still, Kendall (2014) states that online questionnaires denies a researcher the ability to control the sampling process resulting in results that cannot be termed as representative. Further, online questionnaires are inherent of typical errors given their technological nature: emails can be read as spam messages while some designs can be incompatible with subjects’ devices (Minto et al. 2017).

Principles of wording and their importance is questionnaire design

Wording in questionnaires significantly affect the accuracy and reliability of the information collected. Sekaran and Bougie (2016) note that changing even a single word in a question can considerably alter response distribution rate and accuracy. According to Krosnick (2018) the BOSS (brief, objective, simple and specific) principle should be observed when wording every question. Krosnick (2018) argues that a brief question should not exceed 20 words and should not use more than three commas. In the questionnaire, a researcher should ask one question at a time thus avoiding hidden questions. In addition, the researcher should avoid leading questions in that they push the respondent towards a specific answer lowering the validity and reliability of the results (Krosnick 2018). Further, a researcher should avoid technical terms, slang, and jargon thus use direct, simple, and expressions that are familiar to respondents (Song et al. 2015). Moreover, a researcher should avoid biased questions; a question like what are the negative effects of TV on children hints that the researcher is negative on TV use by children thus the respondent is likely to respond in a way that promotes the bias. In such as case, an appropriate question would be what are your views about the effects of TV on children? Therefore, wording in questionnaires significantly affects the accuracy and reliability of the information collected.

Validity and reliability of mixed methods

Mixed methods combine different data collection methods in the same study. For example, a researcher could use questionnaires, interviews and observation in the same study. Research shows using mixed methods has a significant effect on validity and reliability of the results. Saunders et al. (2015) define reliability as the extent to which data collection and analysis methods will provide consistent results while validity relates to whether the results are really about what they appear to be. By using mixed methods, a researcher is able to overcome subject error and subject bias thus promoting the reliability of the findings (Green et al. 2015). Similarly, Archibald (2016) argues that mixed methods eliminate observer bias and observe error making the findings more reliable. Through mixed methods, a researcher is able to collect data from more subjects which promotes external validity (Sekaran and Bougie 2016). Similarly, mixed methods allows a researcher different levels of control over sampling and data collecting, which promotes validity of the results. Further, mixed methods allow a researcher to confirm the results with different data collection methods which promotes validity. Therefore, mixed methods of data collection are said to enhance validity and reliability of research findings.

A critical analysis of the statement that every data collection method has its own built-in biases hence mixed methods compounds the biases

This part is written against the above mentioned statement and argues that mixed methods are essential in eliminating biases in singular data gathering methods resulting in unbiased findings. According to Archibald (2016), mixed methods of data collection enables confirmation of gathered data, initiates new lines of thinking, and provides richer details thus challenging and eliminating biases in individual data collection methods. On the other hand, Saunders et al. (2015) write that mixed methods allow a researcher to employ different safeguards into his/her data collection instruments thus minimising bias and other sources of invalidity that could possibly exist in every single data collection method. Similarly, Sekaran and Bougie (2016) note that mixed methods allow corroboration, expansion and complementarity of findings gathered using different data collection instruments thus eliminating bias with singular methods. In the same vein, Pluye and Hong (2014) write that researchers who use mixed methods research are more likely to select approaches and methods that better answer their research questions, rather than methods that relate to their preconceived biases about the outcome resulting in more valid and reliable results. In fact, Zou et al. (2014) see mixed methods as a technique of bridging the schism between different data collection methods resulting in higher validity. Therefore, using mixed methods is the only way a researcher can be able to overcome the limitations of singular data collection methods which enhances the validity and reliability of the findings.

Conclusion

The aim of this portfolio was to discuss the concepts of observation, questionnaire, and mixed methods of data collection while considering their advantages and disadvantages. The paper establishes that observation is a natural way of collecting data but some dimensions of observation such as concealed observation are faced with serious ethical issues limiting their applicability. The paper also establishes that the internet has significantly influenced data collection methods resulting to the rise of online questionnaires which can be administered using emails, websites, apps, and pop ups. Online questionnaires are seen to have several advantages such as cost reduction, faster collection and analysis of data, personalised design for different targets, flexibility, lack of influence of researcher’s presence, functionality, increased convenience given that respondents complete the questionnaires where and when they prefer, and usability. Nonetheless, the paper establishes that online questionnaires have extremely low response rates and denies a researcher control over sampling which lowers generalizability of the findings. Finally, the paper establishes that mixed methods are a way of increasing validity and reliability of research findings.

References

Alston, M. and Bowles, W., 2019. Research for social workers: An introduction to methods. Routledge.

Archibald, M.M., 2016. Investigator triangulation: A collaborative strategy with potential for mixed methods research. Journal of Mixed Methods Research, 10(3), pp.228-250.

Calvey, D., 2017. Covert research: The art, politics and ethics of undercover fieldwork. Sage.

Couret, D., Boumaza, D., Grisotto, C., Triglia, T., Pellegrini, L., Ocquidant, P., Bruder, N.J. and Velly, L.J., 2016. Reliability of standard pupillometry practice in neurocritical care: an observational, double-blinded study. Critical Care, 20(1), p.99.

Green, C.A., Duan, N., Gibbons, R.D., Hoagwood, K.E., Palinkas, L.A. and Wisdom, J.P., 2015. Approaches to mixed methods dissemination and implementation research: methods, strengths, caveats, and opportunities. Administration and Policy in Mental Health and Mental Health Services Research, 42(5), pp.508-523.

Holman, L., Head, M.L., Lanfear, R. and Jennions, M.D., 2015. Evidence of experimental bias in the life sciences: why we need blind data recording. PLoS biology, 13(7), p.e1002190.

Jamshed, S., 2014. Qualitative research method-interviewing and observation. Journal of basic and clinical pharmacy, 5(4), p.87.

Johnson, B., 2014. Ethical issues in shadowing research. Qualitative Research in Organizations and Management: An International Journal, 9(1), pp.21-40.

Jorgensen, D.L., 2015. Participant observation. Emerging trends in the social and behavioral sciences: An interdisciplinary, searchable, and linkable resource, pp.1-15.

Kahan, B.C., Rehal, S. and Cro, S., 2015. Risk of selection bias in randomised trials. Trials, 16(1), p.405.

Kendall, L., 2014. The conduct of qualitative interviews: Research questions, methodological issues, and researching online. In Handbook of research on new literacies (pp. 151-168). Routledge.

Khan, S.N., 2014. Qualitative research method: Grounded theory. International Journal of Business and Management, 9(11), pp.224-233.

Krosnick, J.A., 2018. Questionnaire design. In The Palgrave handbook of survey research (pp. 439-455). Palgrave Macmillan, Cham.

Malone, H., Nicholl, H. and Tracey, C., 2014. Awareness and minimisation of systematic bias in research. British Journal of Nursing, 23(5), pp.279-282.

Minto, C., Vriz, G.B., Martinato, M. and Gregori, D., 2017. Suppl-1, M3: Electronic Questionnaires Design and Implementation. The open nursing journal, 11, p.157.

Oh, J.H., 2014. Research methods.

Ostrov, J.M. and Hart, E., 2014. Observational methods. TD Little, pp.286-304.

Pluye, P. and Hong, Q.N., 2014. Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annual review of public health, 35, pp.29-45.

Quinlan, C., Babin, B., Carr, J. and Griffin, M., 2019. Business research methods. South Western Cengage.

Roberts, L.D., 2015. Ethical issues in conducting qualitative research in online communities. Qualitative Research in Psychology, 12(3), pp.314-325.

Saunders, M.N., Lewis, P. and Thornhill, A., 2015. Research Methods for Business Students EBook. Pearson Australia Pty Limited.

Sedgwick, P., 2014. Bias in observational study designs: prospective cohort studies. Bmj, 349, p.g7731.

Sekaran, U. and Bougie, R., 2016. Research methods for business: A skill building approach. John Wiley & Sons.

Song, Y., Son, Y.J. and Oh, D., 2015. Methodological issues in questionnaire design. Journal of Korean Academy of Nursing, 45(3), pp.323-328.

Spicker, P., 2011. Ethical covert research. Sociology, 45(1), pp.118-133.

Spradley, J.P., 2016. Participant observation. Waveland Press.

Suen, H.K. and Ary, D., 2014. Analyzing quantitative behavioral observation data. psychology press.

Sutton, J. and Austin, Z., 2015. Qualitative research: Data collection, analysis, and management. The Canadian journal of hospital pharmacy, 68(3), p.226.

Traniello, J.F. and Bakker, T.C., 2015. Minimizing observer bias in behavioral research: blinded methods reporting requirements for Behavioral Ecology and Sociobiology.

Vet, H., 2014. Observer reliability and agreement. Wiley StatsRef: Statistics Reference Online.

Walters, S.R. and Godbold, R., 2014. Someone is watching you: The ethics of covert observation to explore adult behaviour at children’s sporting events. Journal of bioethical inquiry, 11(4), pp.531-537.

Zou, P.X., Sunindijo, R.Y. and Dainty, A.R., 2014. A mixed methods research design for bridging the gap between research and practice in construction safety. Safety science, 70, pp.316-326.

Dig deeper into Focus on the most important variables of interest with our selection of articles.