{"id":8194,"date":"2017-11-07T10:14:12","date_gmt":"2017-11-07T09:14:12","guid":{"rendered":"https:\/\/www.munich-business-school.de\/insights\/?p=8194"},"modified":"2023-06-28T12:45:51","modified_gmt":"2023-06-28T10:45:51","slug":"trustworthiness-of-research","status":"publish","type":"post","link":"https:\/\/www.munich-business-school.de\/insights\/en\/2017\/trustworthiness-of-research\/","title":{"rendered":"Trustworthiness of Research"},"content":{"rendered":"\n

Trustworthiness is the highest premise of good research. But when is a paper considered trustworthy? Quantitative and qualitative research have different criteria for trustworthiness – we explain them to you<\/strong>.<\/p>\n\n\n\n

\n\n\n\n

\u201eA man is more likely to believe something if he would like it to be true.\u201c<\/em><\/p>\n\n\n\n

Francis Bacon <\/em><\/p>\n\n\n\n

Quantitative and qualitative research<\/a> is based on different research paradigms that reflect the researcher\u2019s scientific worldview. Due to this, methodology, methods and results naturally differ accordingly (Goertz & Mahoney, 2012). In order to appreciate the quality of academic research, it makes sense to apply different criteria for (1) quantitative and (2) qualitative methods.<\/p>\n\n\n\n

1. Trustworthiness of quantitative research<\/h2>\n\n\n\n

Bryman and Bell (2005, p. 154) define quantitative research as \u201eentailing the collection of numerical data and exhibiting the view of relationship between theory and research as deductive, a predilection for natural science approach, and as having an objectivist conception of social reality\u201c. Data is usually generated by means of surveys and experiments and analyzed through statistical tests such as ANOVA (e.g. Black, 1999) and Cohen\u2019s Kappa (Cohen, 1980).<\/p>\n\n\n\n

Therefore, its quality can be assessed by its (a)<\/strong> internal and (b) external validity, (c) reliability and (d) objectivity<\/strong> (Lincoln & Guba, 1985), which are described below:<\/p>\n\n\n\n

a. Internal validity<\/h3>\n\n\n\n

A study is internally valid if it is able to determine whether a causal relationship exists between one or more independent variables and one or more dependent variables (Heffner, 2017), i.e. it is explanative. A study is internally valid if there are as little confounding variables as possible. Confounding variables are variables that the researcher fails to control or eliminate, allowing the results to show false correlation (Shuttleworth, 2008).<\/p>\n\n\n\n

Therefore, on the one hand, internal validity refers to how well the study is run, e.g. research design, operational definitions, how variables are measured, and what is (not) measured (Huitt, Hummel & Kaeck, 1999). On the other hand, internal validity determines how confidently it can be concluded that the change in the dependent variable was produced solely by the independent variable as opposed to extraneous ones (ibid.).<\/p>\n\n\n\n

Campbell and Stanley\u2018s (1966) seminal work on experimental and quasi-experimental designs identifies and describes eight threats to internal validity:<\/p>\n\n\n\n

    \n
  1. History<\/strong> \u2013 Studies that collect data over long time periods are likely to be affected by research subjects\u2019 unique experiences over time that function like extra and unplanned, therefore independent variables.<\/li>\n\n\n\n
  2. Maturation<\/strong> \u2013 Like the above, this effect also draws on the normal passage of time. Research subjects may become more or less motivated and thus affecting the internal validity of the study.<\/li>\n\n\n\n
  3. Testing<\/strong> \u2013 Research design often use pre-tests. These may change, i.e. contaminate the outcome of the actual study.<\/li>\n\n\n\n
  4. Instrumentation<\/strong> \u2013 Changing measurement methods or their administration may affect what is measured.<\/li>\n\n\n\n
  5. Statistical Regression <\/strong>\u2013 Re-testing research subjects that were originally chosen because of extremely high or low scores can be expected to produce a distribution closer to the entire population\u2019s distribution.<\/li>\n\n\n\n
  6. Selection<\/strong> \u2013 The results of the study will be biased if the research subjects in the control and experimental groups are different from another at the beginning of the study.<\/li>\n\n\n\n
  7. Experimental Mortality <\/strong>\u2013 If the comparison groups experience different or high levels of withdrawal (mortality) of research subjects, it becomes questionable whether the observed differences between the groups are due to different or high drop-out rates or are produced by the independent variable.<\/li>\n\n\n\n
  8. Selection Interactions <\/strong>\u2013 If the selection method of research subjects interacts with one or more of the above threats, the study\u2019s results will be biased.<\/li>\n<\/ol>\n\n\n\n

    b. External validity<\/h3>\n\n\n\n

    External validity describes the ability to generalize a study, which is particularly threatened if people, places, or times are poorly chosen (Trochim, 2006).<\/p>\n\n\n\n

    As measuring an entire population is impossible, a sample, or a subset of the population is studied. The sample chosen needs to represent the whole population in order to allow inferences to be drawn (Landreneau, 2009).<\/p>\n\n\n\n

    A good sampling model firstly identifies the population that it should generalize, subsequently drawing a sample from that population, conducting research and finally generalizing the results back to the original population (Trochim, 2006). External validity will improve the more a study is replicated (ibid.).<\/p>\n\n\n\n

    The sufficient sample size depends on the minimum number of participants required to identify a statistically significant difference and increases the smaller the anticipated effect is (Burmeister & Aitken, 2012).<\/p>\n\n\n\n

    As quantitative methods are often used in natural science, the very fine tools and criteria developed for that realm may be adapted and used for business research.<\/p>\n\n\n\n

    As an example, a set of eight criteria was developed to identify high quality evidence in the public health sector (Effective Public Health Practice Project, 1998). Each of the criteria (selection bias, study design, confounders, blinding, data collection methods, withdrawals and dropouts, intervention integrity, analysis) is rated as strong, moderate, or weak, thus achieving an overall methodological rating (ibid.).<\/p>\n\n\n\n

    c. Reliability<\/h3>\n\n\n\n

    In research, reliability means „repeatability“ or „consistency“; it is achieved if a measure will always provide the same result (Trochim, 2006). Reliability issues often come up when researchers adopt a subjective approach to research (Wilson, 2010), which on the other hand is consciously allowed in qualitative research.<\/p>\n\n\n\n

    d. Objectivity<\/h3>\n\n\n\n

    Objectivity demands that \u201eresearchers should remain distanced from what they study so findings depend on the nature of what was studied rather than on the personality, beliefs, and values of the researcher\u201c (Payne & Payne, 2004).<\/p>\n\n\n\n

    The four criteria for trustworthiness of quantitative research overlap at some points, for example validity and reliability: if a study\u2019s results can be generalized, its repetition should offer the same results.<\/p>\n\n\n\n

    2. Trustworthiness of qualitative research<\/h2>\n\n\n\n

    The above described criteria applying for quantitative research are not suitable for qualitative research as it accepts multiple and subjective realities and aims at deep insights. Therefore, these criteria are not sought to be complied with. In order to provide a different set for criteria that can be used for ascertaining the quality, Lincoln & Guba (1985) created a corresponding set of criteria for trustworthiness of qualitative research: (a) credibility<\/strong> (vs. internal validity), (b) transferability<\/strong> (vs. external validity), (c) dependability<\/strong> (vs. reliability) and (d) confirmability<\/strong> (vs. objectivity).<\/p>\n\n\n\n

    a. Credibility<\/h3>\n\n\n\n

    Credibility depends on the richness of the data and analysis and can be enhanced by triangulation (Patton, 2002), rather than relying on sample size aiming at representing a population.<\/p>\n\n\n\n

    There are four types of triangulation as introduced by Denzin (1970), which can also be used in conjunction with each other:<\/p>\n\n\n\n

      \n
    1. Data triangulation<\/strong> \u2013 using different sources of data, e.g. from existing research<\/li>\n\n\n\n
    2. Methodological triangulation<\/strong> \u2013 using more than one method, e.g. mixed methods approach, however with focus on qualitative methods<\/li>\n\n\n\n
    3. Investigator triangulation<\/strong> \u2013 using more than one researcher adds to the credibility of a study in order to mitigate the researcher\u2019s influence<\/li>\n\n\n\n
    4. Theoretical triangulation<\/strong> \u2013 using more than one theory as conceptual framework<\/li>\n<\/ol>\n\n\n\n

      b. Transferability<\/h3>\n\n\n\n

      Transferability corresponds to external validity, i.e. generalizing a study\u2019s results. Transferability can be achieved by thorough description of the research context and underlying assumptions (Trochim, 2006). With providing that information, the research results may be transferred from the original research situation to a similar situation.<\/p>\n\n\n\n

      c. Dependability<\/h3>\n\n\n\n

      Dependability aims to replace reliability, which requires that when replicating experiments, the same results should be achieved. As this would not be expected to happen in a qualitative setting, alternative criteria are general understandability, flow of arguments, and logic. Both the process and the product of the research need to be consistent (Lincoln & Guba, 1985).<\/p>\n\n\n\n

      d. Confirmability<\/h3>\n\n\n\n

      Instead of general objectivity in quantitative research, the researcher\u2019s neutrality of research interpretations is required. This can be achieved by means of a confirmability audit that includes an audit trail of raw data, analysis notes, reconstruction, and synthesis products, process notes, personal notes, as well as preliminary developmental information (Lincoln & Guba, 1985).<\/p>\n\n\n\n

      The approach to sampling differs significantly in quantitative and qualitative research. Qualitative samples are usually small and should be selected purposefully in order to select information-rich cases for in-depth study (Patton, 2002). There may be as few as five (Creswell, 1998, p. 64) or six participants (Morse, 1994, p. 225).<\/p>\n\n\n\n

      As seen from the above criteria, qualitative research requires far more documentation than quantitative research in order to establish trustworthiness in. Quantitative research, on the other hand, requires more effort during the research design phase.<\/p>\n\n\n\n

      With qualitative and quantitative research serving different objectives and being designed in a different way, quality assessment criteria must be adapted and adhered to accordingly.<\/p>\n\n\n\n

      Literature<\/strong>:<\/p>\n\n\n\n