Везде
RELIABILITY AND VALIDITY IN TEXTS' CONTENT-ANALYSIS: THE CHOICE OF MEASURES
Table of contents
Annotation Estimate Publication content
References Comments
Share
QR
Metrics
RELIABILITY AND VALIDITY IN TEXTS' CONTENT-ANALYSIS: THE CHOICE OF MEASURES
Annotation
PII
S0205-95920000596-2-1
Publication type
Article
Status
Published
Authors
Tatiana Shatalova  Svetlana G. Kirdina-Chandler
ORCID: 0000-0002-9234-8308
I. Popova Anton N. Oleinik
Abstract
The paper discusses several reliability measures: Scott's pi, Krippendorff's alpha, free marginal adjustment (Bennett, Alpert and Goldstein's S), Cohen's kappa, and Perreault and Leigh's I and the assumptions on which they are based. It is suggested that correlation coefficients between, on one hand, the distribution of qualitative codes and, on the other hand, word co-occurrences and the distribution of the categories identified with the help of the dictionary based on substitution complement the other reliability measures. The paper shows that the choice of the reliability measure depends on the format of the text (stylistic versus rhetorical) and the type of reading (comprehension versus interpretation). Namely, Cohen's kappa and Bennett, Alpert and Goldstein's S emerge as reliability measures particularly suited for perspectival reading of rhetorical texts. Outcomes of the content analysis of 57 texts performed by four coders with the help of computer program QDA Miner inform the analysis.
Keywords
Content-analysis, correlation analysis, interpretation, stylistic texts, rhetorical texts, psychology of reading, reliability measures
Date of publication
03.11.2014
Number of purchasers
1
Views
681
Readers community rating
0.0 (0 votes)
Cite   Download pdf

References



Additional sources and materials




    

  1. Abramenkova V.V. Otrazhenie kartiny mira sovremennykh rossijskikh detej v graficheskikh i verbal'nykh obrazakh//Voprosy psikhologii. 2007. № 6. S. 54-63.
    2. 

  2. Bushev A.B., Aleksandrova S.K. Tekhniki interpretatsii teksta kak sposobnosti razvitoj yazykovoj lichnosti//Gumanitarnyj vektor. 2011. № 4. S. 39-51.
    3. 

  3. Val'dgard S.L. Ocherki psikhologii chteniya. SPb.: Rossijskaya natsional'naya biblioteka, 2010 .
    4. 

  4. Volodarskaya E.A., Logvinova N.Yu. “Roditel'skaya” i “studencheskaya” modeli predstavlenij o semejnom vospitanii//Psikhologicheskij zhurnal. 2005. T. 26. № 5. S. 26-34.
    5. 

  5. Volovikova M.I., Sosnina L.M. Ehtnokul'turnoe issledovanie predstavlenij o spravedlivosti (na primere moldavan i zhivuschikh v Moldove tsygan)//Voprosy psikhologii. 2001. № 2. S. 85-93.
    6. 

  6. Gulevich O.A. Sotsial'nye predstavleniya o prestupleniyakh v mezhlichnostnoj i massovoj kommunikatsii//Voprosy psikhologii. 2001. № 4. S. 53-67.
    7. 

  7. Kirdina S.G., Shatalova T.Yu. Vozrastayuschaya otdacha v sovremennoj ehkonomicheskoj literature: kontent-analiz rossijskikh i zarubezhnykh istochnikov/Fenomen vozrastayuschej otdachi v ehkonomike i politike. SPb.: Aletejya, 2013. S. 18-54.
    8. 

  8. Kondakov I.M., Ushnev S.V. Ehksperimental'noe issledovanie implitsitnykh teorij chteniya//Voprosy psikhologii. 1994.№ 6. S. 110-117.
    9. 

  9. Lysenko N.E., Davydov D.M. Otsenka tekstovykh opisanij stsen nasiliya v zavisimosti ot psikhotizma i polovykh razlichij//Psikhologicheskij zhurnal. 2011. T. 32. № 3. S. 114-127.
    10. 

  10. Mironova M.V. Psikhologiya i sotsiologiya chteniya: Uchebnoe posobie. Ul'yanovsk, 2003.
    11. 

  11. Olejnik A.N., Kirdina S.G., Popova I.P., Shatalova T.V. Kak uchenye chitayut drug druga: osnovy teorii akademicheskogo chteniya i ee ehmpiricheskaya proverka//Sotsiologicheskie issledovaniya. 2013. № 8. S. 30-41.
    12. 

  12. Pavlova N.D. Novye napravleniya issledovanij v psikhologii rechi i psikholingvistike//Psikhologicheskij zhurnal. 2007. T. 28 № 2. S. 19-30.
    13. 

  13. Prantsova G.V., Romanicheva E.S. Sovremennye strategii chteniya i ponimaniya teksta. M.: Forum, 2013.
    14. 

  14. Puz'ko V.I. Sposoby ponimaniya i interpretatsii teksta/Uchebnoe posobie. Vladivostok, 2010.
    15. 

  15. Rubakin N.A. Bibliologicheskaya psikhologiya i literaturovedenie. Doklad obschestvu lyubitelej rossijskoj slovesnosti, chitannyj 11 maya 1927 g. v Moskve//Voprosy psikholingvistiki. 2007. № 6. S.189-206.
    16. 

  16. Rubo I.G. Psikhologicheskij analiz strategij chteniya nauchnogo teksta: Diss.. kand. psikhol. nauk. M., 1988.
    17. 

  17. Stefanovskaya N.A. Metodologicheskie problemy ehmpiricheskikh sotsiologicheskikh issledovanij chteniya//Analitika kul'turologii. 2007. № 7. S. 280-288.
    18. 

  18. Stefanovskaya N.A. Ehkzistentsial'nye osnovy chteniya. Tambov: Izdat. dom TGU im. G.R. Derzhavina, 2008.
    19. 

  19. Tarakanov A.V. Psikhologiya i sotsiologiya chteniya/Uchebnoe posobie. Novosibirsk, 2011.
    20. 

  20. Ushakova T.N., Belova S.S., Valueva E.A. Lingvopsikhologicheskoe issledovanie verbal'noj semantiki//Psikhologicheskij zhurnal. 2010. T. 31. № 6. S. 83-97.
    21. 

  21. Allport G.W. The use of personal documents in psychological science. New York: Social Science Research Council, 1942.
    22. 

  22. Arrow K.J. A Difficulty in the Concept of Social Welfare//The Journal of Political Economy. 1950. 58. Is. 4. P. 328-346.
    23. 

  23. Artstein R., Poesio M. Inter-Coder Agreement for Computational Linguistic//Computational linguistic. 2008. V. 34. Is. 4. P. 555-596.
    24. 

  24. Baldwin A.L. Personal structure analysis: A statistical method for investigating the single personality//Journal of Abnormal and Social Psychology. 1942. № 37. P. 163-183.
    25. 

  25. Bales R.F. Interaction process analysis. Reading, MA: Addison-Wesley, 1950.
    26. 

  26. Bennett E., Alpert R., Goldstein A.C. Communications through Limited-Response Questioning//Public Opinion Quarterly. 1954. V. 18. Is. 3. P. 303-308.
    27. 

  27. Bryman A., Bell E., Teevan J.J. Social Research Methods. 3rd edition. Oxford University Press, Don Mills, 2012.
    28. 

  28. Dijkstra L., Eijnatten F.M. van. Agreement and Consensus in a Q-mode Research Design: an Empirical Comparison of Measures, and an Application//Quality & Quantity. 2009. V. 43 Is. 5. P. 757-771.
    29. 

  29. Hayes A.F., Krippendorff K. Answering the Call for a Standard Reliability Measure for Coding Data//Communication Methods and Measures. 2007. № 1 (1). P. 77-89.
    30. 

  30. Krippendorff K. Content Analysis: An Introduction to Its Methodology. SAGE, Thousand Oaks, 2004.
    31. 

  31. Krippendorff K. Measuring the Reliability of Qualitative Text Analysis Data//Quality & Quantity. 2004. № 38 (6). P. 787-800.
    32. 

  32. Leiva F.M., Rios F.J.M., Martinez T.L. Assessment of Interjudge Reliability in the Open-Ended Questions Coding Process//Quality & Quantity. 2006. № 40 (4). P. 519-537.
    33. 

  33. Lotman Y. Universe of the Mind: A Semiotic Theory of Culture. Indiana University Press, Bloomington, 1990.
    34. 

  34. Neuendorf K.A. The Content Analysis Guidebook. SAGE, Thousand Oaks, 2002.
    35. 

  35. Norris S.P., Philips L.M. The Relevance of a Reader's Knowledge within a Perspectival View of Reading//Journal of Reading Behavior. 1994. № 26 (4). P. 391412.
    36. 

  36. Oleinik A. Mixing Quantitative and Qualitative Content Analysis: Triangulation at Work//Quality & Quantity. 2010. № 45 (4). P. 859-873.
    37. 

  37. Oleinik A., Popova I., Kirdina S., Shatalova T. On the choice of measures of reliability and validity in the content-analysis of texts//Quality & Quantity. 2014. № 48.R 2703-2718.
    38. 

  38. Osgood C.E. Probing subjective culture: Part 1. Cross-linguistic tool-making//Journal of Communication. 1974. № 24 (1). P. 21-35.
    39. 

  39. Osgood C.E. Probing subjective culture: Part 2. Cross-cultural tool-using//Journal of Communication. 1974. № 24 (2). P. 82-100.
    40. 

  40. Perreault W.D., Leigh L.E. Reliability of Nominal Data Based on Qualitative Judgments//Journal of Marketing Research. 1989. № 26 (2). P. 135-148.
    41. 

  41. Salton G., McGill M. Introduction to Modern Information Retrieval. McGraw-Hill Book Co., New York, 1983.
    42. 

  42. Scott W.A. Reliability of Content Analysis: The Case of Nominal Scale Coding//Public Opinion Quarterly. 1955. № 19 (3). P. 321-325.
    43. 

  43. Siegel S., Castellan N.J. Nonparametric Statistics for the Behavioural Sciences. McGraw Hill, New York, 1988.
    44. 

  44. Skinner Q. Visions of Politics. Cambridge University Press, Cambridge, 2002.
    45. 

  45. Warner R.M. Applied Statistics. SAGE, Thousand Oaks, 2008.
    46. 

  46. Weller S.C. Cultural Consensus Theory: Applications and Frequently Asked Questions//Field Methods. 2007. № 19 (4). P. 339-368.
    47. 

  47. White R.K. Black Boy: A value analysis//Journal of Abnormal and Social Psychology. 1947. № 42. P. 440-461.
    48.

Comments

No posts found

Write a review
Translate