Sociologický časopis / Czech Sociological Review 2016, 52(1): 79-106 | DOI: 10.13060/00380288.2016.52.1.243

Estimating the Reliability of Selected Items in Czech EU-SILC Data Using the Quasi Simplex Model Method

Johana Chylíková
Sociologický ústav AV ČR, v.v.i., Praha

The article presents estimates of the reliability of measurement in the Czech surveys carried out in the EU-SILC international longitudinal research project. The reliability estimates were obtained using the Quasi Simplex Model (QSM), which has never before been used in Czech research. An analysis was carried out on all the items in the EU-SILC questionnaire that fulfilled the criteria for the QSM analysis: PH010, the item that asks respondents about their subjective health, HS120, the item that asks about the household's financial situation, and HS130, which asks what the minimum sufficient income of a household is. The analysis drew on all available data from Czech EU-SILC surveys, that is, data from five rotating panel surveys carried out between 2005 and 2012. The QSM analysis showed that for the selected items EU-SILC data are highly reliable; the estimated reliability of each item was around 0.8, for HS130 it was even above 0.9. The steadiness of the results was confirmed by the high consistency of the reliability estimates across all the panels. A small difference was observed between the reliability of data collected using the PAPI mode and data collected using CAPI. Given the attributes of the QSM model, however, it was impossible to test statistically whether the reliability of PAPI and CAPI data differ significantly.

Keywords: quantitative data, reliability, structural equation modelling, Quasi simplex model, EU-SILC

Published: February 1, 2016Show citation

ACS AIP APA ASA Harvard Chicago IEEE ISO690 MLA NLM Turabian Vancouver
Chylíková, J. (2016). Estimating the Reliability of Selected Items in Czech EU-SILC Data Using the Quasi Simplex Model Method. Sociologický časopis / Czech Sociological Review, 52(1), 79-106. doi: 10.13060/00380288.2016.52.1.243.
Download citation
  • BibTeX (.bib)
  • Bookends (.ris)
  • EasyBib (.ris)
  • EndNote (.enw)
  • EndNote 8 (.xml)
  • ISI WoS (.isi)
  • Medlars (.medlars)
  • Mendeley (.ris)
  • MODS (.xml)
  • Papers (.ris)
  • RefWorks (.txt)
  • RefManager (.ris)
  • RIS (.ris)
  • MS Word (.xml)
  • Zotero (.ris)
    • Open full article

    References

    1. Achen, Ch. H. 1975. "Mass Political Attitudes and the Survey Response." American Political Science Review 69 (4): 1218-1231, http://dx.doi.org/10.2307/1955282. Go to original source...
    2. Alwin, D. F. 2007. Margins of Error. A Study of Reliability in Survey Measurement. New Jersey: Wiley. Go to original source...
    3. Alwin, D. F., J. A. Krosnick. 1991. "The Reliability of Survey Attitude Measurement. The Influence of Question and Response Attributes." Sociological Methods and Research 20 (1): 139-181, http://dx.doi.org/10.1177/0049124191020001005. Go to original source...
    4. Andrews, F. 1984. "Construct Validity and Error Components of Survey Measures: A Structural Modeling Approach." Public Opinion Quarterly 46: 409-442, http://dx.doi.org/10.1086/268840. Go to original source...
    5. Biemer, P. B., S. L. Christ, Ch. A. Wiesen. 2009. "A General Approach for Estimating Scale Score Reliability for Panel Survey Data." Psychological Methods 14 (4): 400-412, http://dx.doi.org/10.1037/a0016618. Go to original source...
    6. Blok, H., W. E. Saris. 1983. "Using Longitudinal Data to Estimate Reliability." Applied Psychological Measurement 7 (3): 295-301, http://dx.doi.org/10.1177/014662168300700305. Go to original source...
    7. Bollen, K. A. 1989. Structural Equations with Latent Variables. New York: Wiley. Go to original source...
    8. Byrne, B. 2009. Structural Equation Modeling With AMOS: Basic Concepts, Applications, and Programming. New Jersey.
    9. Cernat, A., P. Lugtig, S. C. N. Uhrig, N. Watson. 2014. "Assessing and Relaxing Assumptions in Quasi-simplex Models." ISER Working Paper Series.
    10. de Leeuw, E. D., J. Hox, G. Snijkers. 1998. "The Effect of Computer-Assisted Interviewing on Data Quality. A Review." Pp. 173-198 in B. Blyth. Market Research and Information Technology. ESOMAR Monogaph.
    11. Dillman, D. A., L. M. Christian. 2005. "Survey Mode as a Source of Instability in Responses across Surveys." Field Methods 17 (1): 30-52, http://dx.doi.org/10.1177/1525822X04269550. Go to original source...
    12. Ehrhardt, J. J., W. E. Saris, R. Veenhoven. 2000. "Stability of Life-Satisfaction Over Time. Analysis of Change in Ranks in a National Population." Journal of Happiness Studies 1: 177-205, http://dx.doi.org/10.1023/A:1010084410679. Go to original source...
    13. Erikson, R. S. 1979. "The SRC Panel Data and Mass Political Attitudes." British Journal of Political Science 9 (1): 89-114, http://dx.doi.org/10.1017/S0007123400001630. Go to original source...
    14. Green, D. P., B. Palmquist. 1990. "Of Artifacts and Partisan Instability." American Journal of Political Science 34 (3): 872-902, http://dx.doi.org/10.2307/2111402. Go to original source...
    15. Groves, R. M. 2004. Survey Errors and Survey Costs. Chichester: John Wiley and Sons.
    16. Heise, D. R. 1969. "Separating Reliability and Stability in Test-retest Correlation." American Sociological Review 34: 93-101, http://dx.doi.org/10.2307/2092790. Go to original source...
    17. Hooper, D., J. Coughlan, M. Mullen. 2008. "Structural Equation Modelling: Guidelines for Determining Model Fit." Electronic Journal of Business Research Methods 6 (1): 53-60.
    18. Jagodzinski, W., S. M. Kuhnel. 1987. "Estimation of Reliability and Stability in SingleIndicator Multiple-wave Models." Sociological Methods and Research 15 (3): 219-258, http://dx.doi.org/10.1177/0049124187015003003. Go to original source...
    19. Jöreskog, K. G., D. Sörbom. 1996. "Structural Equation Modeling." Workshop presented for the NORC Social Science Research Professional Development Trainig Sessions, Chicago.
    20. Jöreskog, K. G., D. Sörbom. 2004. LISREL 8.7 for Windows [počítačový software]. Skokie, IL: Scientific Software International, Inc.
    21. Judd, Ch. M., M. A. Milburn. 1980. "The Structure of Attitude Systems in the General Public: Comparisons of a Structural Equation Model." American Sociological Review 45 (4): 627-643, http://dx.doi.org/10.2307/2095012. Go to original source...
    22. Kenny, D. A., D. T. Campbell. 1989. "On the Measurement of Stabiliy in Over-time Data." Journal of Personality 57 (2), http://dx.doi.org/10.1111/j.1467-6494.1989.tb00489.x. Go to original source...
    23. Kroh, M. 2006. An Experimental Evaluation of Popular Well-Being Measures. DIWDiskussionspapiere, No. 546.
    24. Lord, F. M., M. R. Novick. 1968. Statistical Theories of Mental Test Scores. Reading MA: Addison-Welsley Publishing Company.
    25. Lucas, R. E., B. Donnellan. 2011. "Personality Development Across the Life Span: Longitudinal Analyses with a National Sample from Germany." Journal of Personality and Social Psychology 101: 847-861, http://dx.doi.org/10.1037/a0024298. Go to original source...
    26. Mysíková, M. 2011. "EU-SILC a jeho metodologická úskalí: mezinárodní srovnatelnost a příjmové proměnné." Data a výzkum - SDA Info 5 (2): 147-170.
    27. Peter, J. P. 1979. "Reliability: A Review of Psychometric Basics and Recent Marketing Practices." Journal of Marketing Research 16: 6-17, http://dx.doi.org/10.2307/3150868. Go to original source...
    28. Revilla, M., W. E. Saris. 2013a. "The Split-Ballot Multitrait-Multimethod Approach: Implementation and Problems." Structural Equation Modeling 20: 27-46, http://dx.doi.org/10.1080/10705511.2013.742379. Go to original source...
    29. Revilla, M. A., W. E. Saris. 2013b. "A Comparison of the Quality of Questions in a Faceto-face and a Web Survey." International Journal of Public Opinion Research 25 (2): 242-253, http://dx.doi.org/10.1093/ijpor/eds007. Go to original source...
    30. Řehák, J. 1998. "Kvalita dat II. Přístupy k ohodnocování výzkumných instrumentů založené na modelování kovariančních struktur." Sociologický časopis 34 (2): 195-204.
    31. Saris, W. E., I. Gallhofer. 2003. Report on the MTMM Experiments in the Pilot Studies and Proposals for Round 1 of the ESS. University of Amsterdam.
    32. Saris, W. E., I. N. Gallhofer. 2007. "Estimation of the Effects of Measurement Characteristics on the Quality of Survey Questions." Survey Research Methods 1 (1): 29-43, http://dx.doi.org/10.1002/9780470165195.ch12. Go to original source...
    33. Saris, W. E., I. N. Gallhofer. 2014. Design, Evaluation, and Analysis of Questionnaires for Survey Research. New Jersey: Wiley, http://dx.doi.org/10.1002/9781118634646. Go to original source...
    34. Saris, W. E., A. Sattora, G. Coenders. 2004. "A New Approach to Evaluating the Quality of Measurement Instruments: the Split-ballot MTMM Design." Sociological Methodology 34 (1): 311-347, http://dx.doi.org/10.1111/j.0081-1750.2004.00155.x. Go to original source...
    35. Schilling, O. 2006. "Development of Life Satisfaction in Old Age: Another View on the Paradox." Social Indicators Research 75: 241-271, http://dx.doi.org/10.1007/s11205-004-5297-2. Go to original source...
    36. Schräpler, J. P., J. Schupp, G. G. Wagner. 2006. "Changing from PAPI to CAPI: A Longitudinal Study of Mode-Effects Based on an Experimental Design." Discussion Papers. Berlin: German Institute for Economic Research.
    37. Schwartz, N., F. Strack, H.-J. Hippler, G. Bishop. 1991. "The Impact of Administration Mode on Response Effects in Survey Measurement." Apllied Cognitive Psychology 5: 193-212, http://dx.doi.org/10.1002/acp.2350050304. Go to original source...
    38. Trusinová, R. 2014. "No Two Ageisms Are the Same: Testing Measurement Invariance in Ageism Experience Across Europe." International Journal of Social Research Methodology 17 (6): 659-675, http://dx.doi.org/10.1080/13645579.2013.823003. Go to original source...
    39. van der Veld, W. M., W. E. Saris. 2005. "The Nature of Measurement Error in Panel Data. Estimating Opinion Stability in Panel Surveys." Paper prepared for the first meeting of the European Association for Survey Research, Barcelona, 2005, July 18-21.
    40. Wiley, D., J. Wiley. 1970. "The Estimation of Measurement Error in Panel Data." American Sociological Review 35 (1): 112-117, http://dx.doi.org/10.2307/2093858. Go to original source...

    This is an open access article distributed under the terms of the Attribution-NonCommercial 4.0 International (CC BY-NC 4.0), which permits use, distribution, and reproduction in any medium, provided the original publication is properly cited. No use, distribution or reproduction is permitted which does not comply with these terms.


    Return to the content