Republican pollster Frank Luntz got smacked down by the American Association for Public Opinion Research for failing to disclose his methodology.
Perhaps we should complain to AAPOR about Warren Mitofsky.
Time and again I've read about researchers complaining they can't get the raw data or a clear explanation of methodology from Edison/Mitofsky's National Election Pool exit polling. If true, that may conflict with AAPOR's Code of Professional Ethics and Practices.
From AAPOR's news release:
"In particular, the AAPOR inquiry focused on Luntz's reporting, prior to the November elections in 1994, that his research showed at least 60 percent of the public favored each of the elements in the GOP "Contract with America."
When later asked to provide some basic facts about this research, Luntz refused.
"AAPOR holds that researchers must disclose, or make available for public disclosure, the wording of questions and other basic methodological details when poll findings are made public. This disclosure is important so that claims made on the basis of opinion research findings can be independently evaluated. Section III of the AAPOR Code states: "Good professional practice imposes the obligation upon all public opinion researchers to include, in any report of research results, or to make available when that report is released, certain essential information about how the research was conducted."
http://www.aapor.orgIncidentally, the AAPOR web site makes passing reference to other associations (CASRO and NCPP) that apparently have similar guidelines for disclosure of polling data and methodology. Here's a paste from the site:
Disclose all methods of the survey to permit evaluation and replication.
Excellence in survey practice requires that survey methods be fully ddisclosed and reportedin sufficient detail to permit replication by another rresearcher andthat all data (subject to appropriate safeguards to maintain privacy and confidentiality) be fully documented and made available for independent examination. Good professional practice imposes an obligation upon all survey and public opinion researchers to include, in any report of research results, or to make available when that report is released, certain minimal essential information about how the research was conducted to ensure that consumers of survey results have an adequate basis for judging the reliability and validity of the results reported. Exemplary practice in survey research goes beyond such standards for "mminimaldisclosure," promulgated by AAPOR and several other pprofessionalassociations (e.g., CASRO and NCPP) by (a) describing how the research was done in sufficient detail that a skilled researcher could repeat the study, and (b) making data available for independent examination and analysis by other responsible parties (with appropriate safeguards for privacy concerns).
A comprehensive list of the elements proposed for disclosure by one or more ssources which in combination, exceed the "standards for mminimumdisclosure" proposed by any one of the professional organizations includes:
* who sponsored the survey, and who conducted it;
* the purpose of the study, including specific objectives;
* the questionnaire and/or the exact, full wording of all questions asked, including any visual exhibits and the text of any preceding instruction or explanation to the interviewer or respondents that might reasonably be expected to affect the response;
* a definition of the universe the population under study which the survey is intended to represent, and a description of the sampling frame used to identify this population (including its source and likely bias);
* a description of the sample design, including cluster size, number of callbacks, information on eligibility criteria and screening procedures, method of selecting sample elements, mode of data collection, and other pertinent information;
* a description of the sample selection procedure, giving a clear indication of the methods by which respondents were selected by the researcher, or whether the respondents were entirely self-selected, and other details of how the sample was drawn in sufficient detail to permit fairly exact replication;
* size of samples and sample ddisposition theresults of sample implementation, including a full accounting of the final outcome of all sample cases: e.g., total number of sample elements contacted, those not assigned or reached, refusals, terminations, non-eligibles, and completed interviews or questionnaires;
* documentation and a full description, if applicable, of any response or completion rates cited (for quota designs, the number of refusals), and (whenever available) information on how non respondents differ from respondents;
* a description of any special scoring, editing, data adjustment or indexing procedures used;
* a discussion of the precision of findings, including, if appropriate, estimates of sampling eerror withreferences to other possible sources of error so that a misleading impression of accuracy or precision is not cconveyed and a description of any weighting or estimating procedures used;
* a description of all percentages on which conclusions are based;
* a clear delineation of which results are based on parts of the sample, rather than on the total sample;
* method(s), location(s), and dates of interviews, fieldwork or data collection;
* interviewer characteristics;
* copies of interviewer instructions or manuals, validation results, codebooks, and other important working papers; and
* any other information that a layperson would need to make a reasonable assessment of the reported findings.