Editing Educational Testing Service (section)

==Scientific contributions==

In keeping with the purposes for which it was established, ETS developed a program of research that covered not only [[psychometrics|measurement]] and [[education]] but also such related areas as [[statistics]], [[educational evaluation]], and psychology, particularly [[cognitive psychology|cognitive]], [[developmental psychology|developmental]], [[personality psychology|personality]], and [[social psychology]].<ref name=Advancing>{{cite news| last =Bennett| first =R.E.|author2=van Davier, M. (Eds.).| title =''Advancing Human Assessment: The Methodological, Psychological, and Policy Contributions of ETS''| series =Methodology of Educational Measurement and Assessment| publisher =Cham, Switzerland: Springer Open| year =2017| doi =10.1007/978-3-319-58689-2| isbn =978-3-319-58687-8}}</ref> This broad-based research program attracted many individuals who distinguished themselves in their fields, often while at ETS but also in subsequent professorial positions. Among the more influential scientists have been [[Harold Gulliksen]] (whose book, ''Theory of Mental Tests'', helped codify [[classical test theory]]);<ref name="Gulliksen memorial">{{cite news| last =Burkhart| first =F.| title =Harold Gulliksen, 93, Pioneer in Testing, Dies| work=The New York Times| url =https://www.nytimes.com/1996/11/03/nyregion/harold-gulliksen-93-pioneer-in-testing-dies.html?pagewanted=1| access-date =April 17, 2010 | date=November 3, 1996}}</ref><ref name="Gulliksen book">{{cite news| last =Gulliksen| first =H.| title =Theory of Mental Tests| publisher =New York: Wiley| year =1950}}</ref> [[Frederic M. Lord|Frederic Lord]] ([[item response theory]]); [[Samuel Messick]]<ref name="Messick memorial">{{cite news| last =Landis| first =D.|author2=Tzeng, O.C.S.| title =Samuel J. Messick (1931-1998).| pages =132–133| publisher =American Psychologist, 57(2)| year =2002}}</ref> ([[test validity|modern validity theory]]); [[Robert L. Linn|Robert Linn]] (known for testing and educational policy); [[Norman O. Frederiksen|Norman Frederiksen]] ([[performance test (assessment)|performance assessment]]); [[Tucker decomposition|Ledyard Tucker]] (test analysis, including inventing the "Angoff Method" of [[Standard-setting study|standard setting]]); [[Donald Rubin]] ([[missing values|missing data]] and [[Rubin causal model|causal modeling from observational data]]); [[Karl Gustav Jöreskog|Karl Jöreskog]] ([[structural equation modeling]] and [[confirmatory factor analysis]]); Paul Holland ([[differential item functioning]], [[test equating]], causal modeling); [[Howard Wainer]] ([[differential item functioning]], Testlet Response Theory, statistical graphics); [[John Bissell Carroll|John Carroll]] (language testing and cognitive psychology); [[Michael Lewis (psychologist)|Michael Lewis]] (infant cognitive, social, and emotional development); Irving Sigel (children's cognitive development);<ref name="Sigel memorial">{{cite news| last =McGillicuddy-DeLisi| first =A.|author2=Shafrir, U. |author3=Johnson, J. |author4=Renninger, K. | title =Remembering Irving Sigel| page =253| publisher =Journal of Applied Developmental Psychology, 29(4)| year =2008}}</ref>  [[Herman Witkin]] ([[cognitive styles|cognitive and learning styles]]); [[K. Patricia Cross]] (adult education); [[Samuel Ball (educator)|Samuel Ball]] (an evaluation researcher who documented the positive educational effects of [[Sesame Street]]); [[David Rosenhan]] (known for the [[Rosenhan experiment]], which challenged the validity of psychiatric diagnosis); [[Jeanne Brooks-Gunn]] (the effects of poverty on infant, child, and adolescent development); Robert J. Mislevy (Evidence-Centered Design); and Anthony Carnevale (education and the workforce).

Members of the ETS staff have been among the presidents of the National Council on Measurement in Education (NCME); the [[Psychometric Society]]; the Measurement and Evaluation Division of the [[American Educational Research Association]] (AERA); the Evaluation, Measurement and Statistics Division of the [[American Psychological Association]] (APA); the APA Developmental Psychology Division; and the [[Jean Piaget Society]]. They have been among the executive editors of the ''[[Journal of Educational Measurement]]'', ''[[Journal of Educational and Behavioral Statistics]]'', ''Educational Evaluation and Policy Analysis'', ''Journal of Educational Psychology'', ''Journal of Applied Developmental Psychology'', and ''Discourse Processes''. Major citations received while on staff have included elected membership to the [[National Academy of Education]] ([[K. Patricia Cross]], 1975; Gregory Anrig, 1981; Paul Holland, 2005; [[Randy E. Bennett]], 2022; Irwin Kirsch, 2022); (the APA Distinguished Contributions to Knowledge Award (Norman Frederiksen, 1984), the APA Distinguished Scientific Contribution Award (Frederic Lord, 1988; Howard Wainer, 2009); the AERA E.F. Lindquist Award (William Turnbull, 1981; Frederic Lord, 1988; Samuel Messick, 1994; Paul Holland, 2000; Wendy Yen, 2008; Howard Wainer, 2015; Charles Lewis, 2018; [[Randy E. Bennett]], 2020); the NCME Career Contributions to Educational Measurement Award (Frederic Lord, 1990; Paul Holland, 2004; Howard Wainer, 2007; Neil Dorans, 2010; Linda Cook, 2017; Shelby Haberman, 2019); The Psychometric Society's Lifetime Achievement Award (Howard Wainer, 2013), and the Jean Piaget Society's Lifetime Achievement Award (Irving Sigel, 2002); among many other awards.

ETS has produced both new knowledge and methodology, especially in measurement and statistics, much of which has been taken up by assessment organizations around the world. Among the key scientific contributions were:

*co-invention of [[item response theory]], an integrated framework for asking and answering a variety of practical problems related to the design and analysis of tests;<ref>Lord, F.M. (1980). "Applications of item response theory to practical testing problems" Mahwah, NJ: Lawrence Erlbaum Associates, Inc.</ref><ref name="Lord">{{cite news| last =Lord| first =F.M.| title =A Theory of Test Scores.| publisher =Psychometric Monographs, 7| year =1952}}</ref><ref name="Lord and Novick">{{cite news| last =Lord| first =F.M.|author2=Novick, M.R.| title =Statistical Theories of Mental Test Scores| publisher =Reading, MA: Addison-Wesley| year =1968}}</ref>
*creation of an approach and software for [[structural equation modeling]] and [[confirmatory factor analysis]] ([[LISREL]]), used throughout the social sciences to test theoretical relationships among variables;<ref>{{cite news| last =Joreskog| first =K.G.|author2=Van Thillo, M.| title =LISREL: A General Computer Program for Estimating a Linear Structural Equation System Involving Multiple Indicators of Unmeasured Variables (RB-72-56)| publisher =Princeton, NJ: Educational Testing Service| year =1972| url =http://files.eric.ed.gov/fulltext/ED073122.pdf}}</ref>
*seminal contributions to [[test validity|modern validity theory]], including the idea that validity was a unitary concept and that the evaluation of score meaning requires consideration of the consequences of test use as those consequences may imply functional problems with the test;<ref name="Messick">{{cite news| last =Messick| first =S.| title =Validity| pages =13–103| publisher =In R.L. Linn (Ed.), Educational Measurement (3rd Ed.). New York: MacMillan| year =1989}}</ref>
*development of widely used approaches to data analysis when there are [[missing values|missing data]];<ref name="EM">{{cite news| last =Dempster| first =A.P.|author2=Laird, N.M. |author3=Rubin, D.B. | title =Maximum Likelihood from Incomplete Data via the EM Algorithm| pages =1–38| publisher =Journal of the Royal Statistical Society, 39(1), Series B (Methodological)| year =1977}}</ref>
*generation of approaches to [[Rubin causal model|causal modeling from observational data]];<ref>Rubin, D. ''Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies'', Journal of Educational Psychology, Vol. 66, No.5, (1974), pp. 689.</ref><ref name="Holland">{{cite news| last =Holland| first =P.| title =Statistics and Causal Inference| pages =945–960–103| publisher =Journal of the American Statistical Association, 81(396)| year =1986}}</ref>
*invention of the [[in basket test|In-Basket Test]] (used throughout the world to assess applicants for managerial jobs in a wide variety of industries);<ref>{{cite news | last =Frederiksen| first =N.|author2=Saunders, D.R. |author3=Wand, B. | title =The In-Basket Test| pages =86–88| publisher =Psychological Monographs: General and Applied, 71(9)| year =1957}}</ref>
*development of methods for detecting test unfairness, including invention of the Standardization approach to [[Differential item functioning|Differential Item Functioning]] (DIF) and application of the Mantel-Haenszel method;<ref>{{cite news | last =Holland| first =P.W.|author2=Thayer, D.T.| title =Differential item performance and the Mantel-Haenszel procedure| publisher =In H. Wainer & H.I. Braun (Eds.), Test Validity. Hillsdale, N.J.: Lawrence Erlbaum.| year =1988}}</ref>
*creation of the [[holistic scoring|holistic-scoring]] approach to [[writing assessment]], a means of rapidly and reliably judging the quality of essay text, which allowed direct [[writing assessment]] to become a more affordable alternative to multiple-choice questions for large-scale testing programs;<ref>{{cite news | last =Coffman| first =W.E..| title =Essay Examinations| pages =271–302| publisher =In R.L. Thorndike (Ed.), Educational Measurement (2nd Ed.) Washington, D.C.: American Council on Education.| year =1971}}</ref><ref>{{cite news | last =Elliot| first =N.| title =On a Scale: A Social History of Writing Assessment in America.| pages =160–165| publisher =New York: Peter Lang| year =2005}}</ref>
*development of research-based procedures and standards for occupational licensing and certification.<ref name=ShimbergObit>{{cite news| last =Esser| first =B.F.| author2=Kruger, D.H.|title =Benjamin Shimberg, 85, Expert on Testing in the Professions| work=The New York Times| url=https://www.nytimes.com/2003/09/30/nyregion/benjamin-shimberg-85-expert-on-testing-in-the-professions.html}}</ref>