Every summer Measured Progress welcomes a few standout doctoral students to join the psychometrics team, through the Nambury Raju Internship Program. The program gives up-and-coming psychometricians a chance to put their coursework to practical use, and keeps Measured Progress psychometricians connected to current university programs in the field. This year, we welcomed two accomplished students to the program.
Like our director of psychometrics Dr. Jennifer Dunn, who shared her story in a recent TEDx talk, Jing Jiang became interested in measurement as an undergraduate. Originally from Jiangsu Province in China, Jing majored in statistics with a minor in psychology. She is earning her Ph.D. in Educational Research, Measurement, and Evaluation at Boston College. “Most of my friends and family members are not familiar with psychometrics, so I usually tell them I got an internship at a famous testing company.”
Unhee Ju, originally from Seoul, South Korea, joined Jing in the 8-week program. A doctoral student of Measurement and Quantitative Methods at Michigan State University, Unhee discovered the Measured Progress internship program through the recommendations of her academic advisor and fellow students, and through an AERA/NCME publication.
The core of the internship program is to complete a research project under the guidance and leadership of the Measured Progress psychometrics team. Jing focused her work on post-equating. Equating is used to adjust scores on different test forms so that a particular score indicates the same level of achievement on all of the forms included in the equating study. “Before this internship, I didn’t have much experience in equating and I am new to this field. So choosing this topic was a big challenge. Under the guidance of Measured Progress psychometricians, I quickly become familiar with this topic and developed a specific research direction that is relevant to my current research interest and expertise.”
For her research project, Unhee worked on developing an index to evaluate the efficacy of a particular linking method—the Stocking-Lord method—for the common item nonequivalent design. Unhee explained that “linking is a procedure to place two separate scales (e.g., Grade 3 test scale and Grade 4 test scale) onto the same common scale via a set of common items, which provide the link between the two scales.” Common items are items that are exactly the same in multiple test forms administered to different students. While the Stocking-Lord method is the most widely used linking method, Unhee noted that no index exists to evaluate the performance of the Stocking-Lord linking method in operational settings. “Therefore, my project aims to develop an index and demonstrate its usefulness in evaluating the Stoking-Lord method via both a simulation study and an empirical study.”
Learning About Psychometrics Practice
Beyond their research projects, both Unhee and Jing appreciated being how involved they were in “real-world” operational assessment activities. Jing found that meeting with leadership teams and functional groups was invaluable. “[It was] much more complicated than I thought, but highly organized and cooperative. Also, I learned about staff members’ specific roles, and how they use their expertise in practice, which I could never know in graduate school. In practice, the problems we face might be totally different from those in theoretical research and my responsibility is to develop a solution to deal with some real world questions.”
Unhee also felt her view was broadened. “With the internship, I can learn about concepts and issues associated with operational work and extend the scope of my research,” she said. “In addition, I’m now more aware of the difference between theoretical work and real operational work, which is very important when we think about research topics in education.”
After completing her Ph.D., Jing plans to begin a career in the assessment industry. Unhee hopes to find a dissertation topic that can “contribute to filling the gap between theoretical and empirical work.” Once she completes her Ph.D., she hopes to begin a career in academia or the assessment industry in the United States.
The interns were not the only ones to learn from the internship experience. Staff psychometricians reap benefits, too: They enjoy mentoring their new colleagues, and gain fresh perspective. “Both interns did outstanding work,” said Louis Roussos, senior psychometrician and research scientist. “Their research has given us new tools that will improve the way we evaluate linking items—which is extremely important and relevant to ongoing work for operational assessment programs.”