Contact Us

Assessment Insights

Supporting New Paradigms for Educational Accountability

[fa icon="calendar"] January, 2016 / by Chad Barrett

Chad Barrett

When I learned that Linda Darling-Hammond, an educational researcher and policy advocate at Stanford University, was taking on the role of President and CEO of Learning Policy Institute, I was curious about the new organization and what its goals might be. So I did some research and found several articles that reveal her views on state educational accountability systems. One of the new institute’s goals is to work with states to put these views into practice.

In a 2014 article published by the Stanford Center for Opportunity Policy in Education (SCOPE), Linda Darling-Hammond and co-authors Gene Wilhoit and Linda Pittenger share their views about what should be included in statewide accountability systems. I liked many of their ideas, but have concerns about a couple of their suggestions. In this article I share those concerns and present some ideas about how large-scale assessment companies, like Measured Progress, can support the authors’ vision.

Student-Selected Portfolios for Accountability?

The authors recommend using portfolios for accountability purposes, particularly with high school students. They believe that a student-selected portfolio “serves as evidence that the student has met core competencies for readiness” (Darling-Hammond, Wilhoit, & Pittenger, 2014). Although Darling-Hammond and her colleagues don’t explicitly say that such portfolios should be used for accountability or graduation purposes, they do recommend using common rubrics to score these portfolios.

 Portfolios in which students choose the work products for inclusion are considered problematic assessment instruments. Brian Stecher, in a paper published by SCOPE, went so far as to say “free-choice portfolios should not be considered as performance assessments for use in high-stakes large-scale testing” (Stecher, 2010). Portfolios are poor performance assessments because the differences in the work products selected by students make it difficult for scorers to consistently evaluate a group of portfolios. The reliability of any score resulting from a student-selected portfolio submission is low compared to other assessment instruments. Student-selected portfolios also raise concerns as to whether the student did the work.

Use Large-Scale On-Demand Performance Tasks

I recommend using on-demand performance tasks rather than portfolios. Such tasks can be used to gather evidence of students’ academic readiness for college or the workforce and can be scored with more consistency. Further, scores from these tasks can more readily be used in conjunction with scores from other large-scale assessments. Psychometricians and test developers can monitor the development, implementation, and scoring of on-demand performance tasks. Such efforts increase score reliability and viability for use in making high-stakes decisions. Measured Progress has an extensive and successful history developing, administering, and scoring on-demand performance tasks.

Can Districts Develop Performance Tasks for Accountability?

Darling-Hammond and her colleagues discuss at length how to develop an accountability system for meaningful learning. They advocate for a “system of higher-quality assessments, both state-designed and locally developed,” that include “authentic performance tasks (e.g., classroom-based project and products like those used in other countries) that assess and encourage the development of the full range of higher order skills” (Darling-Hammond, Wilhoit, & Pittenger, 2014).

My concern with this approach is the focus on locally-developed performance assessments. Although some large-city school districts have the capacity to implement such a plan, I do not believe that the majority of districts have such capacity. I spent the majority of my teaching career at one of over 8,000 rural public school districts (NCES, 2004), which comprise about half of the country’s public school districts (NCES, 2004). I worked at a school where there were three math teachers for grades 7–12. All of us had outside-of-class commitments: One was an athletic coach, one worked with the yearbook, and I was trying to start an after-school club. Under these circumstances, we wouldn’t have been able to put in the time to develop, administer, and score the kinds of tasks described by Darling-Hammond and her colleagues.

States Need to Lead the Way

I recommend that states, rather than districts, work with assessment vendors like Measured Progress to increase teacher capacity to develop, administer, and score authentic performance assessments. Measured Progress recently worked with the Ohio Department of Education on such a program. The Ohio Performance Assessment Pilot Project (OPAPP) assembled cohorts of teachers from around the state to participate in a two-year program. Measured Progress worked with each cohort to write performance tasks, pilot those tasks in their classrooms, and then evaluate student work using rubrics and training materials.

This state-level pilot program increased teacher capacity to write high-quality authentic performance tasks, provided sustained teacher professional development, and improved student outcomes. In a 2013 report for the Center for American Progress, Darling-Hammond supported long-term professional training focused on content and evaluation of student work. She and co-author Beverly Falk stated that such programs resulted in the “most positive” results for student achievement.

Conclusion

Linda Darling-Hammond and her colleagues present a compelling vision for new accountability systems. With the passage of the Every Student Succeeds Act, states have the opportunity to redefine accountability systems using Darling-Hammond’s ideas. She and her colleagues noted in the SCOPE paper that states such as California, New Hampshire, and Kentucky are already implementing some of their ideas. 

Large-scale assessment organizations, like Measured Progress, can help states to develop assessments that provide useful information about whether or not a student is on track or academically ready for college or the workforce. We can also support a state’s efforts to build the capacity of in-state educators to do this important work. Such collaborations can help states collect better evidence about whether students are prepared for life after graduation and make better decisions about how to improve education in their state.

Reference

Darling-Hammond, L. & Falk, B. (2013, September). Teacher learning through assessment: How student-performance assessments can support teacher learning. Washington, DC: Center for American Progress. Retrieved from https://www.americanprogress.org/wp-content/uploads/2013/09/TeacherLearning.pdf.

Darling-Hammond, L., Wilhoit, G., & Pittenger, L. (2014) Accountability for a college and career readiness: a new paradigm. Stanford, CA: Stanford Center for Opportunity Policy in Education. Retrieved from https://edpolicy.stanford.edu/publications/pubs/1257.

National Center for Education Statistics (NCES). (2004). Number of public school districts, by locale code (CCD) and state: 2003-04. Retrieved December 15, 2015, from https://nces.ed.gov/surveys/ruraled/TablesHTML/5localedistricts.asp.

Stecher, B. (2010). Performance assessment in an era of standards-based educational accountability. Stanford, CA: Stanford University, Stanford Center for Opportunity Policy in Education. Retrieved from https://scale.stanford.edu/system/files/performance-assessment-era-standards-based-educational-accountability.pdf.

Topics: Accountability

Chad Barrett

Written by Chad Barrett

Chad Barrett is a Measurement Services Senior Advisor at Measured Progress, working with content and psychometrics teams to develop test designs and contribute to other large-scale assessment solutions. Chad has more than 15 years of educational assessment experience, particularly in content development and program design.