The challenge and opportunity

Done well, testing provides schools with vital information about what students know and where there are gaps, helping to inform decisions around instruction and resource allocation. It can be a key tool in advancing educational equity for underrepresented groups. However, a single assessment cannot accomplish everything, and overlapping or redundant tests can squander time and money. A balanced system is one in which all assessments work together to provide a useful and coherent profile of information about student achievement to improve teaching and learning.

The call to re-balance assessment, crystallized in a 2001 report by the National Research Council, stems from the realization that most state summative tests have little impact on instructional practice. Yet two decades after that report’s release, examples of well-functioning balanced systems are hard to find. There are several reasons for this, but they tend to fall into four categories, according to a 2019 report from the Center for Assessment: politics and policy; the commercialization and proliferation of interim assessments; a lack of attention to curriculum and learning in the design of assessment systems; and a lack of assessment literacy at multiple levels of the system.

The Educator Leader Assessment Collaborative (ELAC), which focuses on building assessment literacy among teachers and education leaders, tackles the last of these challenges. The pilot is designed to gain a better understanding of how to bring a “balanced” approach to assessment to Hawai’i’s schools. Achieving this goal is no easy task. It requires the planning and engagement of multiple stakeholders and agreement around a shared vision of how assessments can work together with curriculum and instruction to support rich learning environments. It also requires understanding that any one assessment cannot meet all information needs.

In Hawai’i and other states, the aggressive marketing of assessment tools by vendors has created confusion among many teachers. Some have been sold on the idea that testing solves all their problems; others are administering tests because they’ve been told to, without understanding how to use the data to improve their instruction.

Though HIDOE’s assessment section offers workshops on the “why” and “how” of balanced assessment, the section’s limited number of staff don’t have the bandwidth to train everyone, according to Dianne Morada, a test development specialist for math. Rather, through ELAC, staff hope to extend their reach by preparing a cadre of “rockstar educators” to train their peers.

“We’re developing and empowering leaders at the local level,” said Tyler Belanga, who oversees the state grant funding for the collaborative.

Kelsie Pualoa, the ELA test development specialist, said the ultimate goal of ELAC is to help teachers and administrators cut through the marketing noise and learn how to “assess mindfully.”

“…the ultimate goal of ELAC is to help teachers and administrators cut through the marketing noise and learn how to “assess mindfully.”

Pilot recruitment — Getting the right people at the table

While participation in the pilot is voluntary, individuals and participant teams were carefully recruited to help build an understanding of the diversity of assessment approaches across the state, promote cross-pollination of ideas, and build capacity at the local level. The ideal makeup of a school team includes key staff with different aspects of responsibility for assessment at a site, such as school leaders, curriculum coordinators, testing coordinators, staff in research roles, coaches and mentors, resource teachers, and teacher leaders.

According to Allison Brown, NTC’s lead program consultant for the project, the pilot cohort was carefully curated to test ways to improve communication and collaboration between important stakeholder groups. “We’ve seen the most traction and transfer to practice from professional learning opportunities when you had the right mix of staff at the table to actually effect systems change.”

“We’ve seen the most traction and transfer to practice from professional learning opportunities when you had the right mix of staff at the table to actually effect systems change.”

Building assessment literacy — Getting on the same page

Data collected by HIDOE assessment staff indicated that misconceptions and miscommunication across multiple levels from local to state about what a balanced approach to assessment looks like was a big barrier. To address this, initial professional learning was focused on language and terminology, creating shared definitions, and aligning on the purposes and uses of different types of assessment. Another aim was to translate the research on best practices in assessment to make it meaningful, practical, and manageable for educators. Additional learning focused on selecting assessments aligned with the scope and sequence of instruction and learner-centered formative assessment.

Participants’ key takeaways included the need to more effectively engage students and give them voice and choice in the types of assessment used. They also reflected on “how we as teachers should provide more opportunities for students to share their thinking” in the context of assessment and finding ways to use “student talk and feedback more efficiently in the classroom.”

Many participants appreciated thinking about how often we approach assessment with a deficit mindset. It takes persistent reflection and attention to flip the narrative and ensure all students are learning and being assessed fairly.

Assessing current practice — Identifying starting places

NTC also worked with state assessment staff to conduct a rough landscape analysis of current practice using data collected through focus groups. Some effective trends in local testing approaches within or across grade levels or content areas included:

• use of beginning-of-year screeners to inform teaching for the year
• effective use of formative assessments in some sites
• deployment of professional learning communities (PLCs) to examine data with focus on individual students

Participants surfaced challenges such as overtesting, too much focus on summative assessment, lack of buy-in, lack of teacher understanding of how to use results, timeliness of results, teacher turnover and shortages, misalignment with what is taught, and a lack of understanding that the student is one of the most important users of assessment data.

NTC then introduced participants to our assessment inventory and diagnostic tool to analyze current local assessment practices and portfolios. Participants used this resource to create a summary of tests in current use at their sites, how the data is (or is not) used, and strengths and gaps in their school’s overall assessment strategy analyzing for coherence, comprehensiveness, continuity, utility, and efficiency.

Almost half of the participating sites identified the need to increase the efficiency of assessment (addressing redundancies and ensuring that all assessments were used effectively in decision-making). Other areas of focus included coherence — using assessment in alignment with an underlying learning model — and the usefulness of results in decision-making.

One participant shared that decision-making about assessment should be driven by “focusing on the why, the how, and if student learning needs are being met.” Another considered how data teams and PLCs could be organized differently to use assessment data more effectively. Many participants were interested in ways to start the conversation at their schools to look at their assessment practices.

…decision-making about assessment should be driven by “focusing on the why, the how, and if student learning needs are being met.

Setting goals for balanced assessment strategy — Designing action plans

Based on site diagnostics, NTC consultants then began providing individualized assessment strategy consultation, working with individual teams focused first on setting goals and then developing strategic plans to address gaps in assessment practice.

For example, several participants planned to work within grade-level teams or content areas to develop common formative assessments to inform instruction. Many others plan to do more to involve students in the assessment process, through surveys or student-centered assessment practices. The team from Ewa Beach, a working-class community west of Honolulu, near the Army base, has committed to both goals and identifying duplicative or redundant assessments across the K-5 school.

Supporting local leadership — Building change management capacity

In subsequent professional learning, participants explored leadership theory and role-played scenarios that might occur when introducing change to their communities. Building trust and garnering buy-in from other stakeholders at their schools was called out by participants as a key concern. “We need ways to get and keep teachers motivated,” one said.

Participants discussed strategies for fostering student engagement and practiced skills for conversing with resistant or skeptical colleagues, including deep listening. Many participants went away with a plan for engaging other staff at their sites, anticipating roadblocks and finding allies to overcome barriers. “By having a better understanding of the traits of a transformational leader, I now have immediate next steps that I can implement when facilitating our PLC meetings,” one participant shared.

“My big ‘Aha!’ moment was in having a growth mindset and positive attitude in believing that ALL teachers, even the most set in their ways, can have a change in practice if provided good (transformational) leadership,” said another.

A look ahead

In the wake of the COVID-19 pandemic, there was “an understandable panic” about learning loss, and a “huge rush to make sure we’re assessing and getting the data,” HIDOE’s Pualoa said. While this rush was well-intentioned, “other things kind of got lost in the mix.” Through ELAC, the division hopes to restore some equilibrium to the testing landscape.

Pualoa shared they also want to equip educators with the tools to contextualize their data — to consider it in conjunction with other data, recognizing that test scores are “not the whole story.”

By showing educators how they can use testing data constructively, project leaders hope to overcome some of the “negative mindset around assessment,” Morada said.

Staff acknowledged that it will take time to spread balanced assessment across the state. Still, they’re already seeing a shift in participants’ mindsets. During the first session, some doubted their ability to make a difference in their schools, saying things like, “I’m just a teacher.” But by the second session, they started seeing themselves as change agents.

“This group of educators has really inspired us,” said Morada. She added that in a field where so much is mandated by the state or federal government, it’s refreshing and exciting to be part of a grassroots effort.

Future NTC professional learning and coaching will focus on how to analyze assessment data to inform instruction, monitoring the ongoing process of initial strategic plans, and the development of a comprehensive balanced assessment strategy for the 2024-25 school year based on learnings from the pilot. A formal report due out in the fall will focus on lessons learned and bright spots and successes.