What’s Next for the Learning Metrics Task Force?

Why Measure Learning?

Measurement, if done well, can play a critical role at many levels in improving education quality and learning. Effective teachers measure learning in the classroom to adjust and individualize instruction. Head teachers, school administrators and school district leaders measure learning to target resources and improve school quality. Many national governments also measure learning to diagnose the overall health of their education systems and develop policies to improve the teaching and learning process and ultimately student learning outcomes. Civil society actors, donors and development agencies use assessments to measure the effectiveness of programming and advocate for effective education policies and practices.

Global Commitment to Improving, and Measuring, Learning

There has been a longstanding and broad-based commitment among the world’s education ministers to improving, and hence measuring, learning outcomes.  In particular, Goal 6 of Education for All commits world leaders to: “Improving all aspects of the quality of education and ensuring excellence of all so that recognized and measurable learning outcomes are achieved by all, especially in literacy, numeracy and essential life skills”.

In the first phase of the Learning Metrics Task Force (LMTF 1.0), the task force heard from nearly 2,000 people, including more than 50 ministers of education, permanent secretaries or their representatives through a broad consultation process. The vast majority of consultation participants, echoing previous commitments to EFA Goal 6, articulated the importance of improving learning outcomes and the role of measurement in helping to inform actors, from teachers to ministers, about the efficacy of their interventions. Debates were heated, however, about how to best assess learning to improve outcomes. What competencies young people in all countries should develop, what approaches to measurement should be used, and how issues of equity should be addressed were all examples of the tough questions discussed. Ultimately, the task force weighed a wide range of (often conflicting) input and after much debate came to agreement on a series of recommendations.

Recap of LMTF 1.0 Recommendations

The consultative process of LMTF 1.0 was structured around three guiding questions: What learning is important globally? How should it be measured? How can measurement of learning improve education quality?

The collective response in the form of final recommendations was launched during the 2013 U.N. General Assembly meetings as a contribution to the U.N. secretary-general’s Global Education First Initiative (GEFI):

1. Global Focus on Access plus Learning. The task force calls for a return to the vision of the EFA agenda, aligned with the priorities set out by GEFI in order to shift the past focus in the Millennium Development Goals (MDGs) from access to access plus learning. The collection and appropriate use of better data on learning is central to this effort.

Areas of Measurement Recommended for Global Tracking

2. Learning Competencies. The task force calls upon education systems to offer opportunities to children and youth to master a set of competencies in seven domains of learning, which are considered important for all children to learn in addition to the wide range of other learning objectives countries may have for their students.

3. Learning Indicators for Global Tracking. The task force recommends a small set of learning indicators within seven areas of measurement (see figure) to be tracked in all countries. These areas of measurement represent fundamental learning opportunities from early childhood through lower secondary. Some indicators within these areas currently exist, while others need to be developed.

4. Supporting Countries. The task force recommends that support be provided to countries that desire it in strengthening their assessment systems in ways that help improve learning levels. An international, multi-stakeholder partnership would ensure better collaboration among existing agencies and fill essential gaps in support to countries.

5. Equity. Measurement of learning must include an explicit focus on equity, with particular attention to rising inequality within countries. Data on child characteristics (e.g. sex, socioeconomic status, geographical location, disability) together with information on such things as school conditions and teacher quality should be used to ensure equitable learning opportunities.

6. Assessment as a Public Good. Measures for the indicators recommended for global tracking must be considered a public good, with tools, documentation and data made freely available. No country should be precluded from measuring these particular learning outcomes due to financial constraints.

7. Taking Action. The task force calls on all actors to play a role in advancing these recommendations.

The task force acknowledged that it had completed its work as originally set out; however, it also recognized the high demand from stakeholders to take advantage of the momentum and interest gained thus far. In response, the task force has agreed to make the transition into a new stage of work, with a focus on implementing its recommendations.

LMTF 2.0: Debates and Next Steps

With a diverse group of engaged and committed stakeholders representing both the global south and north, and high interest from multiple governments to adapt and try out the recommendations to improve national learning levels, task force members held a “next steps” meeting in November 2013. The task force agreed to transition to a new phase of work—being referred to as LMTF 2.0—and to continue collaborating as a multi-stakeholder group for two more years, running from January 2014 to December 2015. A number of questions around the overarching goal, working model, and results desired were debated during the meeting and are summarized below.

Overarching Goal

The task force acknowledged that the group had only just begun to answer the third guiding question from LMTF 1.0: How can measurement of learning improve education quality? To address this question adequately, task force members were clear that much more attention and energy was needed at the country level, concurrent with the ongoing global-level work.

There was vigorous debate about whether the task force should expand its scope to address improving learning broadly, including the wide range of strategies required—teacher development, teaching and learning materials, school nutrition, etc. Task force members acknowledged that measurement is only part of the answer. Arguments for expanding the scope included the importance of shifting the emphasis in the next phase to improving learning and not representing assessment as an end to itself. Furthermore, members recognized that LMTF is the main global initiative specifically focused on improving learning, and so some argued that it should expand its remit to fully cover the issue. Others argued that expanding to other areas that help improve learning, in addition to better student assessment, would likely be too broad and unmanageable and could possibly encroach on the mandates of other existing initiatives (e.g. UNESCO’s Teachers Task Force for EFA).

Ultimately, the task force decided that the overall goal of LMTF 2.0 should be to help countries improve learning levels, and that it should focus its activities on how to support better assessment systems and better use of assessment data to do just that. Any work at the global, regional and of course country level that the task force takes up as part of LMTF 2.0 should be in service of this overarching goal.

Working Model

Task force members acknowledged that the next phase of work requires a different working model from before. In LMTF 1.0, the task force began by collecting input from technical working groups and broad public consultations on the three guiding questions listed above. Then discussion and debate on the consultation feedback followed, and finally the task force reached consensus on its recommendations. In contrast, LMTF 2.0 will focus on implementing those recommendations, which requires a much more decentralized approach. An expanded partnership of LMTF member organizations will: take on aspects of the work based on their core mandates and competencies; collaborate with others as appropriate; and regularly share, discuss and update each other on progress. Over the next two years, the task force will serve as a network on improving learning with multiple stakeholders engaged; however, it will also be important to develop a clear plan for transitioning the work after two years, either through embedding it within partner organizations or some other approach.

Five Results to Achieve

The task force will work to achieve the following five main results at the end of two years:

1. Technical. Indicators in each of the areas of measurement recommended for global tracking are developed (to different degrees of readiness for implementation).
2. Institutional. At least 10 “Learning Champions” use task force recommendations to support country-level work on learning assessment and use of assessment data to improve learning.
3. Political. Task force work supports ability of access plus learning to feature in the post-2015 development agenda.
4. Assessment as a Public Good. A strategy is developed for advancing an agenda in which learning data is supported as a global public good.
5. Knowledge Sharing. Actors and experts in learning assessment share knowledge and coordinate efforts.

Developing Indicators for Global Tracking

Among the LMTF 1.0 recommendations, the task force proposed seven areas of measurement as desirable and feasible for tracking at the global level (see figure). Technical work is now required to develop indicators under each of these seven areas. Accordingly, a number of LMTF partners have already begun organizing to undertake this work.

In addition, the task force has opted to add one more area of measurement: equity.  Measuring equity is not a stand-alone effort but one that must cut across all the other seven areas; however, a methodology and process for doing this is needed. Hence, a group of organizations will focus on equity, examining how the LMTF global indicators can provide critical information on characteristics that lead to disparities in learning, in order to support equitable improvements in learning outcomes (e.g. gender equality in education). The aim is to expedite the work over the next two years to be ready (or nearly ready) for implementation post-2015. Members agreed it is also important to take into account and support the linking of assessment results from existing regional and international assessments, as well as citizen-led assessments, which can contribute to a healthy ecosystem of assessments for all children, those in and out of school.

Supporting Countries

Consultation participants in LMTF 1.0 identified many challenges to improving learning, and in particular, the difficulty of developing and sustaining robust assessment systems at the country level and most importantly effectively using assessment data to help guide actions that improve learning. In particular, a range of difficulties were identified, including lack of national ownership of the assessment system; lack of national institutions with sufficient technical capacity; insufficient political will to assess learning regularly and make the results publicly available; lack of information about how to use data to guide actions that improve learning; and scarcity of neutral sources of information on the advantages and disadvantages of the various assessment tools available. Many also identified the desire to learn from each other, recognizing that countries have different strengths and weaknesses.

To help address these challenges, the task force invites “Learning Champion” countries, states, districts and cities to join LMTF 2.0 and work together to advance the goal of improving learning outcomes. Learning Champions will have the opportunity to connect with and learn from each other as well as other education systems and LMTF partners in the region and across the globe; receive technical guidance on navigating the many options related to learning assessment; and play a leading role in developing global good practice on assessment and learning. In turn, LMTF partners will look to Learning Champions to provide country-level expertise and feedback during the development of indicators described above, as well as potentially pilot new assessment instruments relevant to identified national priorities. The task force will also work with relevant regional bodies to facilitate coordination and communication.

Mitigating Unintended Consequences

The task force recognizes the potential unintended consequences of measurement, such as the overuse and misuse of high-stakes standardized tests, corruption and cheating, and the potential advantage for those who can afford tutoring and courses to boost test scores. Still, these consequences are not inevitable, and there are ways to address these concerns—for instance, ensuring that decisions on measurement are country-driven rather than prescribed at the global level; ensuring that data used for accountability are fair and accurate representations of education quality and learning; and ensuring that at least some test items are freely available to low and middle-income countries (e.g. a free bank of common items countries can incorporate into their existing assessments). In its ongoing work, the task force will continue to provide recommendations to mitigate the harmful effects of testing and focus on leveraging the power of better information to make decisions that improve learning for all children and youth.

Opportunities to Participate

As in the previous phase, LMTF 2.0 will continue to work through an open, multi-stakeholder approach. Education ministries interested in signing on as Learning Champions are invited to submit letters of interest by May 15, 2014. In the coming weeks, there will be additional opportunities for organizations and individuals to get involved as partners and working group members. Sign up to receive these updates by emailing [email protected].