Donald J. Leu, Ph.D.
Director, New Literacies Research Lab and Professor of Education
University of Connecticut
ePIRLS: An International Assessment of Reading for New Times
We change the world when a child learns to read. Learning to read opens windows to the world and new opportunities for everyone. Despite our best efforts, far too many lack access to high quality reading instruction and many students continue to struggle, increasing the possibility that they will drop out of school. It is essential for all students to become fully prepared in reading so that they are able to succeed in school, fulfill individual goals, and make our world a better place through their accomplishments. Thus, reading has long been universally recognized as a core area of educational policy and instruction for every nation. Today, the Internet has changed what it means to become a reader. New educational policies and practices, based on data, are required to adapt to these changes.
The Changing Nature of Reading
The Internet is an inherently disruptive information and communication technology that has changed lives in profound ways in every nation. These changes affect both the nature of reading and the sources of information that we use for learning (Le Bigot & Rouet, 2007). In addition to being able to read traditional texts, we now require the skills, strategies, and practices that enable us to read and learn online. In response to this changing dynamic, IEA’s TIMSS & PIRLS International Study Center has developed a new assessment of online reading, ePIRLS.
ePIRLS is a performance-based assessment of students’ ability to read and learn online with school-based assignments in science and social studies. The ePIRLS assessment tasks are delivered on a computer and include lessons with web pages, information graphics, animations, multiple tabs, pop-up windows, and an avatar that guides students through the research tasks. The results from this new assessment of online reading and learning provides educators with needed information to guide policy in a rapidly changing area. Fourteen countries and two benchmarking participants joined ePIRLS 2016. As a result, they now have important data on how well their fourth grade students are prepared for the demands of online informational reading and learning.
The focus on students in their fourth year of schooling is a wise one, especially for the evaluation of online reading. A reasonably accurate aphorism that applies to reading is: “We learn to read in the first three to four years of school and we read to learn in subsequent years.” By focusing on fourth grade students, ePIRLS is the only international assessment to provide us with data situated at this crucial, developmental nexus for reading. The grade cohort is perfectly positioned to inform us about both aspects of reading development, especially preparation for the important task of reading to learn from online sources.
In addition to a new assessment of online reading and learning, ePIRLS also includes surveys of teachers and school leaders in participating countries. This triad of data provides policy makers with information essential to developing appropriate public policies in this new area of reading, thereby improving the well-being of their citizens in a digital age of online information.
Why should anyone invest a tremendous amount of effort, time, and resources to develop and use an assessment of online reading? There are many reasons.
- First, between 40 and 50 percent of the world’s population currently has access to the Internet (UNESCO, 2014). At the current rate of adoption, all, or nearly all, of the world’s population will have access in just eight more years. This means that children who are in fourth grade today will graduate from secondary school and enter a universal world of online information. To adequately prepare young students for this future requires that we gather information about the nature of their progress with reading in online contexts. This enables us to prepare them for the world of online information they will inherit.
- Second, students have increasing access to online information at home and on mobile devices and they use these often. Data show that students in some countries spend more time on computing devices than in books. Outside of school, students in the United States from ages 8 to 18 spend three times more time reading on a computing device than they spend reading traditionally printed pages offline (Rideout, Foehr, & Roberts, 2010). This and other developing trends in online reading (see, for example, Bråten, McCrudden, Lund, Brante, & Stømsø, 2017; Leu, et al., 2016) present challenges for educators as we look to support learners in a digital age (cf. Goodman, Sands, & Coley, 2015; Kirsch, Braun, Yamamoto, & Sum, 2007; Larson & Dwyer, 2015). Students are telling us what their reading lives are like so nations and classrooms can either adapt to this new reality or else become less relevant to the lives of this generation.
- Third, students are unskilled with reading information online to learn. With the development of ePIRLS, we recognize the need to plan for high levels of critical thinking in the reading of online texts, beginning with younger readers. Although today’s students grow up in an online world and are developing skills in gaming, social networking, media creation, and texting, research is showing how limited students’ skills are with online reading. They are not skilled at locating information online (Bilal, 2000; Guinee, Eagleton, & Hall, 2003; Kuiper & Volman, 2008) or critically evaluating it (Walraven, Brand-Gruwel, & Boshuizen, 2008). Many students find it difficult to judge the accuracy, reliability, and bias of information that they encounter during online research (Bennett, Maton, & Kervin, 2008; Graham & Metaxas, 2003). In fact, adolescents overgeneralize their ability to read and evaluate online information effectively, a perception informed by their ability to engage successfully with online social networking, texting, and video games (Kuiper & Volman, 2008).
- Fourth, research indicates that online reading comprehension is not isomorphic with offline reading comprehension (Afflerbach & Cho, 2010; Coiro & Dobler, 2007). Thus, the argument is no longer sustainable that we do not require additional reading assessments because online and offline reading are identical. Recent studies have shown important differences between online and offline reading. Afflerbach and Cho reviewed 46 studies that focused on reading strategy use during Internet and hypertext reading. Their analysis showed evidence of strategies that “appear to have no counterpart in traditional reading” (p. 217). Many strategies centered around a reader’s ability to apply methods to reduce their levels of uncertainty while navigating appropriate reading paths in a shifting problem space. Examples include the use of keywords and search engine results during reading and problem solving with online information. It also includes critically evaluating the reliability of online information using links and strategies not found with traditional text. By gathering information about the online reading ability of fourth grade students, we generate greater awareness and understanding of these differences and that allows us to introduce classroom experiences to develop proficiency in the additional areas required for online reading.
- Finally, issues of equity have become increasingly important and a separate online reading achievement gap is appearing. This is essential to consider at a time when just 62 individuals own the same wealth as the lower half of the world’s population (Oxfam, 2016). If we are to address issues of income inequality, appropriate performance data must be collected and equal opportunity for all students to learn to read online must be available in schools, even when access is not available at home. Data from ePIRLS will permit these important goals to be realized.
ePIRLS Results: 2016
The results from the first iteration of ePIRLS are already providing important direction for our work ahead in reading. They have provided us with an important new model for the assessment of online reading comprehension, one that according to students provides an engaging online simulation of school assignments in science and social studies. The use of a performance-based simulation is especially innovative and important. According to de Klerk, Veldkamp, and Eggen (2015), simulations have at least three advantages over other forms of assessment: engagement increases, thereby increasing flow (Csikszentmihalyi, 1991) and avoiding anxiety; knowledge application, rather than simple knowledge replication, is emphasized; and richer data can be captured, including process data.
A simulation of online reading and learning has another important function. ePIRLS permits educators to see what online reading looks like in action and shows them the additional new skills that may be needed by students for successful learning. This will play an essential role in helping teachers to understand the nature of online reading and assist them in expanding the nature of classroom reading instruction to include the development of these skills.
Early analysis indicates a number of ePIRLS results are important to consider. These include:
- Students liked taking the assessments. The percentage of students who liked each assessment activity ranged from 83 to 93 percent. This may provide additional support for the use of simulations during assessment since we are able to assess students’ optimal performance only when they are engaged.
- Students had little difficulty in managing the assessment. Students were able to navigate to a high percentage of the items, with almost all completing the assessment in the time allotted. Students reported a high degree of self-efficacy in computer use.
- Countries that reported higher mean ePIRLS scores compared to PIRLS scores were countries that traditionally do better with assessments of offline reading. This suggests that the benefits of good instructional contexts are likely to generalize to both types of reading.
- Girls appear to do better than boys in ePIRLS at both levels of comprehension. This suggests that technology advantages, often enjoyed by boys, may not apply to reading and learning with online information. There was no country where boys performed higher than girls in ePIRLS.
ePIRLS: New Opportunities and New Challenges
With the development of ePIRLS, combined with PIRLS, we now have a comprehensive set of tools to measure reading on an international scale and at an age point especially important to reading development, the fourth grade of schooling. The importance of this accomplishment needs to be highlighted. Being able to provide comprehensive information about reading performance in both online and offline contexts, at an important developmental point, is a major achievement and will continue to benefit policy decisions for nations. Thus, those countries participating in ePIRLS may be doubly advantaged in reading today as students must navigate in both online and offline reading contexts.
ePIRLS has also pointed us to new opportunities as nations consider ways in which to encourage greater uptake in technology and the sciences by girls and women. We have known that girls generally perform at high levels and are highly motivated by reading. Finding that this pattern also appears in the reading of online information, especially in science, may provide us with a new direction for thinking about the delivery of science instruction in ways that support high levels of performance and engagement by girls. This would enable many nations to enhance their scientific workforce in important ways.
ePIRLS also points us to new opportunities to understand optimal patterns of classroom instruction and home use of the Internet at a crucial period in the development of children. As we begin to understand these relationships better, we will be able to craft even more insightful means to measure instruction in online reading and home use of the Internet, providing valuable information about how best to support development.
An especially important opportunity has been made possible with the development of a performance-based assessment in this area, which has provided us with a new model for the assessment of reading. This will likely lead to new approaches to the assessment of student learning. Especially important in this regard is the observation that students, both boys and girls, liked working on the ePIRLS tasks. By sustaining high levels of enjoyment during assessment we are able to evaluate optimal levels of performance.
There are several challenges that also come with the changing nature of reading and reading assessment. Recent events in the world remind us of the special importance of being able to read critically and evaluate online information carefully. The Internet includes a diverse set of voices and lacks traditional gatekeepers for information. As a result, it challenges readers by requiring an especially high level of critical evaluation.
Our inability to evaluate online information has already been demonstrated to have serious political, social, and economic consequences. Thus, it is important that we continue to consider tracking the important area of students’ ability to critically evaluate the reliability of online information.
In addition, we will be challenged by the changes to reading that will continue to take place. The Internet changes the nature of what it means to read and learn online and it does so continuously. We have already seen that to be literate yesterday, in a world defined primarily by static book technologies, does not ensure that one is fully literate today. To be literate tomorrow will be defined by even newer technologies that have yet to appear and the new skills, strategies, and social practices these will require. ePIRLS has stepped boldly into this new, continually changing context, providing us with essential information about our future, the children who are in school today. It is a remarkable achievement and will be remembered as an important historical milestone.
References
Afflerbach, P.A., & Cho, B.Y. (2010). Determining and describing reading strategies: Internet and traditional forms of reading. In H.S. Waters & W. Schneider (Eds.), Metacognition, strategy use, and instruction (pp. 201–255). New York: Guilford.
Bennett, S. J., Maton, K., & Kervin, L. (2008). The ‘digital natives’ debate: A critical review of the evidence. British Journal of Educational Technology, 39(5), 775-786.
Bilal, D. (2000). Children’s use of the Yahooligans! Web search engine: I. Cognitive, physical, and affective behaviors on fact-based search tasks. Journal of the American Society for Information Science, 51(7), 646-665.
Bråten, I., McCrudden, M.T., Lund, E.S., Brante, E.W., & Stømsø, H.I.(2017). Task-oriented learning with multiple documents: Effects of topic familiarity, author expertise, and content relevance on document selection, processing, and use. Reading Research Quarterly. Advance online publication. doi: 10.1002/rrq.197
Coiro, J., & Dobler, E. (2007). Exploring the online reading comprehension strategies used by sixth-grade skilled readers to search for and locate information on the Internet. Reading Research Quarterly, 42(2), 214–257.
Csikzentmihaly, M. (1991). Flow: The psychology of optimal experience. New York: Harper Perennial.
de Klerk, S., Veldkamp, B.P., & Eggen, T.J. (2015). Psychometric analysis of the performance data of simulation based assessment: a systematic review and a Bayesian network example. Computers & Education, 85, 23-34.
Goodman, M., Sands, A.M., & Coley, R.J. (2015). America’s skills challenge: Millennials and the future. Princeton, NJ: The ETS Center for Research on Human Capital and Education. Retrieved from https://www.ets.org/s/research/30079/asc-millennials-and-the-future.pdf
Graham, L., & Metaxas, P. (2003). “Of course it’s true, I saw it on the Internet!”: Critical thinking in the Internet era. Communications of the ACM, 46(5), 70-75.
Guinee, K., Eagleton, M.B., & Hall, T.E. (2003). Adolescents’ Internet search strategies: Drawing upon familiar cognitive paradigms when accessing electronic information sources. Journal of Educational Computing Research, Annual, 2003.
Kirsch, I., Braun, H., Yamamoto, K., & Sum, A. (2007). America’s perfect storm: Three forces changing our nation’s future. Princeton, NJ: Educational Testing Service. Retrieved from www.ets.org/Media/Research/pdf/PICSTORM.pdf
Kuiper, E., & Volman, M. (2008). The Web as a source of information for students in K–12 education. In J. Coiro, M. Knobel, C. Lankshear, & D. Leu (Eds.), Handbook of research on new literacies (pp. 241–246). Mahwah, NJ: Erlbaum.
Larson, L., & Dwyer, B. (2015). Digging deeper with reader response: Using digital tools to support comprehension of literary texts in online learning environments. In T. Rasinski, K.E. Pytash, & R.E. Ferdig (Eds.), Using technology to enhance reading: Innovative approaches to literacy instruction (pp. 121–130). Bloomington, IN: Solution Tree.
Le Bigot, L., & Rouet, J.F. (2007). The impact of presentation format, task assignment, and prior knowledge on students’ comprehension of multiple online documents. Journal of Literacy Research, 39, 445–470.
Leu, D.J., Forzani, E., Rhoads, C., Maykel, C., Kennedy, C., & Timbrell, N. (2016). The new literacies of online research and comprehension: Rethinking the reading achievement gap. Reading Research Quarterly, 50(1). 1-23. Newark, DE: International Literacy Association. doi: 10.1002/rrq.85. Retrieved from http://www.edweek.org/media/leu%20online%20reading%20study.pdf
Oxfam. (2016). An economy for the 1%: How privilege and power in the economy drive extreme inequality and how this can be stopped. Oxford, UK: Oxfam International. Retrieved from https://www.oxfam.org/sites/www.oxfam.org/files/file_attachments/bp210-economy-one-percent-tax-havens-180116-en_0.pdf
Rideout, V., Foehr, U., & Roberts, D. (2010). Generation M2: Media in the Lives of 8- to 18-Year-Olds. Retrieved February 1, 2010 from http://www.kff.org/entmedia/mh012010pkg.cfm
UNESCO. (2014). The state of broadband 2014: Broadband for all. Geneva, Switzerland: United Nations. Retrieved from http://www.broadbandcommission.org/documents/reports/bb-annualreport2014.pdf
Walraven, A., Brand-Gruwel, S., & Boshuizen, H.P.A. (2009). How students evaluate information and sources when searching the World Wide Web for information. Computers and Education, 52(1), 234-246.