FC: BEYOND THE BUBBLE By Bill Tucker CliffsNotes
1: Table of Contents A Decade of Incremental Progress Promising Models - Testing Complex Skills - Recording More Data - Linked to Classroom Instruction - Right Here in River City Aligning all the Parts - Standards and Cognitive Models - Effective Instruction - Infrastructure and Equity
2: A Decade of Incremental Progress Technology could enable educational testing to reinvent itself in three stages: increase efficiency by automating existing processes; test questions, formats for response, and scoring would become more sophisticated due its ability to measure skills more comprehensively; and testing could be merged with instruction, which would allow teachers & students to use feedback from testing to adjust teaching to improve student achievement. Over the past two decades, there has been incremental progress in the possibility of using technology-based innovations in student assessment. Despite facing roadblocks with state budget shortfalls and NCLB, experts believe that testing will increasingly be delivered via computer and the Internet for several reasons.
3: With Internet-based testing, physical distribution, storage, data entry and scanning, and collection of test booklets and materials will be eliminated. Therefore, states will be able to analyze and distribute test results at a faster rate. The test will also produce a more realistic estimate of a student's achievement because the test will adapt to a student's skill level. Results from the test will be immediately available for teachers, administrators, and district officials, so instruction can be tailored to the students' needs.
4: Testing Complex Skills One of the largest efforts to pilot new forms of technology-based assessment is the Problem Solving in Technology-Rich Environments (TRE) project. Launched in 2003, when a sample of 2,000 students participated in a study to explore how information technology could be incorporated into the National Assessment of Educational Progress, TRE's goal was to create a form of testing that would assess a student's ability to solve a real-world problem. TRE tests a student's ability to find information about a given topic, judge what information is relevant, plan and conduct experiments, monitor one's efforts, organize and interpret results, and communicate a coherent interpretation. This promising research project is one of a number of technology-enabled assessment systems that could transform testing in the future.
5: Recording More Data TRE permits students to take various approaches and even allows students to test multiple solutions through a simulation exercise. Databases can record descriptive data about strategies used and actions taken by students. This data provides a greater range of information, which allows instructors to make better judgments about students approaches, challenges, and performance. Linked to Classroom Instruction TRE assesses both process and content through simulated exercises, and this requires assessments that are closely linked with classroom instruction. Unfortunately, the NAEP cannot assume that all students in the nation's 14,000 school districts have all covered the same science content. There are other projects, like the Caliphers project, that is trying to develop a high-quality, affordable assessment that can be used for large-scale testing and in the classrooms to inform instruction.
6: Right Here in River City Simulations provide an opportunity to embed assessment in the learning process. The River City project is a virtual environment where middle-school students explore a mysterious illness in a turn-of-the-century town. Students become scientists in this virtual world. The project focuses on inquiry practices, student observations, "chats" with townspeople, development of hypotheses and experiments to determine the cause of the epidemic. The goal for River City researchers is to be able to present data about what students are doing in the virtual environment in a way that helps teachers organize and individualize instruction; currently, teachers used traditional methods of assessment. If the researchers reach their goal, they will be able to use data to create performance-based summative assessments that will be valid, reliable, and cost-effective. Cisco Networking Academy is working on a project called Packet Tracer, which is attempting to embed assessment into learning.
7: Aligning All the Parts Education is a complex and decentralized public sector system, funded and governed at multiple levels. As a result, successful changes to assessment will require parallel, and equally challenging, revisions to standards, curriculum, instruction, and teacher training. Standards and Cognitive Models Cognitive research stresses the importance of aligning assessments with curriculum and instruction and the need to base testing on a model of cognition and learning. Today, most state standards focus on discrete sets of disconnected facts. Currently, our assessments do not align with what we know about how students learn and do not tell us enough about how to help students do better. This disconnect is most visible in science education. Cognitively Based Assessment of, for and as Learning (CBAL) is a project that is trying to bridge this disconnect. This cognitive model uses both summative and formative assessment, which enables
8: researchers to begin differentiating students and instructional responses to those students based on their performance on the CBAL assessments. Effective Instruction The most useful classroom-level applications of technology-enabled assessments provide detailed data that give educators better information about how students are progressing and why they are performing at their current levels. Also, they provide insight into what changes to instruction might be the most effective. In order for theses assessments to be effective, teachers must have deep content expertise and specific skills to understand these sophisticated assessments. Furthermore, Andrew Boyle, a leading researcher on technology-enabled assessment, feels that "at the present time, sophisticated tasks for e-assessment may be characterized as expensive and slow to develop, and not easily written by a non-specialist teacher." Creating high-quality assessments and making them adaptable by teachers for easy classroom use is the foreseeable challenge.
9: Infrastructure and Equity Besides the teacher-quality challenge, most technology infrastructures are still insufficient for advanced, large scale, Internet-based testing. The NAEP faced serious challenges in 2008 when its initial testing of computer-based questions for the 2009 Science Assessment. NAEP relied on schools' computers and technology infrastructure to administer the exams, but due to a wide variety of problems with the schools' hardware, software configurations, and Internet access, technical problems impeded half the students in the first weeks of the trial. Eventually the NAEP had to bring in its own laptops to complete the case study. Furthermore, disadvantaged (low income) students have limited access to computers, which will be another hurdle that will need to be overcome. The NAEP technology-enabled assessment projects, Math Online and Reading Online, found that students with greater exposure to computers outside of school scored higher. Some studies show no difference between paper and computer-based tests, which indicates that more research needs to done to
10: Changing Course It will be difficult to overcome these barriers. It will also be a challenge to help all students reach challenging standards for learning as well as be prepared for future success. Educators must adapt to reach these goals by having access to better data and also, having a deeper knowledge of how students learn. It is important for us to change the course of assessment. We, now, have the tools to create improved assessment systems and practices. The NAEP's main goal is to "go beyond what can be measured using paper-and-pencil." Unfortunately, changes in assessment affect our entire educational system and infrastructure, so we will not see the benefits from technology-enabled assessments without attention from policymakers and deliberate strategies to create change. In the meantime, policymakers, educators, and a various stakeholders need to take steps immediately to ensure that progress continues to move quickly in the next decade.