Skip Navigation

Improving Science Assessments

Publications

  • Upgrading High-Stakes Assessments

    Because of the large number of students involved, statewide assessments used throughout the United States are designed to be highly efficient with respect to administration and scoring. This constrains the formats that can be used and thus limits testing to a subset of competencies typically associated with the education standards being assessed. CALA is developing an alternative approach that is designed to help expand the range of skills evaluated by statewide assessments and add a formative aspect to these assessments. Read the paper…

  • The Capabilities-Complexity Model

    In assessment, the ability to construct test items that measure a targeted skill is fundamental to validity and alignment. The ability to do the reverse is also important: determining what skill an existing test item measures. This paper presents a model for classifying test items that builds on procedures developed by others, including Bloom (1956) and Webb (2002). An advantage of the proposed model is that it references both the type of cognitive ability involved and the complexity of skill being assessed. This model is referred to as the Capabilities-Complexity Model. Read the paper…

  • Science Competencies That Go Unassessed

    Researchers at CALA are examining options for upgrading high-stakes, statewide assessments in middle school science. During the research, strategies will be devised for measuring critical science competencies that are not measured within conventional large-scale testing programs. Science Competencies That Go Unassessed identifies the seventh-grade science knowledge and skills implicit in Florida’s current Sunshine State Standards that cannot be measured by the Florida Comprehensive Assessment Test. The identified competencies will become the subsequent focus of our complex science assessments. Read the paper…

  • Specifications for Complex Science Assessments

    CALA is designing a series of performance assessment specifications for middle school science. Teachers and project researchers are using these specifications to independently design summative performance assessments that measure critical competencies not assessed by conventional large-scale tests such as the Florida Comprehensive Assessment Test. Teachers also are using the specifications to create separate assessments for formative use during instruction. Eight specifications have been developed during the first two years of the project, including sample performance assessments and scoring plans. View the specifications…

  • Enriching Assessment of the Core

    This is an invited presentation made at the September 2012 IES Principal Investigators' meeting in Washington, DC. The presentation was part of a panel session titled “Shifting Sands: How Do Researchers Respond to Changing Standards and Assessments.” The session focused on challenges researchers face as a result of the shifts in state standards and assessments, along with potential responses. The primary purpose of the present presentation is to describe a strategy being examined by one IES project for expanding the types of science competencies that are included in both large-scale assessments such as those administered statewide and classroom assessments that are developed and administered by teachers. The assessment strategy includes significant formative and summative components. View the presentation…

  • Reliability of Scores on the Summative Performance Assessments

    Reliability is concerned with the consistency of a measure. Examining reliability is important, in part, because the presence of inconsistencies indicates that something is influencing observations unrelated to what supposedly is being measured. This, in turn, represents a direct threat to an assessment’s validity. This paper reports results of reliability studies that were conducted during our project’s pilot year. Generalizability theory was used as the framework for these studies because of the flexibility this approach provides for examining sources of inconsistency within a complex assessment. Read the paper…

  • Types of Changes That Occur as Declarative Knowledge Increases

    Chi and Ohlsson (2005) propose seven types of changes that occur as declarative knowledge increases. As our project team develops middle-school science assessments, these changes provide our framework for selecting student tasks whenever the focus of the assessment is on declarative knowledge. A given assessment, however, typically addresses a subset of the seven types. This paper provides a summary description of each type of change and offers illustrations relevant to science education. Read the paper…

  • A Strategy for Large-Scale Science Assessment

    Our research seeks to determine the feasibility of a strategy that would expand the range of skills evaluated by large-scale assessment programs. Sampling, item specifications, and formative assessment training are designed to make performance measures more cost effective and the results more instructionally useful. This paper summarizes the first six months of the study, focusing on seventh-grade science competencies. Read the paper…