Assessment
Summary of revisions:
- CASAS, the U.S. Department of Education, and SBCTC/ABE require that all CASAS Cadre members be re-certified each year.
- An assessment policy section has been added to clarify the unallowable assessment practices that are considered to be improper, unethical, and fraudulent test use.
- NRS requires that all ABE providers “administer all pre- and post- assessments used to measure educational gain of distance education (and all other) students for NRS reporting in person, at a proctored program site within the state that” complies with the NRS and Washington state assessment policies. “Assessments not conducted through face-to-face interaction with a trained test administrator in a secure setting are not allowed for NRS reporting.”
- NRS requires that all attendance (instructional, contact) hours be verifiable actual time when the ABE student and ABE program staff actually interact, and through which the learner identity is verifiable. That contact can be in a classroom or learning lab, in a tutoring session, or, for distance learners, through telephone, video, teleconference or on-line communication with program staff, when the learner’s identity is verifiable.
- The CASAS POWER assessment series for adults with intellectual or developmental disabilities, has been removed from this assessment policy, since it cannot measure an educational functioning level (NRS) or WA Adult Learning Standards level completion. This assessment policy only discusses assessments usable for state and federal accountability reporting.
2003 Federal Audit Report and Impact on the WA State Assessment System
Audit findings
Required changes to assessment system – As part of the Adult and Family Literacy service provision, states that accept federal funding are subject to audit by staff sent by the federal Office of Vocational and Adult Education [OVAE] to gauge compliance with the requirements of their grants. In early June of 2003, Adult Basic Education at the State Board for Community and Technical Colleges was audited by a team consisting of federal OVAE staff and state directors from Wisconsin and Arizona. The team cited many commendable practices carried out by the state and at the organizations they visited, including ambitious efforts in the area of performance task development and authentic assessment. They also cited areas of non-compliance that require corrective action – specifically regarding standardization of the assessment system. Below is the assessment excerpt from their report:
Required Action 1: Establish Statewide Assessment Policies and Adopt Standardized Assessments
Washington State has demonstrated a long-standing commitment to identifying skills and competencies required of adult Washingtonians and to high-quality assessment that is aligned to those competencies. Washington has committed to the development of a performance assessment system and is attempting to use that system for its accountability reporting. The performance assessment system is not currently standardized for the National Reporting System as required by the national Data Quality Standards. The Statewide policy in place allows use of non-standardized performance tasks that do not meet federal guidelines. Local programs are allowed to use standardized tests or performance tasks in determining Educational Functioning Levels (EFL) and the policies in place do not specify the necessary conditions for administering the assessments. Therefore, the reliability of learning gain data in Washington falls short of meeting federal validity and reliability standards. Washington falls below the minimum acceptable data quality standards for the NRS. In its Corrective Action Plan (CAP), the State must:
- Develop a statewide policy that requires use of standardized assessments as defined by the NRS State Data Quality
Standards to at least the Superior Level of Quality for its submission of 2002-2003 data;
- Re-evaluate its NRS Data Quality Checklist in light of the findings pertaining to inadequate assessment policies
and procedures. The state must resubmit its checklist to reflect an accurate presentation of its data quality for FY
2001-2002 with a corresponding CAP as required by the Data Quality Standards.
- Federal Audit Report (full)
- Corrective Action Plan (CAP)
- NRS State Data Quality Standards
- NRS Implementation Guideline
Implications and Timeline
Standardized assessment is critical to the national effort to garner support from Congress and the public for increased funding of Adult and Family Literacy services. The data is also used to award federal incentive funds to high performing states. It is therefore a high priority for OVAE to ensure that all states meet the requirements of a standardized assessment system. Until Washington State meets these requirements, OVAE will not consider any data from Washington as valid or acceptable. Without acceptable data, state performance is invalid and federal funding is jeopardized. In consultation with the Council for Basic Skills (CBS), Adult Basic Education at SBCTC has launched an ambitious implementation effort to bring the state into compliance as quickly as possible. The features of this effort are briefly outlined below.
Federal Accountability Assessment Implementation
- Standardized test selection – With input from the Council for Basic Skills membership and its executive board, CASAS was selected for ABE and ESL appraisal, pre- and post-testing.
- Policy development for NRS compliance – By reviewing assessment policies from CASAS and other states,
and in consultation with the CBS compliance task group, the following policies were submitted to OVAE for approval.
- Training for test administration - A total of 12 CASAS test administration training sessions were carried out in late August and early September. These training's had two purposes: 1) to prepare as many provider staff as possible to administer CASAS tests as soon as possible and 2) to prepare designated staff – "cadre members" nominated by their directors – to be certified to train peers at their institutions and to serve as contact people for information regarding assessment system developments. The chief concerns in this effort are: capacity building, facilitating information flow, and consistency of processes required to achieve compliance status. A "job description" for cadre members is in development and meetings for further information and training are being scheduled for early fall.
- WABERS adaptation – Changes to the state assessment system coincide with planned changes to the WABERS system for data reporting. SBCTC Information Technology (IT) staff have been converting WABERS to an internet format and have been working closely with ABE at SBCTC and CASAS to streamline scoring and reporting for the new federal compliance (CASAS) assessments. In addition, the IT staff has re-designed the system for reporting placement, progress and completion that the State has been using for the last several years.
- Data Quality Standards – As indicated in the bulleted section of the report excerpt above, the state must establish policies and procedures in compliance with the NRS State Data Quality Standards checklist at the Superior Level. Establishing standardized testing and reporting are two aspects of this requirement, but the standards also include data collection and verification processes, data analysis, and targeted staff development practices. Achieving this part of compliance will require systemic development on both the state and local levels. ABE at SBCTC is working together with CBS directors to support implementation of these standards.
- Funding the change – Implementation of this new system is not going to come cheaply. In consultation with CBS compliance group members, ABE at SBCTC has identified costs that can most efficiently be born by centralized purchasing and those that must be born by individual programs. Funds available at the state level have been distributed to programs as special assessment awards to buffer the effects of these expenditures. Programs can also use their regular program, administration, and professional development funds. At the state level, this effort will require that all reserves be tapped and that funding for special projects may be reduced.
Top of page