Assessment Guidelines

---

January 1999

In an effort to provide guidance and support for these campus-based assessment projects, the WACC Student Outcomes Task Force has defined a set of guidelines for the development of Institutional Assessment Plans, including: 1) a series of potential questions to be addressed in an institutional assessment program, 2) some examples of current projects related to these questions, 3) general suggestions for implementing a successful outcomes assessment program, and 4) a brief selected bibliography of assessment-related resources.

These guidelines are designed to be a starting point for an ongoing campus wide conversation about institutional outcomes assessment activities. As you move forward in that discussion and your overall planning process, you are strongly encouraged to utilize the State Board staff – in particular, Bill Moore (360 704-4346) or Loretta Seppanen (360 704-4348) – to help your campus assessment team think through and design the Assessment Plans.

Establishing an Institutional Assessment Program

The design and implementation of a comprehensive institutional assessment program does not involve anything particularly mysterious or complex; it does, however, take time, energy and commitment on the part of the entire college community. Seen as a whole, institutional assessment can also be a large and overwhelming process, and is often avoided because of its apparent difficulty. The approach taken in these guidelines is to break the overall process down into discrete steps in the form of a series of fundamental questions about the educational mission of the college. It is not likely that a comprehensive institutional assessment program can be put into place with the time or funding available for 1990-91; rather, the expectation is that college faculty and administrators will collaborate to define the question area/s most immediately important to them and attempt to achieve some kind of reasonable consensus on initial responses.

These questions, and some examples of the ways specific institutions have addressed each area, are provided below. These examples are by no means comprehensive, but should give you some idea of how colleges around the state have taken the first steps toward establishing institutional outcomes assessment programs. The Task Force hopes that these examples will facilitate, not constrain, your thinking about possible assessment directions for your college in 1990-91 and beyond.

1.  Clarifying Goals

What is our college intending to accomplish? What are the learning goals we have for our students? More specifically, what should our students know or be able to do when they graduate from or leave our institution?

Defining these goals means determining the educational outcome dimensions that are most significant and appropriate for your college. For the goal statements at this step to be most functional for the overall assessment process, they should:

• explicate the purpose/mission of the college.
• be developed across several organizational levels.
• describe outcomes or results, not processes or treatments.
• be measurable in some fashion.
• establish targets for excellence as well as minimum standards.

Many colleges may have defined these learning goals already through internal planning processes, perhaps tied to accreditation. If such work has been done, there is no need to repeat it for the purposes of an assessment program.

Examples

Skagit Valley College is working through its existing general education committee to reach consensus on student outcomes related to general education, including researching the literature and other colleges, surveying students, and talking with representatives from business and industry.

Seattle Central Community College has used a Student Outcomes Task Force to create and present to the College as a whole a "Model of Student Learning" defining outcomes categories and dimensions for all of their students. The model is currently being reviewed and discussed by the College community.

Tacoma Community College is using a joint faculty/administrator General Education and Outcomes Assessment Committee to survey the literature, visit other campuses and model programs, attend conferences, and work with consultants in order to develop campus-level expertise in outcomes assessment as a way of laying the foundation for a comprehensive institutional assessment program.

Lower Columbia College has a campus-wide faculty committee working to define and assess the outcomes of general education, create a coherent general education curriculum based on those outcomes, and then pilot the assessment of the outcomes in a student "Capstone" experience.

2.  Defining Measures

How will our college know how well our students are doing in achieving these outcomes? In other words, what measures and means will be used to collect assessment data? By whom? When?

This area addresses basic methodological concerns about assessment prior to collecting any outcome data, and may include reviewing existing measures to see how well they meet your needs, and/or taking the time to develop specialized measurement tools tailored to your specific situation and outcome goals.

Examples

Yakima Valley College has created a Liberal Arts and Sciences Student-Faculty Congress and is using this group to define and measure institutional and student expectations and results/outcomes.

Spokane Falls Community College has formed a writing outcomes assessment team to identify the components of writing competence, pilot test the most promising methods of assessing these outcomes, and present a draft of the entire process to the general faculty.

Centralia College has established faculty/administrator teams meeting to define criteria and possibly develop measures for assessing student competencies in reading, writing, and quantitative skills linked to the outcome of successful progress in academic and vocational programs.

3.  Assessing Outcomes

What are students at our college learning? Are they achieving the outcome goals as we have defined them? How much progress are they making toward the specified learning goals? Alternatively, what are the consequences and/or outcomes of a community college education, goal-related or not?

This step begins the action, data-collection phase, gathering assessment information designed specifically to answer the outcome questions defined in question 1) or more generally to clarify the range of actual outcomes produced.

Examples

Seattle Community Colleges are involved in a project analyzing student transcripts and surveying graduates in an effort to identify patterns of courses and abilities, knowledge and attitudes characterizing successful transfer students, moving toward a district-wide definition of transfer outcomes.

Shoreline Community College, having previously constructed a general education plan defining twenty cross-curricular learning outcomes, is focusing on one specific aspect of those outcomes, using writing portfolios as a way of evaluating and tracking student writing proficiencies.

4.  Evaluating Learning Opportunities

Are changes in student performance or learning related to the college experience, and if so, how? How effectively are we providing the learning opportunities presumed to be necessary for students to achieve these outcomes? Are our learning processes working the way we expect them to work?

This question involves evaluating or validating specific aspects of the college curriculum or experience to determine specific effects or influences on student outcomes.

Examples

Pierce College is surveying faculty and assessing student writing samples to determine what impact, if any, its writing-intensive "W" courses have had on the writing abilities of students, and plans statewide distribution of a resource paper on their results.

Tacoma Community College, through a Tutor Program Advisory Committee composed of faculty, administrators, and tutors, is evaluating a campus-wide tutoring program to determine its influence on student academic success.

South Puget Sound Community College has established a joint faculty/administrator Student Success Strategies Committee and is surveying students and faculty attempting to define the basic skill needs and academic preparedness of students whose first language is not English, leading to an evaluation of the influence on student learning and satisfaction of a college-wide program designed for these students.

5.  Using the Results

How will findings about our students' outcomes be used to focus specifically on improvement at our campus? How can the assessment efforts, from design to data collection to reporting results, be incorporated into the ongoing operation of the college?

This step is crucial if the assessment efforts are to be taken seriously and considered meaningful. In general, answers to these questions should:

• involve a wide range of people.
• focus assessment on issues people care about.
• choose appropriate levels of analysis.
• provide small quantities of useful information rather than large amounts of data.
• emphasize uses for improvement rather than punishment.
• establish high expectations and meaningful incentives.

Examples

Most of the preceding examples include components addressing this area as well, but are too early in their implementation to offer details. Nationally, institutions with successful assessment programs are using their results in a variety of significant ways:

Alverno College in Wisconsin uses its assessment data in numerous ways, but perhaps most importantly as a means to increase students' success through feedback on their learning progress;

Northeast Missouri State University uses its results primarily as a feedback tool for departments and programs as a way to focus on curricular successes and areas for improvement;

Mt. Hood Community College in Oregon uses its assessment results in an overall internal strategic planning cycle, but also very successfully in external marketing efforts aimed at the local community.

General Suggestions for Implementing a Successful Outcomes Assessment Program

For colleges just beginning the process of thinking about outcomes assessment, following step-by-step the linear approach suggested by the preceding questions may work very well. On the other hand, how a college goes through the steps is not as important as making some kind of progress. The adage "Ready–Fire–Aim!" may well describe the most productive approach for your campus; that is, to struggle with the later questions about "what is?" and work backwards to the initial questions about goals. Regardless of how you approach the task, or the questions you ask first, outcomes assessment work around the state and the country suggests that the following elements seem to characterize successful programs:

Derived from a clear understanding of the college mission, its expectations of student progress, and its goal statements of desired student outcomes

Meaningful assessment efforts need to be driven by a genuine quest for understanding students better–by real questions about students and the teaching/learning process in a particular college community.

Built on existing assessment data and procedures

As Peter Ewell suggested at the May, 1990 Evaluation of Student Learning conference, one of the first steps in the assessment process is taking an inventory of existing data and points of contact with students to see what is currently available and how well these natural contacts are being utilized. Given the difficulties in involving students in "add-on" assessment requirements, there is growing interest nationally in "embedded" assessment efforts: through the admissions and registration processes, through existing course testing and evaluation activities, and so on. Successful programs demonstrate a commitment to assessment while attempting to minimize the burden of extra tasks on both faculty and students.

Supported by top-level leaders, strong faculty involvement and broad participation by constituent groups

At institutions with successful outcomes assessment activities, assessment is "everyone's business," not a project delegated to a single office or individual and then ignored. This kind of comprehensive approach requires both "top-down" and "bubble-up" efforts as well as active collaboration among faculty, administrators, and students.

Visible on campus; integrated into campus life and the mainstream of decision-making

While this characteristic does not occur overnight, successful assessment programs become a positive and expected part of the campus culture, through ongoing efforts to help all segments of the community understand the role and value of assessment activities to themselves and to the institution. Through faculty workshops, meetings with student groups, articles in the campus newspaper or other media outlets, the benefits of a quality assessment program can be promoted–including, to paraphrase Pat Hutchings, the questions and issues raised, the conversations fostered, and the information gleaned from assessment efforts.

Based on multiple indices of multiple outcome areas

Ultimately, the best assessment programs address a range of student learning outcomes with a variety of measurement approaches, acknowledging that college students and educational processes are not easily reduced to single dimensions.

Defined by context specific issues for each individual institution

Successful assessment programs are very much a reflection of the college or university culture in which they operate. A model program at one institution can generally not be transplanted without adaptation to another setting.

Established over a long period of time

Quality assessment efforts take time–for planning, discussion, building consensus, developing ownership, gathering data, revising procedures, and so on. As Bonnie Neumann of San Diego State said at the May conference, assessment is an evolutionary, not a revolutionary, process.

Reporting Requirements and Formats

The legislature provided dollars in the 1990 supplemental budget for institutional assessment but added the requirement that "the state board for community college education shall approve college-specific assessment plans before releasing funds to the individual community colleges." To comply with this requirement, we are asking you to submit a Preliminary Assessment Plan to the State Board by June 15. A more complete Institutional Assessment Plan is requested by November 15, 1990, and a combination Progress Report/1991-92 Assessment Plan is requested by May 15, 1991.

• The Preliminary Assessment Plan should be a concise (no more than two pages) document providing an overview of your plans for 1990-91, addressing the following questions in at most two to three paragraphs each:
  • What areas of student learning do you expect to address in your assessment efforts for 1990-91, and why?
  • How do you propose to approach the overall assessment planning process? (some examples: through an existing committee, the creation of a campus wide task force, a small work team drafting a proposal for a larger campus group, etc.) How do you see this planning process operating in 1990-91 and beyond?
  • Who (individuals or groups) will be involved in the assessment planning process on your campus? Describe your rationale for including them in the process.
  • Provide a description of how you anticipate spending the allocation based on your preliminary plans.
  • Designate a liaison person for the institutional assessment activities related to this allocation.
• These Preliminary Plans will be evaluated on the following criteria:
  • How well does the plan reflect an emphasis on student learning outcomes?
  • How well does the planning process involve a broad range of groups from the campus community?
• The plans will be reviewed prior to the June State Board meeting in order to seek the Board's approval for the appropriations. If questions arise in this review process, your institution's liaison person will be contacted for clarification and further information as needed.

• The Institutional Assessment Plan, due November 15, 1990, is the formal document defining your overall plans for assessment related to institutional improvement. This report should reflect a thoughtful college-wide process addressing the following questions, and also will offer an opportunity to make any appropriate revisions to the preliminary plan submitted in June:
• What question areas (as defined in the attached guidelines) do you intend to address? Describe what you intend to do with the given area, and how you plan to approach it.
• What is your timetable for addressing these questions?
• With respect to any assessment data-gathering proposed for 1990-91, what do you want to know about your students, and what measurement procedures will you use?
• As a result of the assessment activities funded by this allocation, what do you hope to learn about your students and/or your college?
• Based on what you think you may discover, what do you anticipate doing about it?
• Please provide a budget for the 1990-91 institutional assessment activities related to this funding, including a description of anticipated in-kind contributions on the part of the college.

• The Progress Report/1991-92 Assessment Plan will not be a lengthy report of data and analyses but will focus more broadly on your achievements related to institutional assessment and any revisions in your ongoing assessment plans for the following year. Additional details on the structure of this report will be provided at a later date, but the basic format will involve responses to three basic questions:
  • What exactly did you do in your 1990-91 assessment activities? Did you accomplish what you planned to accomplish?
  • What did you learn as a result of your 1990-91 assessment activities–about your students, your curriculum or other campus programs/services, about your college as a whole?
  • Based on what you've learned, what, if anything, are you doing about it? Ideally, what do you hope to do about it?
  • What are the next steps for your institutional assessment efforts for 1991-92?
  • Describe any changes to the budget breakdown proposed in the Fall plan for 1990-91. What are the budget implications of the 1991-92 plans defined in D?

These reporting deadlines are short, especially given the nature of the planning process being undertaken. Please, note, however, that what is being expected by the end of 1990-91 is not a fully operational comprehensive institutional outcomes assessment program, but a reasonable, good-faith effort toward planning and building such a program, with institutional improvement and student learning as the principal focus. The most important audience for your work in outcomes assessment, in 1990-91 and beyond, is your own campus community: faculty, administrators and students.

The reports described above will help the WACC Task Force and the State Board facilitate a climate in which you have some resources to pursue these areas in a thoughtful and institutionally meaningful way. These reports will be helpful to the Task Force and the State Board in understanding specific aspects of institutional assessment efforts, not to compile numbers and statistics. Please note that assessment scores are not being requested--only information on what you've learned in your planning/data gathering process.

Expenditures

The overall intent of the $60,000 allocation is to supplement not supplant current institutional activities related to assessment. The list below represents expenditures that the Task Force believes follow or fail to follow that intent:

Recommended

*Faculty time: Stipends for faculty involved in planning or implementing the college's assessment plan, or replacement costs for faculty on full or partial release time from teaching load to work on college assessment activities.

*Staff time: Cost of hiring additional staff to carry out institutional assessment activities or replace existing staff redirected to institutional assessment activities.

*Consultants: Fees for knowledgeable individuals to conduct faculty end staff development activities related to assessment.

*Travel: Support for faculty and staff investigation of successful assessment efforts, such as attending conferences or visiting colleges with significant achievements in assessment.

*Materials: Cost of improving the college's knowledge base about institutional assessment through publications; cost of conducting surveys or interviews or testing to evaluate student learning.

Not Recommended

*Program improvements: Activities that implement improvements in programs or services are more appropriately funded with quality enhancement dollars.

*Entry-level, basic skills testing: Basic skills testing for the purpose of advising and placing individual students in particular courses may be funded with quality enhancement dollars.

*Equipment: Equipment may be funded with your equipment allocation or quality enhancement dollars.

Selected References: College Outcomes Assessment

Adelman, C. (Ed.) (1988). Performance and judgment: essays on principles and practice In the assessment of college student learning. Washington, D.C.: Office of Educational Research and Improvement.

Alverno College Faculty (1979). Assessment at Alverno College. Milwaukee, Wl:Alverno Productions.

Astin, A. (1985). Achieving educational excellence: a critical assessment of priorities and practices in higher education. San Francisco: Jossey-Bass.

Banta, T.W. (Ed.). Implementing outcomes assessment: promises and perils. New Directions for Institutional Research, #59. San Francisco: Jossey-Bass.

Bok, D. (1986). Higher learning. Cambridge, MA: Harvard University Press.

Educational Testing Service (1987). Assessing the outcomes of higher education: proceedings of the 1986 invitational conference. Princeton, NJ: Educational Testing Service.

Ewell, P. (1989). "About halfway: assessment et the balance point." Assessment Update, 1 (1), 1-7.

Gray, P. J. (Ed.) (1989). Achieving assessment goals using evaluation techniques. New Directions for Higher Education, #67. San Francisco: Jossey-Bass.

Grossman, G.M. & Duncan, M.E. (1988). Assessing the institutional effectiveness of community and technical colleges. Columbus, Ohio: National Center for Research in Vocational Education. ERIC Document Reproduction Service (ED 303 193).

Halpern, D.F. (Ed.) (1987). Student outcomes assessment: what institutions stand to gain. New Directions for Higher Education, #59. San Francisco: Jossey-Bass.

Hanson, G.R. (1982). Measuring student development. New Directions for Student Services, #20. San Francisco: Jossey-Bass.

Hutchings, P. (1989). "Behind outcomes: contexts and questions for assessment." Assessment Forum resource paper, June, 1989. Washington, D.C.: American Association for Higher Education.

Jacobi, M., Astin, A., & Ayala, Jr., F. College student outcomes: a talent development perspective. Higher Education Report #7. Washington, D.C.: ASHEERIC.

Kinnick, M.K.,Westine, J., & Kemper, K. (1987). Beyond "front door" access: attaining the bachelor's degree. Paper presented at the 27th annual forum of The Association for Institutional Research, Kansas City, Missouri, May, 1987.

Klemp, G. (1982). "Assessing student potential: an immodest proposal." In C Taylor (Ed.), Diverse student preparation: benefits and issues. New Directions for Experiential Learning, #17.San Francisco: Jossey-Bass.

McClenney, K.M. (1389). "Community college assessment: premises and consequences." Assessment Update, 1 (4), 7-9.

McMillan, J.(Ed.) (1989). Assessing students' learning. New Directions for Teaching and Learning, #34. San Francisco: Jossey-Bass.

Mentkowski, M. (1988) "Paths to integrity: educating for personal growth and professional performance." In S. Srivasta & Associates, Executive integrity: the search for high human values in organizational life (pp. 89-121). San Francisco: Jossey-Bass.

Pascarella, E.T., Smart, J.C., & Ethington, C.A. (1986). "Long-term persistence of two-year college students." Research In Higher Education, 24, 47-71.

Paskow, J. (1988). Assessment programs and projects: a directory. Washington, D.C.: MHE Assessment Forum.

Study Group on the Conditions of Excellence in American Higher Education. (1984). Involvement in learning: realizing the potential of American higher education. Washington, D.C.: National Institute of Education.

Terenzini, P. (1989) Assessment with open eyes: pitfalls in studying student outcomes." Journal of Higher Education,60(6), 644-664.

Vaughan, G.B. & Templin, R.G. (1987). "Value added: measuring the community college's effectiveness." The Review of Higher Education, 10 (3), Spring, 1987, p. 235-245.

Voorhees, R. (1987). "Toward building models of community college persistence: a logit analysis." Research In Higher Education, 26,115-129.

Waileri, R. (1987). A longitudinal study of ‘guided study’ students. Paper presented at the 27th annual forum of The Association for Institutional Research, Kansas City, Missouri, May, 1987.

Top of page