UPPS 01.03.04 - Outcomes Assessment for Continuous Improvement

Outcomes Assessment for Continuous Improvement

UPPS No. 01.03.04
Issue No. 4
Effective Date: 7/15/2024
Next Review Date: 6/01/2029 (E5Y)
Sr. Reviewer: Assistant Vice Provost for Program Accreditation and Assessment and SACSCOC Liaison

POLICY STATEMENT

Texas State University is committed to establishing and maintaining a systemic process of evaluating and enhancing the quality and effectiveness of programs, services, and operations that ensures alignment with institutional goals, mission, and strategic objectives while fostering accountability, innovation, and growth.

SCOPE
1. As indicated in the mission statement and shared values, Texas State University values continued reflection and evaluation to ensure that our strengths as a community always benefit those we serve. Outcomes assessment is an essential function for gathering critical information used to inform decision-making leading to continuous improvement of programs and services.
2. Texas State complies with the regional accrediting body’s compliance standards relative to Institutional Planning and Effectiveness (Section 7) and Student Achievement (Section 8), and in accordance with Standards 7.3 and 8.2. a, b, and c of The Principles of Accreditation: Foundations for Quality Enhancement of the Southern Association of Colleges and Schools Commission on Colleges (SACSCOC):
  
  “The institution identifies expected outcomes of its administrative support services and demonstrates the extent to which the outcomes are achieved.” (7.3 Administrative effectiveness)
  
  “The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of seeking improvement based on analysis of the results for student learning outcomes for each of its educational programs.” (8.2.a Student outcomes: educational programs)
  
  “The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of seeking improvement based on analysis of the results for student learning outcomes for collegiate-level general education competencies of its undergraduate degree programs.” (8.2.b Student outcomes: general education)
  
  “The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of seeking improvement based on analysis of the results for academic and student services that support student success.” (8.2.c Student outcomes: academic and student services).
3. Texas State has an established comprehensive outcomes assessment program that identifies and assesses the attainment of outcomes for administrative support services and student achievement in educational programs, general education, and academic and student services.
4. This policy describes the process by which Texas State schedules, conducts, reviews, and approves assessment activities to ensure data are collected and used for seeking improvement.
5. For Texas State’s purposes, outcomes assessment activities are expected to be informative and not punitive, and are conducted predominantly for the purposes of enhancing student achievement and improving programs and services. The focus should be placed on the area, rather than singling out individuals.
6. Texas State encourages each area to employ methods that will provide indicators for improvement. Texas State acknowledges that this may involve submitting results with low success rates with respect to the target in question and considers this a mark of effective assessment. The aim is not to show that the area is performing well. Rather, the aim is to determine how instruction and services might be improved.
7. The annual assessment cycle followed at Texas State includes two phases (Phase I – Planning and Phase II – Results Reporting). In early fall of each year, assessment plans are developed or revised to include the mission statement, outcomes, methods, and measures of assessment. In late spring for student outcomes and summer for administrative outcomes, results, plans of action, and evidence of improvement are reported.
8. All outcome assessment plans are maintained in an electronic repository, the Texas State Outcomes System. Reports are accessible to all Texas State faculty and staff with a user ID. Refer to the Outcomes Assessment website for specific cycle due dates, tools, and other pertinent information.
DEFINITIONS
1. Action Plans – future steps to be taken toward attaining continuous improvement based on the reported results.
2. Administrative Department – includes those that directly report to a unit head.
3. Area – includes all responsible for providing assessments (i.e., office, department, unit, school, college, center, or institute, depending on the type of outcome).
4. Assessment – in higher education, includes gathering information in a prescribed manner and interpreting that information to make decisions about policy, curriculum, instruction, initiatives, or assessment practices.
5. Evidence of Improvement – current performance level achieved as compared to previous levels achieved, expressed in specific terms; intended to present result patterns from semester to semester, year to year, etc.
6. Measures – observations of achievement reported in numeric terms.
7. Methods – strategies, along with tools or instruments, used to gauge progress toward attaining outcomes. Method statements should clearly indicate who will conduct the assessment; who will be assessed; when, where; and how it will be assessed; and what specifically will be assessed.
  1. Direct – assessment tools that evaluate actual performance. Examples include exam scores, portfolios, observations, records of error, time, cost, efficiency, and productivity.
  2. Indirect – assessment tools that evaluate perceived learning or performance based on opinions. Examples include student or employer perceptions, perceived satisfaction levels, perceived timeliness, and perceived capability.
8. Outcome – the results that an area desires to attain, expressed in general terms.
9. Results – the findings amassed from the corresponding method.
10. Targets – desired level of performance or attainment expressed in specific terms.
11. Unit – includes areas that directly report to a vice president or special assistant to the president.
ADMINISTRATIVE SUPPORT SERVICES OUTCOME PROCEDURES
1. Administrative support service outcomes identify core functions for providing effective and efficient administrative services in support of the academic mission of the university. Administrators for each identified administrative support service area will lead staff discussions to gain collective input and consensus to create and assess appropriate, measurable expected outcomes for each administrative support service area.
2. Areas that directly report to a vice president or special assistant to the president (units), as well as those that directly report to unit heads (administrative departments) as demonstrated on official published division organizational charts, are required to identify and measure administrative support service outcomes.
3. Phase I consists of creating a system for assessment of administrative support service outcomes. Phase I begins with a thorough review of the area’s mission. Mission statements should be:
  1. brief and clearly stated;
  2. distinctive to the area;
  3. reflective of the core purpose, primary functions and activities of the administrative support service area; and
  4. consistent with the mission of the university, division, unit, department, and office.
4. Once the mission is in place, units with 10 or more full time staff members should identify at least two measurable outcomes specific their administrative support service. For units with less than 10 full time staff, only one measurable outcome is required. Units are free to report on more than the minimum number of outcomes, if they desire. These outcomes should align with the mission and primary functions of the administrative support service area. Ideally, these outcome statements should reflect continuous improvement of services. Outcomes:
  1. focus on issues pertinent and important to the administrative support service area as specifically defined in the mission statement;
  2. focus on key areas where outcomes assessment will enable the administrative support service area to improve, emphasizing the end result and not the means to get there; and
  3. clearly articulate the intended result or action desired.
5. Each identified outcome should include two methods of measurement that will be used to assess achievement of the corresponding outcome. Multiple methods of measurement are necessary to build reliability and validity. Direct methods are preferred. Method statements should:
  1. align with the intent of the outcome;
  2. apply to service, regardless of location or mode of delivery;
  3. allow for the collection of accurate, valid, and reliable data for each method and measure in a reasonable and efficient manner within the academic year;
  4. provide detail on who will conduct the assessment; who will be assessed; when, where, and how it will be assessed; and what specifically will be assessed; and
  5. clearly indicate the desired level or target of performance.
6. Once the outcomes, methods and measures are in place, a system should be developed for implementation and assessment. This system should identify:
  1. how, where, and when the service will be provided;
  2. who will be involved in the implementation and measurement process;
  3. what instruments and scoring rubrics will be needed to quantify results;
  4. when the services will be measured; and
  5. how the data will be evaluated.
7. Phase II consists of collecting and analyzing the assessment data. Assessment results should be reported in a brief, but detailed summary for each method and measure. Discussions of findings should include:
  1. a reflection upon and discussion of the findings amassed from the corresponding method and measure;
  2. results based on reliable and valid data collection methods;
  3. a level of achievement of performance targets;
  4. indicators for further improvement that can be gleaned from the results;
  5. evidence of achievement related to the outcome aggregated for all services and disaggregated based on mode or location of delivery and other pertinent factors; and
  6. effectiveness of previous action plans for attaining results.
8. Based on the assessment results, a plan of action should be developed. Action plans should be feasible considering the availability of time and resources, and should lead to continuous improvement of services.
9. A synopsis of year-to-year evidence of improvement gained from actions taken based on previous outcomes assessments should also be provided. This evidence of improvement should focus on continuous improvement of services and should be documented with improvements cited in the assessment results.
10. Once completed, outcomes reports are reviewed and approved by the appropriate unit head and vice president.
11. At the end of each phase of the assessment cycle, outcomes reports are audited by the Institutional Effectiveness staff and published on the Outcomes System. Audit reports provide useful feedback for improving outcomes reports.
EDUCATIONAL PROGRAM OUTCOME PROCEDURES
1. Educational program student learning outcomes identify the knowledge, skills, and attitudes that students are expected to demonstrate by the end of their course of study in an educational program. Chairs, directors, and program coordinators for each educational program will lead faculty discussions to gain collective input and consensus to create and assess appropriate, measurable student learning and administrative outcomes for each program.
2. All educational programs identified by a distinctive four-digit CIP code are required to identify and measure student learning and administrative outcomes.
3. Phase I consists of creating a system for assessment of educational program outcomes. Phase I begins with a thorough review of the program mission. Mission statements should be:
  1. brief and clearly defined;
  2. distinctive to the program;
  3. reflective of the core purpose, primary functions, and activities of the program; and
  4. consistent with the mission of the university, college, and department or school.
4. Once the mission is in place, identify at least three measurable student learning outcomes for undergraduate programs, three for graduate programs, as well as two administrative outcomes specific to the educational program. These outcomes should align with the mission and overall goals of the program, as well as reflect the level of the program. Ideally, the student learning outcome statements should demonstrate higher levels of cognitive, psychomotor, and affective learning domains. Outcomes:
  1. describe the knowledge, abilities, and attitudes of a successful graduate;
  2. focus on issues pertinent and important to the program;
  3. reflect on learning throughout the program curriculum rather than through elective courses or extracurricular activities;
  4. focus on student learning rather than instructional or administrative processes; and
  5. focus on key areas where outcomes assessment will enable the program to improve.
5. Each identified outcome should include two methods of measurement that will be used to assess achievement of the corresponding outcome. Multiple methods of measurement are necessary to build reliability and validity. For student learning outcomes, direct methods are required. Method statements should:
  1. effectively and accurately reflect learning addressed in the outcome;
  2. align with the type and level of learning indicated in the outcome;
  3. apply to multiple sections of the same course, regardless of location or mode of delivery;
  4. allow for the collection of accurate, valid, and reliable data for each method and measure in a reasonable and efficient manner within the academic year;
  5. provide detail on who will conduct the assessment; who will be assessed; when, where, and how it will be assessed; and for student learning outcomes what specific knowledge, skill, value, or attitude will be assessed, or for administrative outcomes, what specifically will be assessed; and
  6. clearly indicate the desired or target level of performance.
6. Once the outcomes, methods and measures are in place, a system should be developed for implementation and assessment. This system should identify:
  1. how and where the concepts, skills, and values are being taught;
  2. who will be involved in the implementation and measurement process;
  3. how individual student learning will be measured;
  4. what instruments (i.e., exams, assignments) and scoring rubrics will be needed to quantify student learning;
  5. when the learning will be measured; and
  6. how the data will be evaluated.
7. Phase II consists of collecting and analyzing assessment data. Assessment results should be reported in a brief, but detailed summary for each method and measure. Discussions of findings should include:
  1. a reflection upon and discussion of the findings amassed from the corresponding method and measure;
  2. results based on reliable and valid data collection methods;
  3. level of success in achieving the desired performance target;
  4. indicators for further improvement that can be gleaned from the results;
  5. evidence of learning related to the outcome aggregated for all sections of the course and disaggregated based on delivery, location, mode, and other pertinent factors;
  6. student accomplishments and success;
  7. improvement over previous assessment cycles; and
  8. effectiveness of previous action plans for attaining results.
8. Based on the assessment results, a plan of action should be developed. Action plans should be feasible considering the availability of time and resources, and should lead to continuous improvement in student learning.
9. A synopsis of evidence of improvement gained from actions taken based on previous outcomes assessments should also be provided. This evidence of improvement should focus on continuous improvement in student learning and should be documented with improvements cited in the assessment results.
10. Once completed, outcomes reports are reviewed and approved by the appropriate department chair or school director and college dean.
11. At the end of each phase of the assessment cycle, outcomes reports are audited by members of the Council of Associate Deans for Academic Assessment and published on the Outcomes System. Audit reports provide useful feedback for improving outcomes reports.
GENERAL EDUCATION OUTCOME PROCEDURES
1. General education student learning outcomes identify the core components and competencies that students are expected to demonstrate by the end of their course of study. Members of the General Education Council’s (GEC) Assessment Subcommittee will lead faculty discussions to gain collective input and consensus to create and assess appropriate, measurable student learning outcomes for general education.
2. The GEC, with the guidance of the GEC Assessment Subcommittee, will develop and assign a mission statement for each component. The GEC will also develop an outcome for each component and competency and assign outcomes to courses approved in each component.
3. All academic units providing general education core courses are required to measure general education student learning outcomes as identified by the GEC.
4. Phase I consists of creating a system for assessment of general education student learning outcomes and includes a review of the mission statement and outcomes assigned to the approved general education course.
5. For each identified outcome, a method of measurement should be identified to assess achievement in student learning. Direct methods are expected. Method statements should:
  1. effectively and accurately reflect learning addressed in the outcome;
  2. apply to all sections of the same course, regardless of location or mode of delivery;
  3. allow for the collection of accurate, valid, and reliable data for each method and measure in a reasonable and efficient manner within the academic year;
  4. provide detail on who will conduct the assessment; who will be assessed; when, where, and how it will be assessed; and what specific knowledge, skill, value, or attitude will be assessed; and
  5. clearly indicate the desired level or target level of performance that is, at a minimum, 70 percent achievement.
  It is preferable for each outcome to have its own distinct method of assessment. One method should not be used to assess all of the competency outcomes. If more than one outcome is being assessed by the same exam, the method should describe which sections or questions on the exam apply to the particular outcome being assessed.
6. Once the outcomes, methods and measures are in place, a system should be developed for implementation and assessment. This system should identify:
  1. how and where the concepts, skills, and values are being taught;
  2. who will be involved in the implementation and measurement process;
  3. how individual general education student learning will be measured;
  4. what instruments (i.e., exams, assignments) and scoring rubrics will be needed to quantify student learning;
  5. when the learning will be measured; and
  6. how the data will be evaluated.
7. Phase II consists of collecting and analyzing assessment data. Assessment results should be reported in a brief, but detailed summary for each method and measure. Discussions of findings should include:
  1. a reflection upon and discussion of the findings amassed from the corresponding method and measure;
  2. results based on reliable and valid data collection methods;
  3. level of success in achieving the desired performance target;
  4. indicators for further improvement that can be gleaned from the results;
  5. evidence of learning related to the outcome aggregated for all sections of the course and disaggregated based on delivery, location, mode, and other pertinent factors;
  6. student accomplishments and success;
  7. improvement over previous assessment cycles; and
  8. effectiveness of previous action plans for attaining results.
8. Based on the assessment results, a plan of action should be developed. Action plans should be feasible considering the availability of time and resources and should lead to continuous improvement in general education student learning.
9. A synopsis of evidence of improvement gained from actions taken based on previous outcomes assessments should also be provided. This evidence of improvement should focus on continuous improvement in general education student learning and should be documented with improvements cited in the assessment results.
10. Once completed, outcomes reports are reviewed and approved by the appropriate department chair or school director and the vice provost for Academic Innovation and Success on behalf of the GEC.
11. At the end of each phase of the assessment cycle, outcomes reports are audited by the appropriate member of the GEC’s Assessment Subcommittee and published on the Outcomes System. Audit reports provide useful feedback for improving outcomes reports. The GEC Assessment Subcommittee also conducts holistic reviews of assessment for each component and competency on an established schedule.
ACADEMIC AND STUDENT SERVICES OUTCOME PROCEDURES
1. Academic and student service outcomes identify core functions for providing academic and student services in support of student learning. Directors and administrators for each identified area across applicable divisions (e.g., Student Success, Academic Affairs) will lead faculty and staff discussions to gain collective input and consensus to create and assess appropriate, measurable outcomes for each academic and student service area.
2. All academic and student service areas are required to identify and measure academic and student service outcomes.
3. Phase I consists of creating a system for assessment of academic and student service outcomes. Phase I begins with a thorough review of the area’s mission. Mission statements should be:
  1. brief and clearly stated;
  2. distinctive to the area;
  3. reflective of the core purpose, primary functions and activities of the academic and student service area; and
  4. consistent with the mission of the university, division, unit, department, and office.
4. Once the mission is in place, units with 10 or more full-time staff will identify at least two measurable outcomes specific to the academic and student service area. Units with fewer than 10 full-time staff will identify a minimum of one outcome. Units are free to assess more outcomes as they see fit. These outcomes should align with the mission and primary functions of the academic and student service area. Ideally, these outcome statements should reflect continuous improvement of services at impacting student learning. Outcomes:
  1. describe impact of academic and student service on student success;
  2. focus on issues pertinent and important to the academic and student service area as specifically defined in the mission statement;
  3. focus on key areas where outcomes assessment will enable the academic and student service area to improve emphasizing the end result and not the means to get there; and
  4. clearly articulate the intended result.
5. Each identified outcome should include two methods of measurement that will be used to assess achievement of the corresponding outcome. Multiple methods of measurement are necessary to build reliability and validity. The use of direct methods is encouraged, when and where they are feasible. Method statements should:
  1. align with the intent of the outcome;
  2. apply to academic or student service, regardless of location or mode of delivery;
  3. allow for the collection of accurate, valid, and reliable data for each method and measure in a reasonable and efficient manner within the academic year;
  4. provide detail on who will conduct the assessment; who will be assessed; when, where, and how it will be assessed; and for student learning outcomes, what specific knowledge, skill, value, or attitude will be assessed; and
  5. clearly indicate the desired level or target level of performance.
6. Once the outcomes, methods and measures are in place, a system should be developed for implementation and assessment. This system should identify:
  1. how, where, and when the service will be provided;
  2. who will be involved in the implementation and measurement process;
  3. how impact of service on student success will be measured;
  4. what instruments and scoring rubrics will be needed to quantify results;
  5. when the services impacting student success will be measured; and
  6. how the data will be evaluated.
7. Phase II consists of collecting and analyzing the assessment data. Assessment results should be reported in a brief, but detailed summary for each method and measure. Discussions of findings should include:
  1. a reflection upon and discussion of the findings amassed from the corresponding method and measure;
  2. results based on reliable and valid data collection methods;
  3. level of success in achieving the desired performance targets;
  4. indicators for further improvement that can be gleaned from the results;
  5. evidence of learning related to the outcome aggregated for all services and disaggregated based on mode or location of delivery and other pertinent factors;
  6. student accomplishments and success, when possible;
  7. improvement over previous assessment cycles; and
  8. effectiveness of previous action plans for attaining results.
8. Based on the assessment results, a plan of action should be developed. Action plans should be feasible considering the availability of time and resources, and should lead to continuous improvement of academic and student services.
9. A synopsis of improvement gained from actions taken based on previous outcomes assessments should also be provided. This evidence of improvement should focus on continuous improvement in academic and student services and its impact on student success and should be documented with improvements cited in the assessment results.
10. Once completed, outcomes reports are reviewed and approved by the appropriate unit head and vice president.
11. At the end of each phase of the assessment cycle, outcomes reports are audited and published on the Outcomes System. Audit reports provide useful feedback for improving outcomes reports.
REVIEWERS OF THIS UPPS
1. Reviewers of this UPPS include the following:
  
  Position Date
  
  Assistant Vice Provost for Program Accreditation and Assessment and SACSCOC Liaison June 1 E5Y
  
  Vice Provost for Academic Innovation and Success June 1 E5Y
CERTIFICATION STATEMENT

This UPPS has been approved by the following individuals in their official capacities and represents Texas State policy and procedure from the date of this document until superseded.

Assistant Vice Provost for Program Accreditation and Assessment and SACSCOC Liaison; senior reviewer of this UPPS

Vice Provost for Academic Innovation and Success

Provost and Executive Vice President for Academic Affairs

President

Position	Date
Assistant Vice Provost for Program Accreditation and Assessment and SACSCOC Liaison	June 1 E5Y
Vice Provost for Academic Innovation and Success	June 1 E5Y