Assessing student learning within a program is one of the primary objectives of education. Done well, students thrive, programs become more and more innovative, and the educational institution better responds to its stakeholders; poorly done, student motivation, learning, and the institution itself suffer. Thus, a robust assessment process, on the program level, is critical to student and institutional success. By following best practices in programmatic assessment, you can impact the quality of education.
By utilizing best practices, you can ensure that your institution gets the most out of your measurement of learning outcomes. The term ‘learning outcomes’ used here is interchangeable with ‘learning objectives, ‘intended student learning outcomes, ‘program-level learning outcomes, and other terms designed to describe the specific ‘learning’ requirements an academic program plans to assess.
Principles of Assessment in Higher Education
- Outcomes-based – a competency-based assessment approach is designed based on the outcomes for courses/modules, programs, and the academic institution.
- Employer-centric – the academic program’s strategic goal is to produce employment-ready graduates who can advance in their professional journey.
- Data-driven – competency-based assessment must use relevant data so that program managers can make sound, informed decisions about achieving academic excellence.
Based on our experiences and those described in the case studies found in the whitepaper, The Use of Summative Assessment to Improve Quality in Business Administration Programs, we offer the following best practices and tips for competency-based assessment in higher education.
The Assessment Process
When undertaking programmatic assessment, it is best to start by creating a process. The most common tool used in higher education is an adaptation of the Deming Cycle for Continuous Improvement.
Suppose no assessment center exists at the institution. In that case, it is a best practice to constitute a team that will oversee this process. The assessment team can be made up of faculty or program chairs, data managers, designated quality assurance staff, those responsible for overseeing curriculum enhancements, and perhaps staff who are responsible for the state- or country-wide diploma evaluation and recognition. It is essential that all quality assurance initiatives, including assessment processes, have buy-in from the necessary stakeholders and leadership.
Best Practices based on the Assessment Process
The programmatic assessment process is more than administering an exam. Therefore, it is critical to ensure that the systems that support the assessment process are well defined.
1 – Identify Program Level Learning Outcomes
Program-level learning outcomes are statements that encapsulate the overall learning goals of an academic program. They are not to be confused with course/module level learning outcomes which instead measure student learning in individual courses/modules.
Best Practices:
- Identify any specific learning outcomes that are required to be evaluated by accrediting or national quality assurance agencies.
- Recognized that competency-based outcomes include both knowledge and soft skills competencies.
- A set of learning outcomes may consist of those beyond what is required by a specific quality assurance or accreditation agency.
- Learning outcomes reflect what students are expected to know and be able to do and are expected to be aligned with the institution’s mission.
- Use action verbs from the Bloom’s Taxonomy when writing learning outcomes.
- Ensure that each learning outcome can be measured.
- Write an average of five to seven program learning outcomes.
- Learning outcomes should change and evolve over time.
- Use action verbs from the Bloom’s Taxonomy when writing learning outcomes.
- Ensure that each learning outcome can be measured.
- Write an average of five to seven program learning outcomes.
- Learning outcomes should change and evolve over time.
2 – Identify Objectives
Program-level learning outcome objectives serve as indicators to judge the overall effectiveness of academic programs.
Best Practices:
- Ensure that each learning outcome has a specific quantitative and/or observable performance objective.
- Establish criteria thresholds, varying on the degree level and/or specialization.
- Increase criteria thresholds if, over time, established goals are continually met or surpassed.
- Establish performance objectives for each learning outcome that the assessment team should agree upon.
- Learning outcome performance objectives can serve, in part, as ‘Key Performance Indicators’ in an institution’s strategic plan.
3 – Specify Assessment Processes
Programmatic assessment typically involves multiple instruments used to objectively evaluate a variety of skills.
Best Practices:
- Technical skills measurement of ‘direct learning’ can be achieved through a pre-test and post-test comprehensive assessment.
- Assessment topics should be aligned to learning objectives that are in turn aligned to learning outcomes.
- Soft skills assessment tools should measure observable behaviors from multiple evaluators using a 360° approach.
- Experiential learning tools such as a simulation exercise enable the evaluation of both technical and soft skills competencies.
- Measurement tools must be capable of providing objective, quantifiable data.
4 – Collect and Analyze Data
Program-level assessment should be integrated into an already existing course/module such as a capstone course/module, a strategy course/module, or other required courses/modules.
Best Practices:
- The score received on an end-of-program comprehensive assessment should account for a certain percentage of the final grade.
- We recommend 10%, with 5% for completing the assessment and 5% based on the score.
- Other methods can be considered to incentivize the assessment, including faculty recognition (best student learning awards, a prize for pedagogical innovation, etc.).
- Benchmarking student cohort data can be both internal and external to understand strengths and potential gaps in student learning.
- The use of proctored and/or non-proctored online assessments expands the capability of assessing online, hybrid, and in-person programs.
5 – Share Results
Sharing results with faculty, administrators, and accreditation agencies is essential if not mandatory. An assessment report facilitates this sharing and expediates its diffusion.
Best Practices:
- Assessment results should be presented in a report to enable faculty and program directors to see the data.
- Assessment reports should highlight challenges that a program has in delivering specific learning outcomes (students not obtaining the targeted learning objectives).
- Assessment reports should recognize successes and celebrate them with faculty recognition awards, etc.
- Assessment reports need to make concrete and feasible recommendations based upon the data to improve student learning within the program.
- Assessment reports need to provide historical data, i.e., prior assessment cycles, positive and negative trends.
- Assessment reports need to consider data outside the assessment process (budgets, faculty qualifications and deployment, deficiencies in digitalization, etc.)
6 – Make Changes
Schools need to track changes in student learning and program content/delivery resulting from programmatic assessment. While most changes lead to improvement, some do not and should also be evaluated and analyzed.
Best Practices:
- Changes implemented thanks to the assessment report need to be followed up on, i.e., reassessed/measured subsequently (often the following year) to ensure the recommendation worked as planned. If not, a further recommendation is needed.
- Changes and their effectiveness/efficiency should be documented in the assessment report.
- Schools should look at their data points and assessment tools regularly to see if they are working and replace or modify them when needed.
A well-structured assessment system that takes into consideration the best practices mentioned above can help provide institutions with tools and techniques to improve student and institutional success. Key faculty and administrator involvement and feedback to institutional leadership, improves the understanding of the use of external tools to assess, measure, and improve student learning. The assessment process works best if understood and accepted by all stakeholders at the institution and viewed as an integral part of the continuous improvement process.
To further understand how these best practices impact the quality of education. Download the full whitepaper, The Use of Summative Assessment to Improve Quality in Business Administration Programs.
