Assessment
What is Academic Assessment?
Assessment occurs every day as an organic process conducted by faculty and administrators that are dedicated to improving teaching and student learning. It is a systematic process to understand, support, and enhance teaching and learning within an academic program. Through this process, it is possible to identify inconsistencies, gaps, and strengths within the program.
When assessment is performed from a holistic perspective, the focus shifts from a single group of students to understanding the student learning outcomes within an entire academic program. The concept is to ascertain what students are learning and how effectively everyone involved is in helping them learn.
Why is Academic Program Assessment Important?
There are numerous benefits to assessment initiatives. These fall into two categories:
Improvement
Assessment provides a platform for faculty and administrators to examine the current state of the curriculum and identify its strengths and weaknesses.
Accountability
Assessment creates the method of demonstrating academic progress and achievements to stakeholders.
These stakeholders include prospective students, employers, and accrediting bodies like NASPAA (Network of Schools of Public Policy, Affairs, and Administration) which focuses on MPA accreditation and MSCHE (Middle States Commission on Higher Education), which focuses on Baruch College accreditation.
Marxe Assessment Spotlight
Interview with NASPAA’s Chief Accreditation Officer, Ms. Martha Bohrt
At Marxe, a component of our academic assessment program includes the work of fostering a positive assessment culture (in addition to implementing our assessment cycles). This aspect of the work is critical to the overall success of assessment, as it aids in cultivating support and participation. As a result, we have incorporated assessment guides, presentations, workshops, a website, and now a newsletter to our initiatives.
Although tremendous strides have been made in terms of both the actual assessments as well as the efforts to ensure a supportive culture, there is always more that can be done to bridge the gap and ensure the faculty and staff have a stronger understanding of and relationship with NASPAA and its mission. As a result, we have invited Chief Accreditation Office, Martha Bohrt to speak directly from a NAPAA lens regarding key areas/questions of accreditation that often arise.
How does NASPAA accreditation serve as a quality assurance mechanism for students and other stakeholders? NASPAA’s accreditation process is driven by public service values, is mission- and outcomes-based, and is grounded in an in-depth self-evaluation. Programs seeking accreditation use the self-study process to support ongoing program improvement, strengthen their commitment to public service education, and showcase their accomplishments. NASPAA-accredited programs establish observable goals and outcomes, and use information about their performance to guide program improvement. They contribute to the knowledge, research, and practice of public service, practice truth in advertising and ensure their students achieve learning objectives in five domains essential to public service.
What role should faculty play in the process of demonstrating and maintaining NASPAA accreditation standards? How does the process of complying with the standards and maintaining accreditation benefit faculty directly? Program faculty in NASPAA accredited programs participate in program delivery, program governance, and research and/or professional service related to the field of public affairs. These three areas are the ones where programs are expected to demonstrate conformance with the NASPAA Standards. In other words, faculty are at the front line of NASPAA accreditation:ensuring student learning, ensuring strategic program management, and ensuring the advancement of the field of public service. At the same time, faculty can use the NASPAA universal required competencies to determine learning objectives for their courses and the NASPAA accreditation standards to determine mechanisms to assess student learning and success. Additionally, Standard 3 requires programs to demonstrate support of the faculty body’s continuous professional development, diversity, and research agendas. In summary, the NASPAA accreditation process supports faculty in their continuous professional development both in teaching and research.
We understand that NASPAA is not prescriptive about how institutions engage in assessment of student learning. Can you speak about the relative importance of different aspects/phases of assessment (ie, faculty engagement in regular reflection and revision of their goals and assessment instruments, faculty discussions of assessment results, the development of action items and curricular changes). Are they all equally important to NASPAA? If not, which aspect is most important, and why? All aspects that contribute to program improvement are of equal importance to the NASPAA accreditation process. Assessment is like a machine with many components that help each other create a product. One piece of the machine are the assessment instruments, another piece is analysis of assessment results, and the product is program improvement. Faculty are of course the oil. If the program has a series of tools to assess student learning, for example, but is not doing anything to understand the results, then needed modifications cannot be identified or, if they are, they may be the wrong ones, as we only have determined effect, but not cause. The same is true of what may happen without the oil: if the faculty, the frontline of assessment, don’t participate in the assessment processes, then all the parts are missing that first-hand information needed to create appropriate, quality tools.
How can institutions partner with NASPAA to better educate faculty about what improvement in teaching and learning means to NASPAA? Becoming a NASPAA site visitor is the easiest and most fun way to learn what the NASPAA accreditation standards are all about! Through the site visit, volunteers see the standards applied across a wide range of contexts. Over 90% of volunteers report each year that the site visit allows them to learn new practices that can be implemented at their own program. Additionally, NASPAA staff is available year-round to engage with faculty. NASPAA also hosts accreditation institutes throughout the year specifically designed for faculty to learn about the accreditation process. All inquiries about becoming a site visitor and for setting up appointments with staff can be sent to copra@naspaa.org.
Dedicated to our students and the pedagogy of our programs, the Marxe School strives to provide the growth, skills, and knowledge necessary to advance as an individual and within a chosen field.
As a result of the School’s resolute passion and commitment, assessment is ongoing both at an individual level and programmatic level. It is a key process in guiding understanding and driving enhancements in academic programs.
Mission-Driven Work
The mission of the Marxe School and its degree programs is to empower the next generation of public and nonprofit leaders, advance knowledge about public life and policy, and engage communities across our city, nation, and world to foster effective, inclusive institutions and societies. This mission is at the core of our academic programs and the outcomes students are expected to attain. It is the School’s mandate to ensure all programmatic aspects demonstrate commitment and consistency to the overall mission. This work recognizes the relationship between mission and the curriculum. Through these efforts, the Marxe School works toward continuous curriculum improvement.
Commitment to Improving Teaching and Learning
Student achievement and success are at the forefront of the Marxe School’s goals, efforts, and activities. In addition to organically assessing individual course sections, faculty are engaged in a multitude of initiatives that promote the culture of improving teaching and learning such as the Marxe faculty seminars, Inclusive Pedagogy workshops as well as writing-specific faculty development workshops and embedded course support.
Accountability
An institution of higher education is a community dedicated to students, to the pursuit and dissemination of knowledge, to the clarification of values, and to the advancement of the society it serves. Recognizing this larger role higher education plays on society as a whole; it is essential to demonstrate to stakeholders the work and value of the education provided. These stakeholders include accreditors, employers, and potential students. Success comes from taking the needs and perspectives of these participants into consideration.
Assessment efforts are collaborative; the key to success is a combination of faculty expertise and the guidance and vision of specialized administration.
Learning Assessment Committee (LAC)
The LAC is responsible for planning and implementing all assessment activities for Marxe graduate and undergraduate programs. The main objective of the committee is to develop and maintain a systematic process for assessing and improving student learning. The Committee is responsible for the oversight of the development of program learning goals and objectives; working collaboratively with the Marxe Curriculum Committee to ensure these goals align with the curriculum; overseeing the development of assessment instruments as well as the collection of the assessments; and analyzing and using this information for continual improvement. The Committee’s conduct shall be consistent with the standards for the assessment of the MPA program required for accreditation by the National Association of Schools of Public Affairs and Administration (NASPAA).
View information on LAC members
Administering Assessment at Marxe
At Marxe, the Learning Assessment Committee (LAC) works in tandem with the administration as well as faculty coordinators to collaboratively establish a vision as well as design and implement assessment initiatives across programs. The assessment-focused administrative roles have continued to evolve as a demonstration of Marxe’s commitment to improving teaching and learning.
The role of the Deputy Director of Academic Programs was transitioned to the Office of Assessment, Accreditation, and Institutional Effectiveness under the Provost’s Office with the new title Associate Director of Assessment. This role will continue to focus on managing and guiding Marxe assessment initiatives; however, it now allows for a stronger connection to the mission and work of the College with the focus of Institutional Effectiveness incorporated.
The Data Analyst at Marxe assists in the documentation, collection, and analysis of data. This role has also expanded to create a more comprehensive assessment culture and portfolio. This individual will now also be responsible for managing the data for the entire School, ensuring the overall statistics regarding the programs are understood.
These recent changes are also considered an action item as the School and College continue to seek to ensure assessment is a priority across the campus and the work of the School mirrors the priorities and expectations of the College.
Guiding Principles of Assessment at Marxe
Assessment is successful when it is mission and value focused. Each aspect of the process should originate from this lens, ensuring that we not only examine what the program currently offers, but also establish what more we may want to provide, change, or enhance.
- Assessment initiatives are most effective when faculty-driven in combination with administrative leadership to provide vision and support (Kinzie, et al., 2014, p.2). Faculty involvement in every step of the process is essential for meaningful content and valid results; administrative guidance and expertise is also key in leading each step forward and providing an assessment lens to the process.
- Assessment initiatives are most effective when a positive culture is cultivated and woven throughout all aspects of the academic environment. As conversations and awareness increases, assessment will become a common theme and a path toward improving teaching and learning. This will lead to increased support and involvement.
- The goal of assessment is continual program improvement. This occurs based on programmatic/curricular actions that take place because of the assessment findings and larger conversations related to what has been uncovered.
Assessment Cycle
Assessment at Marxe is a deliberate and purposeful process. It includes the development and revision of goals; collaborative and inclusive pedagogical discourse; careful design and implementation of methods; and the evaluation of data and realization of action and evolution of content/elements as a result of these various phases. The School’s assessment initiatives ensure protocols prioritize the mission and meaning of the work, develop relationships and cultivate support and participation, and improve systems and measures to produce effective organization and action. Since the progress of Marxe’s assessment program and the partnership between the faculty and the assessment manager, a progressive culture focused on collaboration, intellectual dialogue, and effective management and incorporation of data results has been established.
Project-specific assessment cycles typically operate in three-to-four-year time frames with the first year focused on revisiting the program learning goals, developing or revising the rubric (if applicable), and identifying the ideal method to employ. Methods are primarily direct with embedded assignments or exams with occasional surveys used. The next two years focus on collecting and analyzing the data, looking for patterns, trends, and outliers within the results. The final year is dedicated to the generation of conclusions based on the results with potential actions brainstormed and implemented. Data is currently established based on a complete population (or as much of the population that could be obtained based on faculty participation) with two experts, outside evaluators hired to reduce bias and scoring the student work based on the faculty-approved, program-focused rubric. The evaluators are normed prior to the start of scoring, meeting with faculty and the assessment manager to define scores and practice on sample work, norming those involved and increasing reliability in the method. Evaluator scores are then averaged and aggregated to provide us with a description of the current proficiency of the students related to that particular outcome or area. Assessment’s important functions include a few notable items.
- Assessment fosters evaluation and discussion of current and future expectations of the curriculum through the development and revision of program learning outcomes
- Assessment provides a platform for critical intellectual discourse regarding curriculum through the development and adjustment of embedded assignments
- Assessment allows further understanding of the programs’ strengths and weakness and where adjustments may need to be made
Kinzie, J, Jankowski, N, & Provezis, S. (2014). Do Good Assessment Practices Measure Up to the Principles of Assessment Assessment? Assessment Update. 26(3):1-2, 14-16.
Course and Program Goals within a degree provide an understanding of the significant expectations of that program.
Program Learning Goals (or outcomes) are overarching “promises” of learning that encompass the degree’s mission, values, and overall expectations.
- Program goals address what a student should know and be able to demonstrate by graduation and should be evident at various points throughout the curriculum.
Note: Competencies at Marxe act as categories and are used to develop outcomes within specific areas of learning.
However, the program learning goals are measured to understand the proficiency of student learning and quality of teaching.
Course goals, which are specific to the content of a particular class, are often derived from program goals and provide course expectations and connect to activities associated with achieving them.
- When relevant program goals are emphasized in class, students can align course goals with broader program expectations to draw curricular connections across the program. This allows students to understand their expectations and hold themselves accountable.
Developing Goals
Through various domains of learning (Bloom’s taxonomy), course goals should address the why of what is occurring in the classroom and connect to the program learning goals.
Why are the students being asked to complete specific assignments and participate in discussions? Questions to consider when writing or revising goals:
- What do we expect of students in this course/program?
- How does the course relate to the overall mission, public service values, and goals of the program?
- What do we want students who successfully complete this course/program to know or be able to do after graduating?
- What knowledge and aptitudes do employers look for?
Goals should:
- Use concrete action verbs that address observable performance
- Be clear, simple, and not compound
- Be measurable
- Connect with class work or assignments that develop that goal
Note: Traits such as appreciation, respect, integrity, compassion, etc. are difficult to quantify or define. These constructs are not only difficult to measure, but nearly impossible to teach in a classroom. We can aim to instill these traits by modeling behaviors and discussing experiences and history; however, it is important that goals remain realistic and measurable.
Categories of Cognitive Learning
Bloom’s Taxonomy (Bloom, 1956)
- Knowledge (know and remember specific facts, theories)
- Comprehension (understand, interpret, compare, contrast, and explain)
- Application (apply knowledge to new situations and solve problems using required knowledge or skills)
- Analysis (identify relationships, organizational structures, and principles)
- Synthesis (create something, to integrate ideas into a solution, propose an action plan, or formulate a new classification scheme)
- Evaluation (judge the quality of something based on its adequacy, value, logic, or use)
Affective (Attitudinal) Learning
Attitudes, values, and habits of mind (we will address this further as we move forward)
Curriculum Map
Goals are addressed at different levels within different courses as well as within other aspects of the program, such as through recommended readings, events, and internships
Levels of teaching material include Introduction (I), Reinforced (R), Mastered (M)
Courses should be mapped to the various program goals that they address noting the level in which they are addressed.
Example:
Program Goals |
PAF 9100 |
PAF 9120 |
PAF 9140 |
PAF 9190 |
Learning Goal A: Distinguish and effectively employ the steps in the policy process: problem/goal identification; stakeholder identification and analysis; financing; approval; implementation and evaluation. | I | R | R | M |
Course goals, which are specific to the content of a particular class, should include statements that are often derived from program goals. When relevant program goals are emphasized in class, students can align course goals with broader program expectations to draw curricular connections across the program. This allows students to understand the expectations and hold themselves accountable for what they will need to know and demonstrate by graduation.
Bloom, B. S. (1956). “Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain.” New York: David McKay Co Inc.
The goal of assessment is to enhance the curriculum by evaluating student learning. This is accurately done by collecting a valid sample of evidence in the form of student work, exams, surveys, and other relevant data throughout the program.
- Direct evidence includes student work such as case studies, capstones, portfolios, and oral presentations
- Assignments are kept confidential and only used for assessment purposes
- Actions are determined based on what will best benefit students and the overall program. The purpose of Assessment is to continually evaluate learning in order to enhance program and curriculum quality
- Indirect evidence involves data that demonstrates perceived student learning, such as surveys and graduation rate
Rubric Design:
A rubric is a scoring guide developed to evaluate student proficiency. When designing a rubric, the goal is disaggregated, and criteria is established in the process to define and emphasize that goal’s meaning within the program. Standards established remain consistent across the program where significantly addressed.
While standards or criteria clearly define the goal, descriptions are provided for each score to provide an understanding of how that score is achieved.
Program Rubrics:
- Define and clarify each specific Program Learning Goal.
- Should be goal-focused and universal, with criteria that can be applicable to that goal wherever the goal is significantly addressed.
- Rubrics should be developed prior to the assignments or tools used to test the students’ proficiency with a particular goal.
Reliable Rubrics: New Innovative Template Design at Marxe
The rubrics designed for program assessment are meant to define the standards for the broader program learning goals and as a result should have a universal quality. This is essential to developing a consistent and reliable scoring instrument that will measure a goal addressed at multiple points across a program.
Conversely, discipline-specific courses and assignments comprise of more narrowly focused expectations. Assessing a program goal within a course using an embedded assignment means that this must be addressed within the scoring guide as well to supplement the more general, rubric descriptions.
The revised rendition addresses both with:
- The consistent, reliable universal descriptions addressing the meaning of the goal under each score.
- Assignment-specific inserts that allow for distinct specificity and tailoring depending on the course or assignment used to assess the goal.
Recent Marxe Program Rubrics can be referenced below:
Marxe Comp#1 A&B: Management Rubric
Marxe Comp #1, A&B: Management Rubric with PAF 9163 customization
Marxe Comp #2, Goal B: Policy Rubric
Marx Comp #3, Goal A&C: Critical Thinking/Analysis Rubric
Marxe Comp #3, Goal B: Critical Thinking/Analysis Rubric
Marxe Comp #4, Goal A: Public Service Perspectives
Assessment occurs naturally in the ongoing engagement of teaching and learning; however, it is often beneficial to have tailored information to provide clarity and guide future work. If more in-depth material is required, contact MSPIA.Assessment@baruch.cuny.edu
Writing Program and Course Goals
Baruch College Writing Learning Goals
Rubrics
AACU Value Rubrics and Resources
Assessment Journals/White Papers/Websites
National Institute for Learning Outcomes Assessment
Research and Practice in Assessment
Association for the Assessment of Learning in Higher Education (AALHE)
American Journal of Evaluation (AJE)
New Directions for Teaching and Learning
CUNY Assessment 101
CUNY Assessment 101 is a new, free, online course now available to all CUNY faculty and staff. This initiative was developed by the CUNY Assessment Council with the goal of providing a comprehensive assessment resource available to all. It is an asynchronous course that can be taken at your own pace.
The Marxe community has committed to building a strong foundation of values throughout our School and programs. From our mission, we continue this work through a comprehensive lens, including efforts within the curriculum. We expand our scope of inclusion, diversity, and equity to go beyond recruitment and retention and address what students should learn and be able to demonstrate in relation to these values.
As we prepare students for positions of responsibility and leadership in public service, it is essential that we provide them with skills to demonstrate accountability and results within their own organizations and that we model what we teach through our own systematic and thoughtful assessment processes.* (Rubaii & Calarusse, 2014)
As an element of our dedication, the Marxe community has collaborated on and finalized a policy statement that represents a collective definition of inclusion, diversity, and equity and how these perspectives and behaviors should be represented within academia. It is an important statement that supports our mission and advancement of work throughout all programs.
Marxe’s Inclusion, Equity, and Diversity in Academia (IDEA) Statement
*Rubaii, N., & Calarusse, C. (2014). Preparing Public Service Professionals for a Diverse and Changing Workforce and Citizenry: Evaluating the Progress of NASPAA Programs in Competency Assessment. Journal of Public Affairs Education 20(3), 285-304.
Marxe is one of three Schools within Baruch College and operates under Middle States Commission on Higher Education (MSCHE) Accreditation Standards. Baruch College’s next Accreditation Review is 2026-27.
Middle States Mission
Middle States Standards
Resources
The Master of Public Administration program degree receives program accreditation from the Network of Schools of Public Policy, Affairs, and Administration (NASPAA). Marxe’s next Accreditation Review will be 2021-2022. This means that the Self-Study Report would be submitted by August 15, 2021, the site visit would take place sometime between February and April of 2022, and the accreditation decision would be made by the Commission in June of 2022.
NASPAA Mission
NASPAA Standards
Resources
For additional information, the Office of Assessment, Accreditation, and Institutional Effectiveness at Baruch College provides comprehensive guidance and resources to support all aspects of assessment and accreditation across campus.
Marxe Program Learning Goals address what a student should know and be able to demonstrate by graduation:
- Broad statements that address higher order learning as well as encompass the degree’s mission, values, and overall expectations
- Overarching, broad expectations that are addressed in different places throughout the curriculum
- Teachable and measurable
- Students should be able to align their course goals with the broader program expectations in order to draw curricular connections across the program
- Ensuring program goals are clear and present throughout the curriculum allows students to understand their expectations and hold themselves accountable
Learning goals are developed from various cognitive frameworks including Bloom’s taxonomy. These frameworks are organized by levels based on the intricacy of skill and may be summarized into three categories:
- Knowledge and conceptual understanding
- Application, analysis, synthesis, and evaluation
- Attitudes, values, dispositions, and habits of mind
Below are the program learning goals listed for each of Marxe’s degree programs. Upon graduation, students within the Master of Public Administration (MPA) degree programs at Marxe are expected to obtain knowledge and demonstrate skills associated with the following Program Learning Goals approved by the faculty and fall under five Universal Competencies established by NASPAA.
Management:
1. Use management and leadership theories to analyze the design, operation, and governance of public and nonprofit organizations.
2. Develop recommendations, based on multiple analytical frames, to improve the management of public and nonprofit organizations and achieve the organizational mission.
Policy:
3. Engage in the steps of the policy process including identifying a problem and making a proposal.
4. Identify alternative policy options; assess their potential feasibility and effectiveness; describe how the policy options are expected to affect outcomes (mechanisms).
Analyze and Think Critically:
5. Use social science methods to analyze, evaluate, and draw valid conclusions from data to apply to public policy questions, and/or decision-making situations, and/or issues of public concern.
Public Service Perspectives:
6. Demonstrate an understanding of the public interest in an assignment.
Communication:
7. Exhibit an awareness of strengths and weaknesses in verbal and written communication and articulate a plan to maximize communication effectiveness.
The program in Higher Education Administration at the Marxe School has established program learning goals that graduates are expected to achieve. The HEA program is dedicated to the overarching value of transparency in higher education. Toward that end, accountability is not only taught, but also embedded into the fabric of the program. The following program learning goals not only serve to guide students throughout the program – they also serve as the benchmarks against which the program judges the effectiveness of the School’s work.
(revised program learning goals pending approval from full faculty in Spring 2023)
Program Learning Goal 1: Identify and navigate the complex structures of higher education to inform effective change.
Program Learning Goal 2: Analyze, think critically, and make evidence-based recommendations that recognize historical context and/or incorporate organizational mission and/or strategic planning to advance higher education.
Program Learning Goal 3: Communicate and interact effectively with a highly diverse set of constituencies within the field of higher education.
Program Learning Goal 4: Articulate and apply a set of values for cultivating an equitable, inclusive, and just higher education environment.
Program Learning Goal 5: Use leadership and management theories to analyze the design, operation, and governance of higher education institutions.
- Understand and apply policy analysis to international domains
- Manage and lead programmatic initiatives in governmental and nongovernmental organizations addressing international affairs
- Study international policy convergence and policy diffusion
- Understand and apply theories and methods of comparative public policy and administration, and international and national governance systems and interactions
- Utilize analytic tools on the impact of regionalization and/or globalization, including efforts to harmonize or coordinate domestic and international policymaking and governance
- Assess sub-national, national, trans-national, and supranational policies and political actors and their consequences for the problem-solving capacity of governance systems
- Use media tools, old and new, to promote the interests of institutions
Program learning goals are the broad statements that address higher order learning, encompassing the degree’s mission, values, and overall expectations. These goals are addressed at various points throughout the curriculum.
Student completing the BSPA program will:
- Articulate how public policy is formulated, implemented, and evaluated.
- Write documents that articulate purpose, utilize evidence, logic, and apply pertinent values to provide analysis and arguments relevant to the question or issue and appropriate for a chosen audience.
- Produce oral presentations that articulate purpose, utilize evidence, logic, and apply pertinent values to provide analysis and arguments relevant to the question or issue and appropriate for a chosen audience.
- Analyze data and draw valid conclusions.
- Apply research findings to public policy questions or decision-making situations.
- Demonstrate an understanding of pertinent values, such as diversity, integrity, ethical conduct, and professionalism; and articulate how they can impact public policy and decision-making situations.