Assessment Is Data Collection, Data Analysis, and Data Driving

Regardless of discipline, division, goal, or area of the College, the assessment process consists of three broad elements:

• DATA COLLECTION means gathering evidence -- of proficiency in learning outcomes, progress toward goals, or other performance indicators. In learning outcomes assessment, data collection usually means evaluating and documenting students’ achievement of the learning outcome via assessment measures, such as lab reports, capstone projects, speeches, or clinical evaluations. In the Academic Program Review process, data collection refers to academic performance measures, such as retention and completion. In AES and other types of non-academic assessment, data collection can take the form of survey responses, pass rates, retention rates, graduation rates, etc. Data collection is only one part of assessment. Assessment also involves the collection of data/information about the assessment process so it can be used to improve the process.
• DATA ANALYSIS: Collected data needs well-informed people to make meaning from that data. Collected data can’t do much all by itself. With the goal of improving outcomes for our students, people analyze the data by identifying areas where we are exceeding, meeting, or falling short of benchmarks and goals and taking note of trends. Reviews of collected data could occur anywhere that concerned people gather: in a department meeting, with an advisory board, as part of a unit meeting, or as part of collecting reflections or feedback on a process or event. Recording how and when data analysis occurs (documentation) is important for consistency and reporting.
• DATA DRIVING: You’ve heard of “data-driven decision making"? That is what we’re talking about. Once data is collected and analyzed, departments, divisions, or units should decide how to “drive” the data into actions that can close gaps, sustain successes, or try innovative practices. Examples of data driving might include trying a new teaching method, updating a rubric, revising learning outcomes, engaging in professional development focused on a particular issue, buying new software, revising a course or program, aligning artifacts and outcomes, or improvements in the assessment process.

Three Categories of Assessment @ CCP

Assessment at Community College of Philadelphia falls into three main categories: 1) academic assessment, 2) administrative, educational and student support services (AES) assessment, and 3) institutional assessment, which includes general education.

---

Academic Assessment

The most frequently used type of assessment at the College is course assessment, the assessment of course learning outcomes (CLOs). Course assessment via course learning outcomes can show us how effective our courses, materials, and teaching methods are in helping students acquire the knowledge and skills they need to transfer, get gainful employment, and thrive as citizens. All teaching faculty at the College participate in course assessment in some manner. Course assessment is under the umbrella of academic assessment.

Program assessment involves several different processes at the College, the most widely known being the assessment of program learning outcomes (PLOs). Program learning outcome assessment is often closely related to the results of course assessment. The relationships between CLOs and PLOs are mapped in advance during the assessment planning process, creating a curriculum map or micro-map [link to curriculum map/micro map definitions in glossary]. Program assessment may also take the form of Academic Program Review (APR) or be part of special accreditation by external organizations, such as the American Bar Association (ABA), certifications such as Automotive Service Excellence (ASE), or based on the results of external exams such as the National Council Licensure Examination (NCLEX). Program assessment is another aspect of academic assessment.

A third area of academic assessment is divisional assessment, in which the College’s three academic divisions participate. Divisional assessment is high-level--mostly composed of summaries and highlights from course and program assessment—but it also includes goal setting and progress on goals related to Diversity, Equity, and Inclusion (DEI) as well as broad initiatives aimed at improving retention, solidifying transfer agreements, or resolving barriers to completion that the division has identified. The three academic divisions are 1) Business and Technology, 2) Liberal Studies, and 3) Math, Science, and Health Careers.

Project assessment can be part of academic assessment, although it may not involve the assessment of course or program learning outcomes. Although we use the word “project” to mean a lot of things in higher education – including a certain type of high-impact practice that promotes critical thinking and problem solving – a “project” also refers to a discrete activity or plan with a clearly defined beginning, middle, and end as well as specific needs analysis and expected outcomes. Although these types of projects may be related to learning outcomes, most projects have a scope that goes beyond CLOs or PLOs, such as increasing students’ sense of well-being or belonging, improving faculty skills with data interpretation, or setting up a dedicated peer tutoring project related to an underserved population of the College. Other examples of projects might include creating a dedicated student lounge space for language acquisition practice, doing a pilot project aimed at discovering the most effective methods of scaffolding assignments in gateway courses, or implementing a new software system to track participation in and satisfaction with professional development initiatives. In any case, project assessment works similarly to other kinds of assessment at the College, as it too relies on data collection, data analysis, and data driving. Project assessment can be an element of administrative and educational support services (AES) assessment as well. The end goal of project assessment is to determine whether the project has accomplished its stated purpose.

Assessment and Equity

Although the assessment of learning outcomes (courses, programs) or support outcomes (administrative and educational support units) helps us to know whether we are doing what we say we do (e.g., improving student learning, helping to meet students’ needs beyond the classroom, simplifying the enrollment process, etc.) outcomes assessment does not, in and of itself, let us know if we are achieving the College’s DEI goals, as expressed in the Sixth Pillar of the Strategic Plan: “creating a safe space campus that values, promotes, and amplifies the various voices and experiences for those from traditionally underrepresented or marginalized groups in its curricula, programming, administrative policies, hiring practices, daily interactions, and activities”. Community College of Philadelphia is a racially diverse institution, with 76% of its population identifying as people of color (43% Black, 17% Hispanic, 11%, Asian, 4% Multiracial, .3% American Indian/Alaskan Native, .2% Pacific Islander) . To ensure that all students are learning and that we are adequately responding to all students’ needs, the College must make racial equity part of our assessment process.

There are several ways for the College to work toward its equity goals, including disaggregating assessment data by race/ethnicity, gender, or developing materials, practices, and policies that address the needs of other protected groups, such as people with disabilities, people in the LGBTQ+ community, veterans, and those with intersectional identities. Other means include workshops in equity-centered pedagogy run by the FCTL (Faculty Center for Teaching & Learning), DEI training, creating spaces to analyze equity data and address gaps in achievement, or reviewing policies to ensure that they do not have a disproportionately negative effect on a particular group. But to know whether these measures are getting us closer to our goals, we must assess them.

---

Administrative, Educational and Student Support Services Assessment

Administrative, Educational and Student Support Services (or AES) assessment refers to the assessment of administrative units, such as Government Relations, Facilities Management, or Purchasing, as well as the assessment of those areas of the College that provide outside-of-class support to students, such as Advising, Counseling, the Catto scholarship office, or Records and Registration. While AES assessment was designed to emulate academic assessment in its basic structures and consistency, it is a more flexible format that allows units to capture the many different kinds of activities and objectives assessed in non-academic areas.

Equity in AES assessment

In the assessment of administrative and non-teaching units, as in the assessment of academic programs and activities, it’s important to foreground equitable practices, programs, and policies, and to ensure that ongoing processes do not routinely produce inequitable outcomes for students or staff. All AES plans require that assessment goals be aligned with at least one pillar of the College’s Strategic Plan; units should pay special attention to those aligned with the Sixth Pillar: Diversity, Equity, and Inclusion. Wherever possible, look for opportunities to disaggregate student data in order to critically examine the effects of policies and procedures. While some offices, such at the Center on Disability and the MarcDavid LGBTQ Life Center, have explicitly DEI-driven missions, all units should prioritize continuous improvement strategies that create a more diverse, equitable, and inclusive environment for all members of the campus community. Divisional DEI plans can be a good place to ground this work and to encourage an equity mindset for AES units.

---

Institutional Assessment

Institutional assessment at the College takes many forms. The most common type of institutional assessment is general education assessment, which is the assessment of the six Essential Skills of general education:

• Writing, Research, and Information Literacy (WRI)
• Technological Competency (TEC)
• Quantitative Reasoning (QURE)
• Scientific Reasoning (SCRE)
• Oral Communication/Creative Expression (OCCE)
• Cultural Analysis and Interpretation (CAI)

Another form of institutional assessment is the assessment of the College Strategic Plan, a dynamic document with the College’s Mission at its core that brings together all areas of the College under goals, pillars, and strategic directions. The goals tend to be broad and represent large-scale outcomes for the College, e.g., “Double the graduation rate.” The pillars break the goals into areas of focus, which provides strategic guidance, and strategic directions distill the pillars into actions. Divisional goals are aligned with the Strategic Plan and tied to all forms of assessment at the College.

Every eight years, the College creates a self-study report with supporting evidence for our regional accreditor, the Middle States Commission on Higher Education (MSCHE). Accreditation is an oversight mechanism that the federal government employs to ensure that institutions that benefit from federal funding meet an acceptable standard of quality; only institutions that are “accredited by a ‘nationally recognized’ accrediting agency” can receive federal financial aid dollars, such as Pell grants. This is another form of institutional assessment—or meta-assessment, as the MSCHE process is in large part an assessment of the College’s assessment processes and practices. MSCHE meta-assessment focuses on how the College meets the seven standards of accreditation within the context of the College’s mission and goals. The mission and goals are assessed with an eye on equity gaps and innovative practices tied to the strategic priorities and the six pillars of the College strategic plan. Important among the accreditation assessment parameters are the assessment of College’s processes, procedures, teaching and learning environment, student support services, student outcomes, and the fiscal soundness of the College—all aimed at student success and continuous improvement. Thus, the underlying basis of the MSCHE assessment is to determine whether the College, based on its mission and goals, is providing an environment in which students can successfully accomplish their educational goals.

Finally, institutional assessment can also come in the form of the budgeting process, where there is a transparent systematic allocation of resources tied to assessment findings within a specified timeline. Assessment-based budgeting is mission-focused and ensures that the fiscal, technical, human, and infrastructure resource needs of the College are allocated in a sustainable way to address equity gaps, promote innovation, and ensure continuous improvement. The budgeting process ensures resources are allocated to address the six strategic pillars and priorities of the College. In the spirit of continuous improvement, the budgeting process is periodically evaluated to examine for effectiveness, and where needed, changes are made to ensure alignment with the mission and goals of the College. Thus, allocated resources through the budgeting process have to be assessed for impact to ensure the effective and efficient use of college resources.   

Equity in Institutional Assessment

Equity work at the institutional, divisional, departmental, program, or course level begins with a shared definition of equity. The sixth pillar of the College’s Strategic Plan provides this definition: “Equity is the process of ensuring that processes and programs are impartial and fair. To ensure equitable outcomes, the College will work towards ensuring equal opportunities and access for all persons in the College.” This institutional definition of equity requires large-scale tracking of demographic data regarding retention and time to completion, as well as a critical examination of the barriers faced by students at Community College of Philadelphia —60% of whom identify as Black or Hispanic. This kind of tracking can be found on the Institutional Research dashboards, which are available to all.
 
The sixth pillar also includes the strategic direction to “Become an anti-racist college and fully implement anti-racist training for all administrators, staff and faculty.” To meet the College’s goals of ensuring equitable outcomes and becoming an anti-racist institution, anti-bias and anti-oppression training in every area of the institution is essential. Required and recurring training, assessed through multiple measures (e.g., feedback regarding student, faculty, and staff sense of belonging, data on the achievement of learning outcomes disaggregated by race) is critical to student success. Without explicit attention to institutional racism and how it affects our students, other efforts to improve student success may prove ineffective.
 
Equity in general education assessment involves using disaggregated data, rigorous meta-assessment for relevance to the student population, and the assessment of the essential skill of Cultural Analysis and Interpretation, which requires all students to “analyze case studies, creative works, systems of human thought and behavior, material artifacts, and other primary and secondary sources from a range of academic disciplines to discern and respect diverse perspectives and experiences related, but not limited to, race, ethnicity, gender, sexual orientation, ability, culture, region, country, religion, and/or language.” 
 
Equity in strategic planning is focused on the sixth pillar, established in Spring 2022, which commits the College “to an annual review and assessment through an equity lens of policies and procedures that affect student outcomes, employee hiring and retention, vendor proposals and agreements, and all other areas that may benefit.”
 
Our regional accreditor, Middle States Commission on Higher Education (MSCHE), has revised its standards as of 2023 (Fourteenth Edition), and now integrates the guiding principle of “reflection on Diversity, Equity, and Inclusion” throughout all seven standards. This encompasses DEI planning and goal setting, review of policies, curriculum, student services, assessments, resource allocation, and acknowledging and closing equity gaps. When the College develops its self-study in preparation for the MSCHE reaccreditation visit in 2031, the College must provide evidence of its work in these areas since the 2023 site visit, which makes DEI planning across the College as well as data tracking and equity-minded faculty and staff development a priority.
 
Equity in the budgeting process means creating and maintaining a system by which the College aligns resources to support equity goals. This process is an important direction in which to drive assessment data at all levels and is examined periodically and assessed for impact.

Assessment @ CCP