Re-Evaluation Program Success: A New Model for Nontraditional Program Assessment

The higher education environment of the 21st century is characterized by significant demographic evolution. The once dominant profile of a “traditional” student, typically an eighteen to 22-year-old enrolled full-time, residing on or near campus, and largely unencumbered by major external responsibilities, no longer represents the majority of learners (Coursera 2025). Instead, institutions today increasingly serve a diverse population often categorized as “nontraditional.” This demographic shift creates a fundamental problem with how we measure success in higher education. The metrics traditionally used to gauge student success and program health were designed for the historical “traditional” student and are increasingly inadequate for learners today. These conventional measures often misrepresent the progress and achievements of students who juggle academic pursuits with work, family, and other significant commitments. Defining something as “nontraditional” can actually create a sense of “othering,” implying that the system was not meant for the students who are now increasingly part of it (Gulley 2016).

When an institution’s primary metrics fail to accurately gauge the success of a significant group of students, its strategic decision-making, resource allocation, and fundamental understanding of its own performance become compromised. This incongruency disadvantages the students served by potentially mislabeling their progress and hinders institutions from making fully informed strategic decisions about program viability.

This article presents the DPR framework developed by the author as a promising solution to these challenges. The framework offers a nuanced and actionable system for evaluating program health that aligns with the realities of modern learners. The following sections examine the literature on nontraditional students and traditional metrics, detail the DPR framework’s design and implementation, analyze its strategic value for institutional leadership, and provide recommendations for broader adoption.

Literature Review

Understanding why new evaluation frameworks are necessary requires examining what defines the nontraditional student population, how current assessment practices are developed, and why these practices fail to serve today’s diverse learners effectively. The literature reveals a critical disconnect: While students who balance work, family, and academic responsibilities now constitute the majority in many institutions, the metrics used to evaluate their success remain rooted in the assumptions of linear, uninterrupted, full-time educational pathways. This misalignment reflects a broader theoretical lag, in which assessment practices have failed to evolve alongside models like Bean and Metzner’s framework, which recognize the primacy of external environmental factors for modern learners. The implications extend beyond measurement accuracy; this disconnect perpetuates what some scholars term ‘othering,’ reinforcing the notion that institutional systems were never designed for the students they increasingly serve, thereby compromising both student support and institutional decision-making.

Characterizing the “Nontraditional” Student

Nontraditional students are now the new majority in many colleges and universities across the United States (Coursera 2025; MacDonald 2018). The National Center for Education Statistics (NCES) defines a nontraditional student as someone who meets one or more of the following criteria: they are over 24 years old, took time off after high school, attend college part-time, work full-time (usually defined as 35 hours or more per week), are financially independent of their parents, are single parents, have dependents other than a spouse, or do not have a standard high school diploma (such as having a GED) (Coursera 2025; National Center for Education Statistics n.d.). These students often return to education driven by career advancement, personal development, or the need to improve their employment prospects after years of service in the workforce (Sinclair 2019).

These distinct traits are not just labels; they are deeply connected to specific needs, motivations, and profound challenges. Nontraditional students often juggle multiple responsibilities, including work, family obligations, and academic deadlines, leading to what is often termed “time poverty” (Yates 2024; Mongoose 2024). Financial barriers are a significant obstacle, as rising education costs intersect with existing financial responsibilities (Stevens 2023; Mongoose 2024). Many require childcare support to attend classes or complete assignments (Yates 2024; Mongoose 2024). Due to their external commitments and their often commuter status, they may feel disconnected from the campus community and peer groups, affecting their social integration (Mongoose 2024).

The motivations of nontraditional students to pursue higher education are often pragmatic and tied to career advancement or reskilling. These motivations lead to a preference for practical, applicable learning experiences and a desire to recognize prior learning and professional experience (Sinclair 2019; Mongoose 2024). Nontraditional students require flexible learning options, such as online courses, evening or weekend classes, and streamlined administrative processes that respect their time constraints (Liaison 2024; Mongoose 2024). Additionally, adult learners often face mindset challenges, such as self-doubt about their academic abilities after being out of formal education for a while (Stevens 2023).

Aside from the factors that make up adult learners, the term “nontraditional” has been challenged as potentially damaging and a form of “othering” (Gulley 2016). This perspective contends that if such students constitute the majority, then the term, by definition, reinforces the idea that institutional systems and structures are not fundamentally designed for them. This framing shifts the focus from perceived student deficits to potential systemic misalignment, including the metrics used to evaluate their success.

Building on this critique, Williams (2023) proposes moving beyond the traditional/nontraditional binary altogether, arguing that “the rigidity of these characteristics fails to consider the unique differences of nontraditional students and what they bring to the table.” Williams advocates a new conceptual framework centered on the “modern student,” a classification based on behavior rather than functional status at the time of entry into higher education. This approach recognizes that today’s student makeup includes more dimensions, including those who may have earned college credits in high school, held full-time employment while attending school full-time through distance learning, or enrolled as students while also having children in college.

According to Williams (2023), a student qualifies as a “modern student” if one or more of the following factors are true: they attend part-time or full-time while working (or not working); life factors have caused them to recalibrate their educational goals and priorities; they demand that the educational experience caters to the real world; they expect education to be convenient, flexible, and cutting-edge; they expect the end-to-end experience to fully embrace mass consumerization; or their path to higher education includes college credit or a degree earned while still in high school. This framework acknowledges that “all students have different experiences, be it financially, academically, or personally, and should be given equal opportunity and support to pursue their dreams.”

Throughout the remainder of this paper, the author adopts Williams’s (2023) terminology of “modern student” to reflect this behavioral, student-centered approach rather than the deficit-oriented “non-traditional” designation, except when citing literature that employs the traditional terminology.

The Evolution and Limitations of Conventional Success Metrics

Higher education institutions have historically relied on conventional metrics to define and measure student success and institutional performance. Among the most common are retention rates, graduation rates (typically measured in 4-year, 6-year, and sometimes 8-year intervals for first-time, full-time cohorts), time to completion, and academic performance indicators such as grade point average (GPA) (Kim 2017). The traditional view of a successful student aligns with one who enrolls full-time immediately after high school, persists through subsequent terms without interruption, and progresses to degree completion within a prescribed time frame (Kim 2017).

National data, such as those of the National Center for Education Statistics (NCES), frequently highlight these metrics. For example, the overall six-year graduation rate for first-time, full-time undergraduate students who began seeking a bachelor’s degree at four-year institutions in Fall 2014 was 64 percent (National Center for Education Statistics 2022). The Integrated Postsecondary Education Data System (IPEDS) has been a primary source of these data, traditionally focusing its graduation rate calculations on first-time, full-time (FT-FT) cohorts (Yuen 2019; Lederman 2017). Beyond these macro-level indicators, metrics such as persistence rates and grades are also widely used as proxies for engagement and learning (Janio 2025).

Although efforts have been made to improve national data collection, such as the introduction of Outcome Measures in IPEDS, significant blind spots remain. Historically, the IPEDS graduation rate included only full-time, first-time students, effectively ignoring the outcomes of the majority of community college students, many of whom are part-time or transfer students (Yuen 2019). Although the Outcome Measures now include part-time and transfer students and extend the tracking to eight years (NCES 2024), they are still considered “progress but not perfection” (Lederman 2017). Given that a substantial proportion of entering students are not first-time, full-time (59 percent of the 2014–15 cohort, for example), traditional rates still exclude the majority (NCES 2024).

Why Traditional Metrics Misrepresent Nontraditional Student Journeys

The fundamental issue with applying conventional metrics to nontraditional student populations is that the underlying assumptions of a linear, uninterrupted, full-time educational path do not align with these learners’ realities. This misalignment leads to a misrepresentation of their journeys and achievements in several key ways.

Inflexibility with Diverse Learning Paths and Pacing

Traditional metrics often penalize students who, by necessity or choice, pursue their education differently. These measures frequently portray students attending part-time, those who “stop out” for a period due to life events and then return, or those who simply take longer to complete their degrees because of external commitments.

Weissman (2022) found that fewer than 20 percent of first-time students who enrolled part-time graduated within eight years of their initial institution, compared to 46 percent of their full-time counterparts. Student parents are especially affected by this disparity, often enrolling part-time due to limited time, which in turn increases the time it takes to complete their degree and can even affect their financial aid eligibility (Yates 2024). Traditional graduation rates focusing on FT-FT cohorts do not fully capture the experiences of the significant number of students who enroll part-time or transfer between institutions (NCES 2024).

Ignoring External Realities: Work, Family, and Financial Pressures

A critical flaw of many traditional metrics is their failure to account for the profound impact of external responsibilities on a student’s academic trajectory. Nontraditional students often juggle full-time jobs, significant family caregiving duties, and substantial financial pressures (Mongoose 2024; Stevens 2023; MacDonald 2018).

Student-parents, for instance, face immense systemic barriers, including significantly lower degree acquisition rates (e.g., 18 percent earning a bachelor’s or associate degree in six years compared to 54 percent for dependent, non-parenting students), severe financial hardship, chronic time poverty, and a lack of accessible and affordable childcare (Yates 2024). Students who work significant hours may face a “time bind” that limits their ability to maintain enrollment or take a full course load, thus extending their time to degree, even if such work provides crucial financial support (Douglas and Attewell 2019).

Misinterpreting Engagement and Academic Momentum

Metrics such as credit accumulation rates, often framed as indicators of “academic momentum,” may not apply universally or can be misleading when assessing adult learners. The National Center for Education Statistics (2022) found that adult learners (over age 24) tend to have lower credit accumulation rates (CARs) and credit completion rates (CCRs) in their first year than younger students, even when accounting for enrollment intensity. Although some studies suggest that early momentum metrics correlate with higher completion rates for some student groups (Jenkins and Bailey 2017), their direct applicability to all nontraditional students is questionable. For an adult learner carefully balancing work, family, and study, a slower and more deliberate pace could represent sustainable engagement and a commitment to quality learning, rather than a lack of momentum.

Failure to Capture Actual Learning and Competency

Perhaps the most fundamental critique is that many traditional metrics emphasize “seat time,” progression through a system, and compliance rather than the actual skills, knowledge, and competencies students acquire (Janio 2025). The success of higher education has long been defined by procedural indicators, which do not reveal whether students have developed critical thinking, analytical abilities, or the ability to apply theoretical knowledge to real-world challenges (Janio 2025). Traditional assessment methods may not adequately capture the breadth of knowledge and competencies of nontraditional students, particularly the valuable experiential learning they bring to their studies.

Theoretical Perspectives on Student Persistence and Success

Understanding why students, especially those outside the traditional college-age range, choose to remain in or leave higher education requires examining established theoretical frameworks and their applicability to diverse student populations.

Tinto’s Student Integration Model

Vincent Tinto’s Student Integration Model, originally proposed in 1975 and later revised, has been a cornerstone of student retention research (Braxton, Vesper, and Hossler 1995). Under Tinto’s model, students’ likelihood of persistence is mainly driven by their integration into an institution’s academic and social environments. The model’s underlying premise is that successful integration across both aspects leads to higher institutional commitment and, consequently, higher persistence.

However, Tinto’s model has been criticized for its relevance to nontraditional students. One major concern is that the SIM was designed mainly for traditional residential students and may not be applicable to commuter students, part-time learners, adult learners, or online students (Braxton, Vesper, and Hossler 1995). For these groups, the likelihood and nature of social integration on a residential campus differ significantly. The model has also been criticized for not giving enough weight to external factors, such as finances, family responsibilities, and job opportunities, that are often crucial for nontraditional students (Cotton, Nash, and Kneale 2017).

The Bean and Metzner Model

Recognizing the limitations of models like Tinto’s for nontraditional populations, Bean and Metzner (1985) developed their Conceptual Model of Nontraditional Undergraduate Student Attrition. The Bean and Metzner model offers a significant departure from the Tinto model by emphasizing the role of external environmental variables in the attrition process for nontraditional students, including older learners, part-time students, and commuters.

Bean and Metzner identified four sets of variables that influence a student’s decision to remain: personal background variables, academic variables, environmental variables, and social integration variables (Bean and Metzner 1985). Where the Bean and Metzner model differs from Tinto’s is that, for nontraditional students, environmental variables often have a greater impact on the decision to persist than social integration variables. The lives of these students are typically more deeply embedded in the spheres of work and family outside the campus, making factors such as employment demands and family responsibilities the main drivers of their ability to continue their studies (Wylie 2005).

This theoretical framework highlights the need for interventions that address external realities, such as flexible course scheduling, robust financial assistance and emergency support, accessible childcare, and career services tailored to working students. The Bean and Metzner model provides crucial context for understanding why traditional metrics often fail to capture the dynamics that drive nontraditional student success.

The Degree Progress Report Framework

Given the inherent limitations of traditional metrics and theoretical frameworks, innovative approaches such as the DPR framework offer promising alternatives that align with the realities of modern student experiences. The framework considers that, even in the most advanced institutions, extracting meaning from data to make informed decisions can be challenging (Blair and Zanidean 2020). By simplifying the data output across five key categories, the DPR seeks to streamline data analysis, creating an easily digestible output that aids in measuring the effectiveness of decisions made.

Genesis and Rationale

The DPR framework emerged from a clear institutional need at Embry-Riddle Aeronautical University, specifically the Worldwide campus, to develop a more effective and nuanced method for evaluating program health and student success. Embry-Riddle’s journey toward the DPR framework began with a recognition by Dr. Williams that “traditional higher education metrics fail to address modern higher education in a meaningful and useful way.”

This acknowledgment spurred an effort to create an analytical tool that could more accurately gauge the institution’s performance in achieving student success at the program level. The initial outcome of this endeavor was the “Water Tank Analysis.” This model used data spanning from prospective students to graduation to visualize student flow into, through, and out of the institution, with attrition represented as a ’crack/leak’ in the tank.

Although the Water Tank Analysis was foundational and catalyzed important conversations around student success, it had a significant limitation: it required specialized interpretation. The meaning of the data, for groups such as advising leadership and academic college leadership, was not immediately apparent. To overcome this and truly empower these stakeholders, the DPR was conceived. The aim was to create an analysis that was “easily digestible,” thereby enabling end users to directly understand and act on program performance data without an intermediary interpreter.

Core Architecture: Metrics, Weighting, and Grading Explained

At its core, the DPR framework offers a structured and actionable approach to evaluating the health of degree programs, with a specific focus on metrics that align with institutional goals of student success, engagement, and timely graduation, particularly for modern students. It is based on five key metrics, each with its own weight that adds up to an overall program grade. The definitive weights are as follows: Admit to Enroll (15 percent), Eligible to Active (25 percent), Course load (take rate) (10 percent), Discontinuation rate (Attrition) (20 percent), and Graduation rate (30 percent).

Each of these metrics has its own grading scale (A-F), with specific thresholds:

• Admit to Enroll: An “A” is ≥ 70 percent, reflecting a highly effective program in attracting admitted students.
• Eligible to Active: An “A” is ≥ 85 percent, benchmarking high engagement while allowing for normal variability in student circumstances.
• Course Load: This metric is differentiated for undergraduate and graduate programs. For undergraduates, an “A” is ≥ 4.0 courses/semester (equivalent to 12–15+ credit hours, enabling timely graduation). For graduates, an “A” is ≥ 2.5 courses/semester (equivalent to 7.5 + credit hours, allowing on-time graduation in 2 years).
• Discontinuation Rate: An “A” is < 10 percent, acknowledging that some attrition is normal even in excellent programs.
• Graduation Rate: This is also differentiated. For undergraduates, an “A” is ≥ 15 percent annually; for graduates, an “A” is ≥ 20 percent annually. These rates are adjusted for trends of modern students and realistic completion timelines over extended periods.

The weighted scores from these individual metrics are combined to produce an overall program grade on a 4.0 scale, which is then translated into a letter grade from A to F, or N if there is insufficient data. The specific weighting reveals a balanced philosophy. Although traditional outcomes such as graduation are clearly important (having the highest weight at 30 percent), the key indicators of engagement and persistence relevant to the modern student context, such as “Eligible to Active” (25 percent) and “Discontinuation Rate” (20 percent), are given significant prominence.

Alignment with Nontraditional Student Characteristics and Needs

One of the core strengths of the DPR framework is its explicit and intentional alignment with the unique characteristics and needs of modern students, as defined by Williams (2023). The framework was “developed with modern students in mind,” aiming to balance ”rigorous academic standards with the flexibility required by a diverse student body.” This alignment directly addresses the modern student’s unwavering demand for educational experiences that are ”convenient, flexible, and cutting-edge” while catering to real-world applications (Williams 2023).

The framework design reflects several key aspects of the needs of modern students:

• Flexibility and Workload Balance: The “Course Load (Take Rate)” metric directly acknowledges that ”for modern students, a lighter course load may be more typical due to work and family responsibilities.” This directly addresses the “time poverty” and multiple external commitments faced by modern students, who may attend “part-time or full-time while working full-time, part-time or not at all” (Williams 2023). The metric also recognizes that life factors can cause students to “recalibrate their educational goals and priorities,” requiring institutions to maintain academic rigor while accommodating the necessary flexibility (Yates 2024; Stevens 2023; Liaison 2024; Mongoose 2024).
• Realistic Graduation Expectations: The “Graduation Rate” metric adjusts its benchmarks based on national data for modern student trends, recognizing that these students often take longer to complete their degrees. The framework notes that nontraditional undergraduates often take six-plus years to complete, translating to an annual graduation rate of approximately 10 percent to 15 percent of the population if distributed evenly. This contrasts sharply with traditional metrics that often penalize longer completion times.
• Focus on Engagement and Program Appeal: Metrics such as ‘Eligible to Active’ and ‘Admit to Enroll’ provide insights on student engagement and program attractiveness, which are critical for retaining modern students who expect the end-to-end experience to fully embrace the mass consumerization that has permeated the corporate world to the higher education sector (Williams 2023). High participation suggests strong alignment of the program with modern students’ expectations of relevance in the real world, while a high enrollment yield indicates that the programs meet their demand for cutting-edge, flexible education.
• Actionable Insights for Support: By providing a granular view of program performance across these tailored metrics, the framework enables institutions to identify specific areas where modern students might be facing challenges and implement targeted support strategies. This approach aligns with Williams (2023) assertion that “all students have different experiences, be it financially, academically, or personally, and should be given equal opportunity and support to pursue their dreams.” The framework moves beyond traditional categorizations to focus on behaviors and needs, supporting students with diverse pathways to higher education, including those who earned college credits while in high school or who are balancing full-time work with education.

Empirical Validation of the DPR Framework

To demonstrate the practical application and validity of the DPR framework beyond individual program case studies, data from 44 degree programs across three colleges at Embry-Riddle Aeronautical University’s Worldwide Campus over two academic years (2023–24 and 2024–25) was analyzed. This comprehensive analysis examines descriptive statistics, year-over-year changes, performance comparisons across colleges, and the framework’s ability to differentiate program quality, thereby validating both the framework’s measurement precision and its strategic utility for institutional decision-making.

Descriptive Statistics

Table 1 presents overall descriptive statistics for all five DPR metrics across 88 program-year observations. The data reveal considerable variability in program performance across metrics. Admit to Enroll rates averaged 54.19 percent (SD = 9.73 percent), indicating that just over half of admitted students enrolled, with substantial variation among programs (range: 33.82 percent to 100 percent). Eligible to Active rates showed similar patterns, averaging 59.08 percent (SD = 9.47 percent), suggesting that approximately three-fifths of eligible students remained actively engaged in their programs.

Course Load averaged 3.84 courses per semester (SD = 0.83), reflecting the reality that many modern students attend part-time while balancing work and family responsibilities. Discontinuation rates averaged 18.26 percent (SD = 5.10 percent), meaning that nearly one in five enrolled students discontinued their studies within the measurement period. Graduation rates averaged 13.62 percent annually (SD = 7.98 percent), which aligns with realistic completion timelines for modern students who often take extended periods to complete their degrees.

Year-Over-Year Changes

Paired t-tests examined whether programs demonstrated statistically significant changes in DPR metrics between the 2023–24 and 2024–25 academic years (see Table 2). Results revealed two significant improvements. Eligible to Active rates increased significantly from 58.30percent to 59.87 percent, t(43) = −2.66, p = 0.011, with a small-to-medium effect size (Cohen’s d = 0.401). This improvement suggests enhanced student engagement and retention across programs.

Most notably, Course Load increased significantly from 3.73 to 3.95 courses per semester, t(43) = −6.23, p < 0.001, representing a large effect size (Cohen’s d = 0.939). This substantial increase indicates that students were taking on heavier academic loads, potentially accelerating their progress toward degree completion. This finding is particularly meaningful for modern students, as increased course loads can reduce overall time-to-degree while still maintaining manageable academic demands.

The remaining three metrics showed no statistically significant changes between years. Admit to Enroll rates increased slightly from 52.85 percent to 55.53 percent, but this change was not statistically significant (p = 0.147). Similarly, Discontinuation rates remained essentially stable (18.32 percent vs. 18.21 percent, p = 0.865), as did Graduation rates (13.22 percent vs. 14.02 percent, p = 0.205). The stability in these metrics suggests consistent program performance across both academic years, providing a baseline against which the significant improvements in engagement and course load become even more noteworthy.

Comparison Across Colleges

One-way ANOVA examined whether the three colleges differed significantly in their DPR metrics (see Table 3). Results revealed one statistically significant difference: Discontinuation rates varied significantly across colleges, F(2, 85) = 3.23, p = 0.045, η2 = 0.071. The College of Aviation demonstrated the lowest discontinuation rate (16.87 percent), compared to the College of Business (19.77 percent) and College of Arts & Sciences (19.17 percent). This medium effect size suggests that Aviation programs were notably more successful at retaining students through completion.

Two additional metrics approached statistical significance. Graduation rates showed a marginally significant difference across colleges, F(2, 85) = 2.68, p = 0.074, η2 = 0.059, with Business programs demonstrating the highest graduation rate (16.13 percent) compared to Aviation (13.16 percent) and Arts & Sciences (10.79 percent). Similarly, Admit to Enroll rates approached significance, F(2, 85) = 2.79, p = 0.067, η2 = 0.062, with Business programs showing the highest conversion rate (56.34 percent).

The remaining metrics—Eligible to Active (p = 0.844) and Course Load (p = 0.704)—showed no significant differences across colleges, suggesting relatively consistent student engagement and course-taking behavior across academic disciplines. These findings indicate that, while certain operational outcomes (retention and graduation) vary by college, fundamental measures of student engagement remain relatively uniform, suggesting a shared institutional culture of student support that transcends disciplinary boundaries.

Program Grade Distribution and Performance Validation

Applying the DPR framework’s weighted grading system to all 88 program-year observations revealed a distribution heavily concentrated in the C grade range (see Table 4). Specifically, 62.5 percent of observations (n = 55) received C grades, indicating room for improvement but acceptable baseline performance. An additional 22.7 percent (n = 20) achieved B grades, representing solid program performance. However, 14.8 percent (n = 13) received D grades, identifying programs requiring immediate intervention. Notably, no programs received A grades, suggesting that even the highest-performing programs have room for improvement, and no programs received failing grades, indicating that all met minimum viability standards.

This grade distribution validates the framework’s ability to differentiate program quality across a meaningful spectrum. The concentration of programs in the C range suggests that the grading thresholds are appropriately calibrated—not so lenient that all programs receive high marks, nor so stringent that few programs achieve acceptable ratings. The absence of both A and F grades reveals a system operating within acceptable parameters while maintaining realistic aspirations for excellence, creating both accountability pressure and achievable improvement pathways.

To further validate the framework’s discriminant validity, we compared high-performing programs (grades B or above, n = 20) with low-performing programs (grades C or below, n = 68) across all five metrics (see Table 5). Independent t-tests revealed statistically significant and substantively meaningful differences for four of the five metrics, providing strong evidence that the DPR framework successfully distinguishes program quality.

High-performing programs demonstrated significantly superior Admit to Enroll rates (62.71 percent vs. 51.68 percent), t(86) = 5.04, p < 0.001, Cohen’s d = 1.28, indicating they were markedly more effective at converting admitted students to enrolled students. This large effect size suggests that high-quality programs possess characteristics that attract prospective students and compel them to choose enrollment.

Similarly, Eligible to Active rates were significantly higher in high-performing programs (67.20 percent vs. 56.70 percent), t(86) = 4.90, p < 0.001, Cohen’s d = 1.25. This finding indicates that students in better programs remained more actively engaged, likely reflecting stronger program design, better student support, and more relevant curricula aligned with modern student needs.

Discontinuation rates showed the expected inverse relationship, with high-performing programs exhibiting significantly lower attrition (14.63 percent vs. 19.33 percent), t(86) = −3.91, p < 0.001, Cohen’s d = −1.00. This large effect demonstrates that programs receiving higher DPR grades were substantially more successful at retaining students through program completion.

Most critically for institutional success metrics, Graduation rates differed dramatically between high- and low-performing programs (19.30 percent vs. 11.95 percent), t(86) = 3.91, p < 0.001, Cohen’s d = 1.00. High-performing programs graduated students at nearly double the rate of low-performing programs, representing a large and practically significant effect. This finding validates that the DPR framework’s composite grade meaningfully predicts the ultimate outcome of student success: degree completion.

Interestingly, Course Load showed a statistically significant but inverse relationship, with low-performing programs demonstrating slightly higher course loads (3.95 vs. 3.48 courses/semester), t(86) = −2.27, p = 0.026, Cohen’s d = −0.58. This medium effect suggests a nuanced finding: simply taking more courses does not guarantee program success. Instead, this pattern may indicate that some struggling programs push students toward heavier loads without adequate support systems, potentially contributing to higher discontinuation rates. Conversely, high-performing programs may encourage sustainable course loads that balance timely completion with student capacity, particularly important for modern learners managing multiple responsibilities.

Summary of Empirical Findings

This comprehensive statistical analysis provides strong empirical support for the DPR framework’s validity and utility. The framework successfully identified meaningful variation in program performance (Tables 1 and 4), detected significant improvements in student engagement and course load over time (Table 2), revealed important differences across academic colleges (Table 3), and most importantly, demonstrated strong discriminant validity by differentiating high- and low-performing programs across nearly all metrics (Table 5).

The large effect sizes observed when comparing high- and low-performing programs (Cohen’s d ranging from 0.58 to 1.28) indicate that the differences identified by the framework are not merely statistically significant but also practically meaningful for institutional decision-making. Programs receiving higher DPR grades demonstrate superior performance across the entire student lifecycle—from initial enrollment through active engagement to ultimate graduation—validating the framework’s comprehensive approach to assessing program health.

These findings complement and extend the program-specific improvements documented in the B.S. in Project Management and M.S. in Management of Information Systems case studies (see Tables 6 and 7), demonstrating that the DPR framework operates effectively both as a tool for tracking individual program trajectories and as an institution-wide assessment system capable of distinguishing quality across diverse academic offerings. The framework confirms that it moves beyond simple descriptive reporting to provide actionable intelligence that can guide strategic interventions, resource allocation, and continuous program improvement efforts, particularly for institutions serving modern student populations.

DPR Strategic Value

The design of the framework is only as valuable as an institution’s ability to demonstrate its practical benefits for programmatic decision-making. In an era where higher education faces increasing scrutiny over outcomes, accountability, and return on investment, while simultaneously navigating an intensely competitive landscape, institutions require sophisticated analytical tools that transform raw data into strategic intelligence. The adoption and implementation of the DPR framework offer institutional leaders, including college deans and university administrators, a critical capability that simultaneously addresses multiple strategic imperatives: the ability to assess program health and student success through metrics designed for the students they actually serve rather than the students they traditionally expected to serve. This framework provides actionable intelligence that directly supports core operational priorities: identifying at-risk programs before they fail, optimizing resource allocation based on real student behavior patterns, and enabling evidence-based decisions that improve both student outcomes and institutional sustainability. Even more important, unlike traditional metrics that often reveal problems only after intervention opportunities have passed, the DPR framework enables proactive leadership through real-time insights into program health and student engagement. For institutions seeking competitive advantage in strategic enrollment management and student success outcomes, this shift from assumption-based to evidence-based assessment has profound implications for funding decisions, program development, and student support strategies.

Driving Student Engagement and Improving Retention

The framework’s emphasis on metrics such as “Eligible to Active” and “Discontinuation Rate” enables continuous monitoring of student engagement and retention patterns. High engagement and low discontinuation rates indicate strong program alignment with student needs and satisfaction. When these metrics drop, they offer leaders insight into areas that need attention before problems escalate and hurt the institution’s overall retention numbers.

This proactive capability represents a significant step up from traditional metrics, such as six-year graduation rates, which often highlight problems only after the key points for intervention have passed. This ability to respond immediately is particularly valuable for modern students who may face sudden external challenges.

Assessing Program Viability and Market Responsiveness

The “Admit to Enroll” metric offers an upfront indicator of each program’s attractiveness and ability to convert admitted students into enrolled students. Programs that show high yield rates in this area indicate a strong match with current market demands and student expectations. This insight is key to evaluating a program’s viability and competitiveness, helping the institution position itself effectively, and driving sustainable enrollment growth.

The Admit to Enroll metric links the admissions process to the broader framework of student success, acknowledging that a program’s ability to attract the right students is an early sign of its overall health.

Optimizing Academic Pathways and Course Load for Diverse Learners

Recognizing the varied life circumstances of modern students, the “Course Load (Take Rate)” metric is designed to evaluate the balance between workload intensity, academic rigor, and necessary flexibility. This metric directly addresses the “time poverty” and external commitments that characterize the experiences of many modern students, particularly those who may be attending “part-time or full-time while working full-time, part-time, or not at all” (Williams 2023). The framework acknowledges that modern students often must “recalibrate their educational goals and priorities” due to life factors, requiring institutions to provide flexible pathways that do not compromise academic quality (Yates 2024; Mongoose 2024). By ensuring that programs are designed to facilitate timely completion without overburdening students, institutions can significantly improve retention and graduation rates for this demographic.

Elevating Graduation Outcomes as a Key Performance Indicator

While graduation remains a critical indicator of program effectiveness, the DPR framework approaches this metric with the necessary context for modern learners. The “Graduation Rate” is adjusted to reflect realistic completion timelines for students who may be attending part-time or balancing other responsibilities. High graduation rates, within these adjusted expectations, signify robust program design and strong student support systems, which directly contribute to institutional success metrics.

Empowering Data-Informed Strategic Planning and Resource Allocation

Perhaps one of the most significant values of the DPR framework is its ability to enable data-driven strategic decisions at all levels of the organization. By providing a clear, holistic, and granular view of each program’s performance, the framework enables advising leaders, college deans, and university leaders to make informed, targeted decisions. High-performing programs, as identified by the framework’s grading system (A-F scale), can be celebrated, studied for best practices, and potentially scaled. On the other hand, programs with lower grades receive more targeted support and attention, so resources are used efficiently to make a bigger impact and improve student results. This program-specific granularity fosters a culture of continuous improvement and accountability at the program level, empowering academic leaders with actionable data rather than relying on institutional generalities that can mask important variations and hinder effective resource deployment. These strategic benefits demonstrate the value of the framework, but successful implementation requires careful planning and execution.

Implementation Case Study: Academic Operations and Training

Building on the comprehensive empirical validation presented in the previous section (Tables 1–5), the practical application of the DPR framework at Embry-Riddle Aeronautical University’s Worldwide Campus through specific program case studies demonstrates how data-informed assessment can complement and enhance relationship-centered student support practices. Within the Worldwide Student Success Department, the Academic Operations and Training unit employs a framework centered on appreciative advising and relational engagement with students. Appreciative advising is a strengths-based advising philosophy that emphasizes building trust with students, identifying their individual strengths and aspirations, co-creating educational plans aligned with those aspirations, and providing ongoing support for goal achievement. This approach proves particularly effective with modern students who require personalized support that acknowledges their complex life circumstances while helping them envision and achieve their educational goals. This approach establishes individualized advisor-student partnerships designed to align each student’s ambitions and goals with an optimal academic pathway. Whether students seek a career transition, professional advancement, or broader self-actualization, advisors work to translate aspirations into actionable degree plans.

While the nuanced, relationship-centered nature of this work resists simple quantification, the Degree Progress Report provides advising leadership with measurable indicators across core performance areas that function as a pipeline toward program completion. The framework enables advisors to identify where targeted interventions may strengthen student outcomes while maintaining the personalized approach that modern students expect and require.

Table 6 presents year-over-year comparisons for the B.S. in Project Management program between the 2023–24 and 2024–25 academic years, revealing consistent improvements across multiple retention and progression metrics. The admitted-to-enrolled conversion rate increased from 56.37 percent to 57.53 percent, while the eligible-to-enrolled rate rose from 63.18 percent to 66.94 percent. Student engagement, measured by average course load, grew from 4.42 to 4.69 courses per academic year. Attrition also improved, with discontinuation from active status declining from 17.89 percent to 17.07 percent. Most notably, the program graduation rate increased from 7.36 percent to 10.19 percent—a 2.83 percentage point improvement.

Table 7 presents year-over-year comparisons for the M.S. in Management of Information Systems program between the 2023–24 and 2024–25 academic years, revealing consistent improvements across multiple retention and progression metrics. The admitted-to-enrolled conversion rate increased from 58.97 percent to 71.74 percent, while the eligible-to-enrolled rate rose from 66.67 percent to 68.09 percent. Student engagement, measured by average course load, grew from 3.11 to 3.52 courses per academic year. Attrition also improved, with discontinuation from active status declining from 18.94 percent to 17.73 percent. Most notably, the program’s graduation rate increased from 15.15 percent to 16.31 percent—a 1.16 percentage-point improvement.

While these gains may appear incremental in isolation, collectively they demonstrate the measurable impact of individualized relationship-building practices that honor each student’s unique circumstances and academic journey across both undergraduate and graduate programs. The data suggest that appreciative advising, when systematically applied and assessed through appropriate metrics, contributes to meaningful improvements in student persistence and degree completion. However, it is important to note that appreciative advising represents one element within a comprehensive student support ecosystem. The improvements documented through the DPR framework reflect the combined effects of multiple factors, including appreciative advising practices, institutional support structures, program design, and student engagement initiatives. The DPR framework’s value lies in its ability to measure the aggregate impact of these interconnected support mechanisms, providing evidence that relationship-centered approaches—of which appreciative advising is a significant component—yield measurable outcomes when implemented systematically. This case study illustrates how the DPR framework enables institutions to quantify the effectiveness of student-centered practices that might otherwise remain invisible to traditional assessment paradigms, thereby validating investment in holistic support structures while identifying opportunities for continuous improvement. The particularly notable 12.77 percentage-point increase in the admitted-to-enrolled conversion rate for the M.S. in Management of Information Systems program demonstrates the framework’s sensitivity to program-specific interventions and its utility for identifying successful practices that can be studied and potentially replicated across other programs.

Conclusion: Advancing a More Equitable and Accurate Vision

The contemporary landscape of higher education, characterized by the majority of students who fit the Williams (2023) “modern student” profile, demands a fundamental rethinking of how institutions measure and foster success. Modern students, those whose educational journeys are shaped by demands for convenience, flexibility, real-world relevance, and cutting-edge experiences, require assessment frameworks that go beyond traditional demographic categories to focus on behavioral expectations and diverse pathways. This paper has underscored the significant limitations of traditional higher education metrics when applied to this diverse student body. Metrics rooted in the assumption of a linear, full-time, uninterrupted educational journey invariably misrepresent the progress, engagement, and achievements of learners who balance academic pursuits with complex work, family, and financial responsibilities (Yates 2024; Weissman 2022).

Beyond being a data issue, this represents a problem of equity. The maintenance of outdated measures reinforces systemic barriers within institutions that increasingly serve modern students who expect educational experiences that are “convenient, flexible, and cutting-edge” while catering to real-world applications (Williams 2023). This leads to inaccurate self-assessment and misguided support efforts (Gulley 2016). When institutions rely on metrics that don’t reflect the behavioral expectations and diverse pathways of their student population, they compromise their ability to serve these students effectively.

The DPR framework, developed by Dr. Williams (2023), offers a compelling alternative. Designed with the realities of the modern student in mind, it incorporates metrics such as “Admit to Enroll,” “Eligible to Active,” “Course Load,” “Discontinuation Rate,” and “Graduation Rate,” each carefully weighted and benchmarked to reflect these students’ unique pathways and behavioral expectations. At its core, the DPR framework provides a holistic, actionable, and digestible assessment of program health, empowering institutional leaders to move beyond lagging indicators toward a more proactive understanding of student success.

The comprehensive statistical validation across all 44 programs (Tables 1–5) reinforces program-specific findings through rigorous empirical analysis. Year-over-year improvements in student engagement and course load were statistically significant (p < 0.001) with large effect sizes (Cohen’s d = 0.94), while comparative analysis revealed that high-performing programs achieved graduation rates nearly double those of low-performing programs (19.30 percent vs. 11.95 percent, p < 0.001, Cohen’s d = 1.00). These large effect sizes demonstrate that the DPR framework identifies not only statistically significant but also practically meaningful differences in program quality, validating its utility for data-informed decision-making at the institutional, college, and program levels. The framework’s discriminant validity is further demonstrated by its ability to differentiate program performance across the entire student lifecycle, from initial enrollment conversion (Cohen’s d = 1.28) through active engagement (Cohen’s d = 1.25) to ultimate degree completion.

The practical application of the DPR framework at Embry-Riddle’s Worldwide Campus demonstrates its capacity to quantify the effectiveness of relationship-centered student support practices across both undergraduate and graduate programs while fostering productive collaboration between academic and administrative leadership. Year-over-year improvements documented in the B.S. in Project Management program include a 2.83 percentage point increase in graduation rates (7.36 percent to 10.19 percent), alongside gains in enrollment conversion, student engagement, and retention. The M.S. in Management of Information Systems program similarly demonstrated consistent improvements across all five metrics, including a particularly notable 12.77 percentage-point increase in admitted-to-enrolled conversion (58.97 percent to 71.74 percent) and a 1.16 percentage-point improvement in graduation rates (15.15 percent to 16.31 percent). These measurable outcomes demonstrate that frameworks aligned with modern student realities not only provide more accurate assessments but also enable institutions to identify and amplify effective practices that might otherwise remain invisible to traditional measurement paradigms. Critically, the framework establishes a shared language and a common evidence base that bridges the traditional divide between academic affairs and student success operations, enabling faculty, program chairs, advisors, and institutional researchers to collaborate on data-informed interventions rather than operate in isolated silos.

The framework recognizes that for many modern students, a slower, more flexible path does not reflect a lack of commitment or ability but rather their need to balance educational pursuits with work, family, and other life circumstances, while still demanding real-world relevance and cutting-edge experiences. It acknowledges that engagement can manifest in ways traditional measures fail to capture, particularly for students whose pathways to higher education may include earning college credits while in high school or who must recalibrate their educational goals due to life factors. The demonstrated improvements across multiple performance indicators in real-world implementation across both undergraduate and graduate programs, coupled with the statistically validated differences between high- and low-performing programs, validate the framework’s capacity to support both assessment and continuous improvement, transforming abstract principles into tangible student success outcomes while revealing program-specific insights that inform targeted interventions through collaborative academic-administrative partnerships.

The strategic value of such frameworks is clear: they enable institutions to improve student engagement and retention, assess program viability more accurately, optimize academic pathways for diverse learners, and make data-informed strategic decisions that enhance institutional effectiveness and equity. Moreover, by providing measurable evidence of the impact of student-centered practices across diverse program types, backed by rigorous statistical validation showing large effect sizes across key metrics, the DPR framework helps institutions build compelling cases for sustained investment in holistic support structures while establishing the data infrastructure necessary for productive dialogue between academic leadership and student success professionals, thus creating a virtuous cycle where assessment informs practice, practice drives improvement, collaborative analysis validates effectiveness through empirical evidence, and shared outcomes strengthen institutional commitment to evidence-based student support.

Adopting frameworks such as the DPR framework represents institutional self-awareness and adaptation, a commitment to seeing, valuing, and fairly assessing all students, particularly modern students whose educational journeys transcend traditional demographic categories. This shift toward holistic, student-centered measures ensures that success is defined and measured in ways that genuinely reflect student learning and achievement, regardless of the path. Williams (2023) argues that such an approach is essential as “higher education must be viewed as a developmental process that begins at the earliest inquisitive stages and continues throughout the student’s lifetime.” This perspective is crucial to fulfilling higher education’s mission of serving an increasingly diverse society, where students expect educational experiences that are flexible, relevant, and responsive to their complex, real-world circumstances. The successful implementation of the DPR framework at Embry-Riddle, validated through both program-specific case studies and comprehensive statistical analysis across 44 programs with demonstrably large effect sizes, demonstrates that when academic and administrative staff unite around shared metrics and common goals, institutions can achieve outcomes that neither group could accomplish independently, validating the essential role of cross-functional collaboration in advancing student success.

The evidence presented in this article—from theoretical foundations through statistical validation to practical implementation across undergraduate and graduate programs—demonstrates that maintaining academic rigor while acknowledging complex student realities is not only possible but measurably effective for institutional credibility and student outcomes when supported by robust collaboration between academic and administrative stakeholders. The transformative potential of this work extends beyond improved metrics to encompass the countless individual stories of students who will be seen, supported, and successful because their institutions chose precision over tradition and collaboration over compartmentalization. The critical question facing higher education leadership is not whether change is needed, but whether institutions will proactively embrace frameworks that honor the full spectrum of student experiences and achievements while breaking down organizational barriers that historically separated academic planning from student support services, or continue to rely on outdated metrics and siloed structures that obscure rather than illuminate pathways to success. The demonstrated results from Embry-Riddle’s implementation across multiple programs, supported by statistically significant findings with large effect sizes (Cohen’s d ranging from 0.94 to 1.28), suggest that institutions adopting such frameworks can expect not merely better data, but better outcomes achieved through the synergistic efforts of united academic and administrative teams—a compelling argument for leadership committed to both accountability and equity in service of all students.

Recommendations for Implementation and Future Directions

The DPR framework offers a promising model for institutions seeking to more accurately assess program health and support the success of modern students. For those considering its adoption or developing similar frameworks, several recommendations can guide implementation and ensure ongoing relevance.

Recommendations

Phased Rollout and Comprehensive Training: Implementing a new assessment framework requires careful planning. A phased rollout, perhaps starting with a pilot group of programs, can help identify and address logistical challenges. Comprehensive training for faculty, academic advisors, program chairs, deans, and other relevant staff is essential to ensure they understand the metrics, weighting, grading system, and how to interpret and use the resulting data to improve programs.

• Integration with Existing Processes: To maximize its impact, the DPR framework should be integrated into existing institutional processes, such as academic program review, strategic planning, and resource allocation cycles. The insights generated should directly inform decision-making in these areas.
• Robust Data Infrastructure: The accuracy and utility of the DPR framework depend on reliable and timely data. Institutions must ensure they have the necessary data-collection, management, and reporting infrastructure to support the framework metrics. This may involve collaboration between institutional research, information technology, admissions, student success, and registrar offices.
• Clear Communication and Stakeholder Engagement: Transparency is key. Institutions should establish clear channels for communicating the results, interpretations, and subsequent action plans of the DPR framework to all relevant stakeholders. Engaging faculty and staff in discussions about the framework and its findings can foster trust and a shared sense of responsibility for student success.

Future Research and Refinement

The DPR framework is not presented as a static solution, but as an evolving tool that invites further inquiry and dialogue. Future research should consider the following:

• Benchmarking and Contextualization: Research that compares outcomes and metric sensitivities between different types of institutional or program modalities could improve the applicability of the framework.
• Exploring Additional or Refined Metrics: As institutions gather more data using the DPR framework, opportunities may arise to explore the addition of new metrics or the refinement of existing ones. Future iterations could investigate the feasibility of incorporating measures related to post-graduation outcomes, the development of specific competencies, or more nuanced indicators of student engagement tailored to online or hybrid environments.
• Customization of Expectations: While the current framework differentiates course load expectations for undergraduate and graduate students, further research could explore more granular customization based on specific program types, disciplines, or the intensity of study, particularly for highly specialized or accelerated programs serving modern students. This could include considerations for students who earned college credit in high school or those whose life circumstances require unique educational pathways.

By embracing a cycle of implementation, evaluation, and research-informed refinement, institutions can ensure that frameworks such as the Degree Progress Report remain vital and effective tools for advancing a more equitable and accurate understanding of student success in the evolving higher education landscape. This shift from traditional to modern assessment paradigms represents more than institutional improvement; it is a fundamental commitment to educational integrity that will define higher education’s legacy for future generations. As modern students continue to reshape the demographic majority of higher education, institutions that adapt their measurement systems accordingly will not only better serve their current students but will establish themselves as pioneers of responsive, equitable education.

The DPR framework demonstrates that maintaining academic rigor while acknowledging complex student realities is not only possible but essential for institutional credibility and effectiveness. The transformative potential of this work extends beyond improved metrics to encompass the countless individual stories of students who will be seen, supported, and successful because their institutions chose precision over tradition. The critical question facing higher education leadership is not whether change is needed, but whether institutions will proactively embrace frameworks that honor the full spectrum of student experiences and achievements, or continue to rely on outdated metrics that obscure rather than illuminate pathways to success.