High School Ranking Metrics Put Rural Schools at a Disadvantage by Beth Howd

With the rise in availability of online school rankings, parents and guardians have access to a wealth of school performance information that they can use to learn about their local schools and to make informed decisions on enrollment. Two such ranking systems are the U.S. News and World Report and Niche. These ranking systems offer information to allow people to compare performance at a variety of schools easily, but both use metrics that do not take into account key features of small school districts located in rural areas. The problem with this oversight is that it feeds into a self-perpetuating cycle of disadvantage for rural schools.

Interestingly, U.S. News and World Report changed its methodology significantly between 2018 and 2019 in order to rank a wider variety of schools (Morse & Brooks, 2019b).  Unfortunately, it retained a bias against rural schools. The previous ranking system first eliminated schools based on state assessments or graduation rates. It then ranked schools using a College Readiness Indicator based solely on AP/IB test scores. This procedure meant that schools that use dual enrollment for college credit options instead of AP/IB courses could not be accurately ranked (Morse & Brooks, 2019b). The new system includes more schools by expanding the methodology to include six factors:

• College Readiness Index Rank
• College Curriculum Breadth Index Rank
• Math and Reading Proficiency Rank
• Math and Reading Performance Rank
• Underserved Student Performance
• Graduation Rate Rank

The revised method then compiles these factors to determine a percentile rank that can be used to compare schools.

On the surface, this revised methodology appears sufficient because it includes a wider variety of indicators that should give a more holistic view of a school’s performance. The report even breaks down the geographical location of the top 25% of ranked schools. When grouped according to whether or not the school is near an urban area, 89.4% of the top 25% of schools nationally are in cities, the suburbs, or surrounding towns and nearby rural areas. Only 10.7% of the top quartile are in towns and rural areas not near urban areas (Morse & Brooks, 2019a).  At face value, this distribution reinforces the idea that the large metro area school model is superior. However, a closer inspection of the indicator requirements shows a different picture. The rankings have chosen to narrowly define the College Readiness and College Curriculum Breadth indicators by prioritizing metro area educational approaches to advanced coursework over approaches used in schools with smaller populations. Therefore, 40% of the ranking is skewed against non-metropolitan area schools.

According to the ranking website, “College Readiness” is determined by two sub-factors: the percentage of 12th graders who took at least one AP or IB exam and the percentage of 12th graders who passed (scored three or higher) at least one AP or IB exam. This indicator is 30% of the total ranking score.  Under this revised methodology, schools that do not offer AP and IB courses undergo a score adjustment. This adjustment keeps these schools from automatically falling to the bottom of the rankings. Instead, these schools are adjusted closer to schools that offer only a few AP or IB courses (Morse & Brooks, 2019b).   

Similarly, “College Curriculum Breadth” is also based only on AP and IB exam scores. This 10% of the score is supposed to look at how well seniors in college-level courses are performing across multiple disciplines (Morse & Brooks, 2019b). If schools do not utilize AP and IB courses, they automatically have “N/A” for these two factors, even if a significant number of their students are taking actual college classes through dual enrollment programs with local colleges.

Because many people are unfamiliar with how schools in rural areas and small towns handle upper-level classes, this methodology might appear inclusive. However, this metric focuses on only one way to take college-level classes: AP and IB courses. These courses are taught through local high schools. They require completing an approval process with the overseeing company along with a significant investment in school resources for a teacher, teacher training, and supplies. In many suburban areas, such courses have generally replaced what used to be the advanced courses in many subjects.  These courses generally require that students pass a standardized exam to receive college credit.  

An alternative advanced course program is available in many areas where students can take courses from colleges while in high school (Thomas, Marken, Gray, & Lewis,  2013). This program, called “dual enrollment” or “dual credit,” allows students to enroll at both the high school and local college at the same time and provides course credit both places. Typically, the school system covers the expense of the courses. Because they are courses offered by the local college, these courses are college-level classes. Many students receive both high school and college credit for the courses.  

In areas with lower student populations, dual-enrollment offerings create a way for students to take upper-level courses without the need to expand the existing school infrastructure. However, neither the College Readiness and Curriculum Breadth factors in the U.S. News rankings (Morse & Brooks, 2019b) nor the Academics Grade for Niche (Niche, 2019a) includes dual-enrollment offerings. This omission matters. It means that results from actual college classes are not being included in the College Readiness, Breadth, and Academics Grade categories; instead, only college equivalent classes are being included. Unfortunately, the effects on the rankings are more pronounced for rural schools than for schools in metropolitan areas.

The National Center for Educational Statistics’ (NCES) report “Dual Credit and Exam Based Courses in U.S. Public High Schools:2010-11, First Look” (Thomas et al., 2013) shows data on the percent of students in different locations taking dual credit courses and AP/IB courses. According to the data, 78% of schools with less than 500 students offer dual credit courses. Only 43% offer AP or IB. This difference is significant when compared to the 98% of large schools, enrollment of 1200 or more, which offer AP/IB courses. A similar pattern emerges when comparing dual credit versus AP/IB offerings at schools based on geographic location; 86% of rural schools offer dual credit while only 55% of those schools offer AP/IB.  In contrast,  90% of schools located in towns offer dual credit while 75% offer AP/IB courses, whereas 89% of suburban schools and 79% of urban schools offer AP/IB courses (Thomas et al., 2013, p 6). This data clearly shows that students in towns and rural areas are more likely to have access to dual credit courses while city and suburban students are more likely to have access to AP/IB courses.[1] By omitting dual credit information in the College Readiness indicator, the U.S News and World Report and Niche rankings leave out a significant piece of data that could be used to help more accurately compare schools in less population-dense areas to schools in urban settings and create a better-informed ranking. 

Analysis of Niche’s methodology shows that it considers a more extensive list of conditions than U.S. News and World Report. Still, it also includes AP exams as a significant part of its academic score.  These scores account for approximately 6% for the total school ranking (Niche, 2019a). There is no note on the Niche website that dual enrollment can be used instead of AP exams in its ranking process. Niche also includes a culture and diversity score. This subscore includes racial diversity and economic diversity, but not in equal percentages, 60% and 10% of the subscore, respectively (Niche, 2019b). Since a rural area is more likely to be economically diverse instead of racially diverse when compared with an urban school, this metric also puts the rural schools at a disadvantage in the overall rankings.  

Both U.S. News and World Report’s and Niche’s ranking methodologies continue to enforce the view that the “one type fits all” larger school is the best type of school. As the National Rural Education Association Report (Bard, Gardener, & Wieland, 2006) discusses, even though research does not support the idea that larger, city schools are the best type of school, this idea pervades American educational thinking. The omission of dual enrollment courses matters even more because people use the rankings to help decide where to move and where to send their children to school.

Accurate ranking information is critical so that parents and guardians have the information that they need to make an informed decision when relocating.

If parents believe that a rural school is subpar, then they are less likely to relocate to a rural area. This pattern exacerbates the population issues that rural areas already face and can lead to other problems like school consolidation and the ensuing shrinking of communities. 

While reports ranking schools provide easy access to simplified comparisons of schools, the metrics used to rank the schools contain assumptions about what is important to consider when comparing educational success. An intentional analysis of the underlying assumptions in these ranking methodologies is vital so that rankings do not unintentionally favor a particular size and type of school. It is the responsibility of major ranking systems to make sure that their rankings are as accurate as possible because the use of the rankings has real effects on communities throughout the country. Similarly, it is their responsibility to make sure that their methodologies do not fall prey to location bias and lack of understanding of educational systems in different settings.

[1] The NCES report (Thomas et al., 2013) contains a breakdown of the type of program (dual enrollment, AP/IB, or both) by type and size of school.  While schools in each category use dual enrollment for courses, the town and rural schools have it as a much higher percentage of their advanced options, so they are more hurt by its absence from the ranking metric. The report also shows a high percentage of academic courses being taken in these areas through dual enrollment. The inclusion of dual enrollment with the AP/IB scores would make the rankings more accurate for all of the high schools involved, but particularly for those in areas that rely only on these college courses for advanced courses.

---

Beth Howd is a Ph.D. student in the Education Theory and Policy program at Penn State University.  She has over fifteen years of experience teaching chemistry and physics in multiple settings.  These experiences have created an interest in urban and rural education, school effectiveness, and STEM education.  She holds an M.Div. from Mercer University and B.S. from Furman University.

References

Bard, J., Gardener, C., & Wieland, R. (Winter 2006). National rural education association report: Rural school consolidation: History, research summary, conclusions, and recommendations. The Rural Educator, 27(2), 40-48.  Retrieved from https://eric.ed.gov/?id=EJ783851

Brooks, E. & Morse, R. (April 29,2019a). Demographic Breakdown of the 2019 Best High Schools.  Retrieved from https://www.usnews.com/education/best–highschools/articles/demographic–breakdown  

Brooks, E. & Morse, R.  (June 11, 2019b). How U.S. News Calculated the 2019 Best High Schools Rankings. Retrieved from https://www.usnews.com/education/best–highschools/articles/how–us–news–calculated–the–rankings

Niche. (2019a). Academic Grade Methodology.  Retrieved from https://www.niche.com/about/methodology/public–high–school–academics/

Niche. (2019b). 2020 Most Diverse Public Schools Methodology. Retrieved from https://www.niche.com/about/methodology/best–public–high–schools/

Thomas, N., Marken, S., Gray, L., Lewis, L. & Ralph, J. (February 2013). Dual Credit and Exam-Based Courses in U.S. Public High Schools: 2010-2011, First Look.  Retrieved from https://nces.ed.gov/pubs2013/2013001.pdf