An Alternative College Rankings System Offers a New Take on Influence

Prospective college students looking to narrow down their choices have relied on US News & World Report or The Princeton Review for generations, with a consistent handful of schools leading the pack in each year’s rankings.

Jed Macosko ’94 and a team of data scientists now offer an alternative way to assess institutions of higher education with the website Academic Influence. Macosko believes that traditional ranking systems rely too heavily on self-reported institutional data. Instead, his new site evaluates schools based on how achievements of their faculty and alumni are reflected in the public consciousness.

The status quo is something he’s observed firsthand as a physics professor at Wake Forest University. According to Macosko, traditional ranking systems “send emails to department chairs at different universities around the country and ask them to send back a list of schools that are the best in their department areas. A lot of people just don’t send them back,” he says.

He also believes that many traditional systems reward the same schools year after year.

“They’re not going to bump one school above another unless they have a strong reason, because it would create such a huge earthquake. It’s become their brand at this point,” he says. “It’s almost like each of the major rankings has picked their top school and they just stick with it.”

Instead, Academic Influence has set out to evaluate the impact and influence of people affiliated with the schools—faculty, alumni, and administrators—using what Macosko believes are more objective metrics: Crossref and Wikipedia.

Crossref is a vast research database similar to Google Scholar, but it makes its data available under a public domain license, Macosko says. It offers an in-depth way to measure scholarly output. Academic Influence also relies on the crowdsourced online encyclopedia Wikipedia, combing for references to people all over the world in direct relation to their particular fields—an ever-evolving catalog of achievements.

“The average Joe on the street doesn’t actually read these scientific papers or humanities journals,” Macosko explains. “Your real influence comes from how often you’re mentioned on Wikipedia, in terms of how you’re influencing the world.”

The average Joe on the street doesn’t actually read these scientific papers or humanities journals. Your real influence comes from how often you’re mentioned on Wikipedia.

One of the ways the site helps smaller (and possibly overlooked) schools get their due is by dividing each school’s tally of mentions by its total number of undergraduates for a Concentrated Influence score. Academic Influence’s inaugural 2021 rankings show the fruits of this approach, spotlighting schools on its top-10 liberal arts roster that rarely crack such lists, such as Hampshire, Reed, and Sarah Lawrence.

“A lot of smaller colleges say, ‘You know what? We offer a lot to these students because of our size and attention to students. We have influential faculty, and you get good access to them,’” Macosko says.

Academic Influence also offers a Custom College Ranking tool that pulls a personalized list of institutions based on the factors most important to the user. The site includes written summaries of its findings for many universities (including MIT), and showcases interviews conducted by Macosko and his team with influential figures from various ranked schools. (One recent interview was with Robert Langer ScD ’74, the David H. Koch Institute Professor at MIT, who counts Covid-19 vaccine developer Moderna among the many companies he has cofounded.)

Macosko thinks it’s tough to game his system—say, by tampering with Wikipedia entries, which his team downloads en masse at least once per quarter, if not more—because the data is so vast and because the editorial structure of Wikipedia provides a self-policing mechanism. (He points to 2014 research published in the journal PLOS One that determined that the open-access encyclopedia was 99.7 percent accurate, using drug information as a test subject area.) Academic Influence also tracks suspicious spikes and will downgrade accordingly, and the team omits celebrities from its formula to keep certain schools from soaring to the top.

Macosko’s team is also working to evaluate the potential in its system for baked-in bias—including against women and minorities—that could mirror bias in the source data.

“Our method of ranking people and universities could be susceptible to this kind of bias,” Macosko acknowledges. “But it turns out that our algorithm is not showing signs of racial or gender biases that could affect school rankings. When looking at schools using Concentrated Influence, which takes into account each school’s size, HBCUs [Historically Black Colleges and Universities] and all-female schools do quite well in our state-by-state rankings. Nevertheless, to be safe, we have two students this summer who are trying to investigate this potential problem. They are also addressing the problem of schools in non-English speaking countries being underranked by our algorithm, which improved when we added Crossref, but still has a ways to go,” he says.

Macosko started working on Academic Influence in 2016 with data scientists who had grown disillusioned by their work with longer-established ranking systems. Meanwhile, he was captivated by educational technology in the classroom. He was especially drawn to digital textbooks as a more flexible way to reach students, noting how printed textbooks became almost immediately out-of-date. He wanted to explore an equally responsive way to rank schools, feeling that Wake Forest was often underranked, too.

His motivation for the project is also personal: Macosko has five children, one of whom just completed the college application process. He hopes Academic Influence offers more personalized, evolving assessments for other families—and he appreciates the ability to tackle a common problem in a fresh way.

“Physics excites me and excites my curiosity. But so does solving a problem that many people have to face at some point, and that’s part of being an MIT alum,” he says.

Illustration by Mary Zyskowski/MIT Alumni Association