INTERPRETING MEASURES OF EFFECTIVENESS

The ultimate goal of many of the programs discussed during the Understanding Interventions conferences is to increase the diversity of those in the PhD ranks, which typically has been translated as the number of underrepresented students entering doctoral programs by the end of a program’s funding period.

Chubin-1

Grant reviewers and programs have relied mainly on this metric to determine a program’s effectiveness. Yet this number remains relatively low. One of the plenary sessions at the conference explored issues involving these interpretations of efficiency. Are the right questions being asked? Is the character of programs being misinterpreted based on the data used? Are different metrics needed that better reflect institutional differences and the needs of students?

DePass-2

As explained by DePass, the sciences are actually better than many other fields at producing doctoral degrees among women, given the number of men and women who receive undergraduate degrees in these fields. For example, engineering-related fields produce 11.5 doctoral degrees per 100 undergraduate males who receive engineering-related bachelor’s degrees, while 19.1 women receive doctoral degrees for every 100 undergraduate women who receive degrees in these fields (Flaherty 2014). Even though the overall numbers of women receiving these degrees are relatively low, the “efficiency” of degree production is higher for women than for men.

DePass also pointed to the role of historically black colleges and universities (HBCUs) in providing pathways to STEM PhDs. As noted often at Understanding Interventions conferences, African-Americans make up 12 percent of the U.S. population and 11 percent of all undergraduate enrollments. However, they receive only 9 percent of STEM bachelor’s degrees, 7 percent of all bachelor’s degrees awarded in the biological sciences, 6 percent in the physical sciences, 5 percent in mathematics and statistics, and only 4 percent of the bachelor’s degrees awarded in engineering. HBCUs constitute only 3 percent of institutions of higher education in the United States, yet they produce 19 percent of all bachelor’s degrees in science and engineering awarded to Blacks in 2010 (Gasman and Nguyen, 2014). Approximately one in three Black students who earned bachelor’s degrees in mathematics and statistics attended HBCUs, as did 37 percent of all Black undergraduates who received bachelor’s degrees in the physical sciences.

HBCUs also are strongly overrepresented in terms of the number of students who go on to earn PhDs, said DePass. Among Black STEM PhD recipients who earned their degrees between 2005 and 2010, more than one third were conferred their undergraduate degrees by an HBCU, and 12 percent earned their doctorates at an HBCU.

 DePass-3

These statistics demonstrate that efficient mechanisms do exist for converting bachelor degrees into doctoral degrees for women in STEM and for producing Black undergraduate and PhD degrees in STEM fields. The question is how these results translate into policy, said DePass. “Where should we be spending money? Should we be trying to squeeze more blood out of the stone by holding institutions more accountable to produce more PhDs, or should we take advantage of these engines and . . . look at things like retention, graduation, and broader issues?”

 DePass-6

One way to answer these questions is to identify the strengths of programs and individuals and build on these strengths, said DePass. For example, the ability of HBCUs to produce students who earn PhDs in STEM fields could be studied more deeply and the lessons learned applied more broadly. However, HBCUs today have fewer resources to do such studies than they have had in the past, he added. DePass also noted that institutions have not sufficiently documented what they have been doing, which is one reason why people have not learned as much as they could have from past successes and failures. “That’s why it’s so important for us to make sure that those institutions that have something to say and contribute come to this table and say it. And hopefully that would mean that they got access to other tables where decisions” are being made.