Should Payscale’s Earnings Data Be Trusted?

Despite the large amount of money spent on higher education, prospective students, their families, and the public have historically known very little about the earnings of students who attend college. This has started to change in recent years, as a few states (such as Virginia) began to publish earnings data for their graduates who stayed in state and the federal government publishes earnings data for certain programs through gainful employment rules. But this leaves out many public and private nonprofit institutions, and complete data are not available without a student unit record system.

As is often the case, the private sector steps in to try to fill the gap. has collected self-reported earnings data by college and major among a large number of bachelor’s degree recipients (those with a higher degree are excluded—the full methodology is here). Their 2014 “return on investment” report ranked colleges based on the best and worst dollar returns, with Harvey Mudd College at the top with a $1.1 million return over 20 years and Shaw University at the bottom with a return of negative $121,000.

Payscale data is self-reported earnings among individuals who happened to look at Payscale’s website and were willing to provide estimates of their annual earnings. It’s my strong suspicion that self-reported earnings from these individuals are substantially higher than the average bachelor’s degree recipient, and these are often based on a relatively small number of students. For example, the estimates of my alma mater, Truman State University, are based on 251 graduates for a college that graduates about 1,000 students per year. As many Truman students go on to get advanced degrees, probably about 500 students per year would qualify for the Payscale sample. Yet 102 students provided data within five years of graduation—about four percent of graduates who did not pursue further degrees.

But is it still worth considering? Yes and no. I don’t put a lot of stock in the absolute earnings listed, since they’re likely biased upward and there are relatively few cases. Additionally, there is no adjustment for cost of living—which really helps colleges in expensive urban areas. But the relative positions of institutions with similar focuses in similar parts of the country are probably somewhat close to what complete data would say. If the self-reporting bias is similar, then controlling for cost of living and the composition of graduates could yield useful information.

I hope that Payscale can do a version of their ROI estimates taking cost of living into account, and try to explore whether their data are somewhat representative of a particular college’s bachelor’s degree recipients. Although I commend them for providing a useful service, I still recommend taking the dollar value of ROI estimates with a shaker of salt.

About these ads

About Robert

I am an assistant professor of higher education at Seton Hall University. All opinions are my own.
This entry was posted in Uncategorized and tagged , , . Bookmark the permalink.

5 Responses to Should Payscale’s Earnings Data Be Trusted?

  1. TRM says:

    “those with a higher degree are excluded”

    I have been thinking about this issue for some time now and its applicability to long-term wage outcomes and have decided it may be wrong. It makes sense on the surface, but it ignores the fact that many individuals have to accumulate some kind of ongoing learning to stay up to date, or even licensed. Often this is in the form of some kind of continuing education credit, however it may be degree-eligible credit.Since these activities cannot be excluded, even in fields like engineering and education where we know continuing education is required, I see no reason to exclude higher level degrees simply on the basis we can identify them more readily. It all serves the same purpose.

    • Robert says:

      Agreed. I think that including at least master’s degrees as a separate category would be useful. However, including graduate degrees gets (deeper) into the issue of multiple institutional contributions–something which is probably easier to avoid without a large sample size.

      • TRM says:

        Right. Which is also why we report each degree level separately. And our entering cohort based outcomes limit associate or bachelor degrees, as appropriate.

  2. Pingback: Notes in the Margins: Porn studies, padded CVs and Payscale data | On Campus | Minnesota Public Radio News

  3. Pingback: How Well Does College Pay? It Depends on Who You Ask | Israel Foreign Affairs

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s