I’d lose my HR nerd card if I didn’t occasionally read some academic articles about HR. My wife needs to get in her eye roll when I get excited about the delivery of a new issue of Human Resources Management Review or the Journal of Behavioral and Experimental Economics.

I will also admit I’m a bit behind. You know… work happens. I need a vacation to catch up on things.

I was recently reading the March edition of the Human Resources Management Review, which was a special issue focused on meta-analysis. Meta-analysis is a research approach designed to combine results from multiple studies to develop more generalized or broader insights. It’s kind of like checking multiple sources before deciding something is true.

One article in the study is titled Collective assessment of the human resources management field: Meta-analytics needs and theory development prospects for the future. Yes, that all one title. At the time of posting this, you can download the article for free (no promises on how long that lasts).

This article, authored by a team from Florida State University, did an assessment of other meta-analyses conducted across HR disciplines. It’s a study of studies of studies.

Hubba-What?

They studied meta-analyses, which are themselves studies of studies.

Maybe you don’t find that as fun as I do.

Beyond the novelty of the concept, I started to realize that their work helped articulate a phenomenon I’ve felt over time about the level of rigor we see in researching “best practices.”

In the introduction to the special issue, authors Dianna L. Stone and Patrick Rosopa note four key advantages of using meta-analysis (their words italicized below):

  1. It provides a better estimate of the relation that exists in the population than single studies (which are samples of the population)
  2. The estimates are more precise because there is an increased amount of data and statistic power (more data leads to crisper conclusions)
  3. Hypothesis testing and biases associated with publications can be examined (checking across samples removes the bias of the study/publication)
  4. It helps remove inconsistencies in research, and identifies potential moderating or mediating variables (bigger sample pressure tests and refines conclusions)

All of these are great outcomes. I would then assume that this type of work would happen a lot, since it leads to nice robust outcomes. We might expect that the conclusions drawn in fields highly populated with meta-analysis work are more robust. It’s not a stretch to argue that academics are smart, and thus there will be more studies where the advantages of meta-analyses exist, and fewer studies where the limitations are numerous.

Following nicely down the path of understanding these pros and cons, the team from Florida State evaluated all of the meta-analyses out there in the HR space. The graphic below summarizes what they found, with the full table below.

Table:

HRM Topic Area Number of Studies
Strategic HRM 11
Staffing (internal and external, including promotions, succession, recruitment, and selection) 78
Performance Evaluation 24
Compensation and Rewards 9
Development (training, socialization, mentoring) 34
Organzational Withdrawal (turnover and absenteeism) 11

Staffing is the big winner. Again. I won’t give them any more attention. #soreloser

I was surprised to see Withdrawal so low given all of the emphasis on managing turnover. The authors note that there are over 1,000 studies about turnover in the last century, but for some reason there is less focus on piecing those together.

Compensation and rewards is the biggest loser. The authors note that the “lack of enough primary research studies” is a driver. Interesting that a very quantifiable aspect of HR isn’t particularly quantified. I’ve noticed that in my compensation work: there are lots of “best practices” but not a lot of detailed work to prove them. I do know that some great work is done within companies to analyze and understand the impact of compensation/rewards, but the academic community has not emphasized advancing that body of knowledge. It’s a shame… rewards is kind of a big deal.

The rest of the article talks about research needs in each area, which I would generally agree with and hope to see someday. When good ones come around, I’ll be sure to share.