What is 'data hubris'?

A couple of years ago I read an insightful article by Kate Murphy (no relation) in the Wall Street Journal (May 24, 2021, R1), titled “Don’t Give Up on Focus Groups.” In her article she examined how reliance on big data can lead us astray. Drawing conclusions from numbers alone is cleverly described as “data hubris.” And while the article deals with marketing research, the point applies just as well to compliance work.  

Recent years have seen a fervent pitch toward using big data to determine where to focus on compliance and ethics programs and whether the programs are actually working.  Guidance from the US Department of Justice, for example, makes it clear that if programs are to be credited, they need to engage in data analytics.  

But as the Wall Street Journal article made clear, over reliance on data analytics can lead to bone-headed mistakes. What always needs to be added is input directly from people. In one story, for example, Lego Group made a deep error by looking at larger trends and dumbing down its products. But when it gathered direct input from its target audience it gained the insight that led it to enormous success.  

My perspective is that in the compliance and ethics field we need both. In the area of antitrust compliance, for example, I pushed for the use of a data analytic  technique called “screening” in compliance programs.  I picked up on the important insights from Romy Abrantes-Metz, who championed screening in enforcement activities. Thereafter she, Patrick Bajari, and I produced the first article calling for use of this data analytical tool for antitrust compliance programs, Abrantes-Metz, Bajari & Murphy, “Antitrust Screening: Making Compliance Programs Robust”.  So I have long been a believer in the use of data analytics. 

But we need to recognize that data analytics is, and will never be more than, just one tool. There is no substitute for talking with your people.  The Wall Street Journal article discusses the value of focus groups.  I agree that focus groups will remain an important tool.  So is the old idea of “management by walking around.”  So is that highly technical, analytical tool known as “lunch,” i.e., sitting down with some of your folks at lunch and listening to what they have to say. When doing training, simply having the trainer hang around after the presentation and chat with people can offer insights not available from studies. 

Without question, surveys have an important place. Data analytics also offer an essential tool.  But don’t rely solely on numbers.  Don’t fall victim to data hubris.  Look at the numbers, but then go listen to your people.