The more frequently you receive information about a situation, the better your decisions are likely to be. Or so it would seem—at least up until the point that you’re too distracted by incoming data to think about it. Recent studies by the authors suggest that the upper limit for the benefits of increasing data frequency are much lower than that. In their studies of undergraduate students, Lurie and Swaminathan found that receiving information more frequently led to worse decisions, particularly when there was more “noise” (in the form of random fluctuations) in the data. If it applies more broadly, this result is significant in an age where managers can use information technology to track information in real time and obtain frequent feedback on their decisions.
In their first study, the researchers found that experiment participants who received reports and placed orders daily had lower profits than their peers who got reports once or twice weekly. The worsened performance was especially pronounced when there was a high variance in actual demand. The main problem appears to be what is known as “recency bias”: Students with daily information were more likely to give too much weight to the previous day’s data in making their decisions, rather than looking at a longer time period. The same tendency showed up in a second experiment where participants could access records of past sales. Participants tended to limit their information gathering to the most recent data. This study showed that “increasing levels of feedback leads to performance declines for both high-and low-profit markets.” A third and fourth study reinforced these results.
When decision makers receive more frequent feedback in environments characterized by random noise, their performance declines. The decision makers respond to the faster flow of information by excessively focusing on more recent data. They also fail to sufficiently compare information across multiple time periods. The authors note that the greater the variance in the data, the more likely it is that more frequent reporting can skew decision making, because each individual data point becomes less representative.
Based on these results, the authors urge caution in the design and implementation of real-time information systems. There’s nothing necessarily wrong with real-time data, but it does matter greatly how the data are presented to managers. Among the ways to reduce the risks of examining data too frequently, the authors point to the use of moving averages that help to contextualize recent data and identify variance, and to statistical process control charts that filter data and identify exceptions.