Opinion: Safety Culture Falls Outside Government Purview

Should the federal government be measuring your company’s safety culture? The short answer is “no.” Absolutely not.

Dainoff

In 2017, a National Academy of Sciences panel of experts issued recommendations to improve the Federal Motor Carrier Safety Administration’s Safety Measurement System, on which the agency’s Compliance, Safety, Accountability safety scoring framework is based.

There are two points in these recommendations on which to focus.

The first is the proposal to use an advanced statistical system called Item Response Theory to restructure the method by which vehicle inspection data is translated into behavioral categories.

Currently, SMS creates a scoring system based on a set of “Behavior Analysis and Safety Indicator Categories” or BASICs. Each of these categories reflects some aspect of the real-time state of: (1) the driver (unsafe driving, hours-of-service compliance, fitness, controlled substance/alcohol); (2) the vehicle (maintenance, hazmat); and/or (3) interactions between the two (crash indicators) as assessed by on-road inspections.

The second proposal is to add a new group of variables detailing information on company operations, including turnover rate, type of cargo and information on compensation method. These new variables would require additional sources of data and new methods of data collection.

The first of these recommendations is a great idea; the second … not so great. Here’s why:

According to the NAS panel, the current SMS already is considered a measure of safety culture. The new recommendations supposedly will improve the current method of measuring safety culture, but this argument is problematic in several ways.

One problem is the very term “safety culture.” Recent articles in safety science journals expressed concern that the very notion of a “safety culture” is so overused and misused that it has lost validity as a scientific construct. There is virtually no consensus on even a definition for safety culture, let alone a method for measuring it. For that reason, more than a few objections to the NAS approach to safety culture could be raised. The details are technical, but this is not just academic nitpicking; the practical implications for the future of SMS are crucial.

The proposal from NAS would refine this effort through Item Response Theory, presumably improving the ability for IRT-based BASIC scores to predict crashes.

This is exactly what a governmental regulatory agency should do — define a standard for acceptably safe characteristics of driver and vehicle and monitor compliance with that standard. The fact that this process is labeled “measuring safety culture,” however, is an unfortunate misnomer.

While the NAS panel did not formally define safety culture, it explicitly made the assumption that since some fraction of crashes are due to carrier operations, it is within the FMCSA purview to measure attributes of carrier operations as part of safety culture. These include: scheduling, logistics, driver turnover, fatigue management, and compensation type and level. These variables would be entered into a predictive model so that if FMCSA issued an intervention to a carrier, it would be possible to observe which combination of these variables were modified by the carrier.

A problem, however, is that these metrics, although important to consider and admittedly a strong indicator of a company’s attention to safety, are not, and should not, be considered a comprehensive assessment. Trucking companies are complex systems with many organizational and technical components whose interactions impact safety. Addressing this level of complexity requires a more comprehensive systems approach, which cannot be obtained through quantitative analytics alone.

At this point, we enter a region of overlap among safety management, science and public policy. This expanded definition of safety culture appears to be extreme regulatory overreach. It raises all sorts of red flags regarding misuse of big data, not to mention privacy and security. More fundamentally, this goes beyond the appropriate regulatory function described above, which is to set and monitor standards for vehicle and operator behavior. If a carrier’s drivers and vehicles fall below these standards, it is necessary and appropriate for the regulator to intervene and require the carrier to remedy the situation. However, it is intrusive and inappropriate for the regulator to be concerned with the details of how this is to be accomplished operationally. This is particularly true when the underlying science is unsettled. Bottom line: FMCSA should pursue the IRT modification to the current SMS and drop the expansion effort of collecting operational data.

At the same time, we can’t throw out the baby with the bath water. There is no disagreement among experts that organizational factors can have an important influence on safety outcomes, and formative tools have been developed through scientific research which can be utilized by companies to effectively manage these issues. If various perspectives on safety culture tell us anything, it is that we need to go beyond holding the driver solely responsible for maintaining safety.

Marvin Dainoff, PhD, CPE has been doing ergonomics research in safety and health for more than 40 years. He is a past president of the Human Factors and Ergonomics Society. He was director of the Center for Behavioral Sciences at the Liberty Mutual Research Institute for Safety, where his focus included trucking safety.