Opinion: Naturalistic Driving, Unnaturalistic Science

This Opinion piece appears in the June 29 print edition of Transport Topics. Click here to subscribe today.

Ronald R. Knipling, Ph.D.

President

Safety for the Long Haul Inc.

The Federal Motor Carrier Safety Administration has spent millions of dollars on naturalistic driving studies of commercial driver fatigue. Instead of looking directly at crashes, these studies put devices on trucks to record videos and other data on driver maneuvers. However, the recorded incidents, called “safety-critical events,” do not validly represent crashes, crash risk or crash causation.

Very few SCEs are crashes. Most are abrupt avoidance maneuvers such as hard braking or swerves. A 2011 Virginia Tech study on hours of service had just four crashes in 2,197 SCEs (0.2%). A crash was defined as any contact with objects or other vehicles — even if the damage was a scratch. Apparently, FMCSA and Virginia Tech believe that serious crashes, minor crashes, near-crashes and even less intense incidents are all so similar that combining them can pinpoint factors, including driver schedules, affecting crashes resulting in serious human harm.

The scientific advantages of NDS have been oversold. Yes, you can replay SCE videos to see and analyze driver actions, but how important are the events being analyzed? Yes, the large number of SCEs collected makes NDS statistically powerful. Statistical power is nice, but not if the data don’t portray the actual problem.

Yet, NDS is now the “go-to” method for HOS studies. The method is so implanted that Congress mandated an SCE-based NDS of different restart periods. That study is under way. But no one seems to care whether SCEs accurately portray the genesis of serious crashes — those causing the vast majority of harm.

There are plenty of reasons to doubt SCE validity. We know that crashes themselves are extremely varied. Crashes of different types (rear-end, road departure, etc.) and severity levels (fatal, injury, property damage only, non-police reported) have very different causal profiles.

The 240,000 or so annual police-reported property-damage-only large truck crashes are not statistically representative of the 75,000 or so causing injuries and deaths. For example, we know that night and day crashes differ in many ways. In 2012, 36% of fatal truck crashes occurred at night, versus just 18% of property-damage-only crashes. Serious crashes are more likely to be caused by driver misbehavior (speeding, tailgating) or impairment (alcohol use, fatigue), while minor crashes are more likely to involve common mistakes (“looked but did not see”).

Serious truck-car crashes usually are triggered by the car driver, while fault in minor crashes is split more equally. If minor crashes do not represent serious crashes, how can non-crashes represent them?

I’m a skeptic now, but in years past, I was an NDS insider and believer. I was FMCSA’s research chief when the first large studies were funded and performed. At Virginia Tech, I managed data collection for the landmark 100-Car Naturalistic Driving Study. I designed and supervised data analysis for the first big truck study to look at SCE characteristics and causes. That’s when I soured on the science.

SCE profiles simply did not match the objective profiles of crashes. For example, 43% of truck SCEs would have been rear-end hits into other vehicles, if the averted crash actually had occurred. In the Large Truck Crash Causation Study of serious crashes, the corresponding percentage was just 12%. Trucks were “at-fault” in 81% of their SCEs, versus 56% of LTCCS crash involvements. Imagine a national Gallup Poll with such disparities between the poll sample and the voter population. No one would accept it.

FMCSA’s HOS studies treat the SCE rate as a surrogate measure of fatigue, yet SCEs and fatigue are near-opposites. Most SCEs involve active, abrupt driver maneuvers in traffic. In contrast, fatigued drivers have lowered responsiveness and are usually alone on empty highways. SCEs peak during daytime rush hours, reflective of traffic interactions. Fatigue peaks during predawn hours, reflective of circadian physiology.

Using two different, established drowsiness measures, a Virginia Tech review of NDS videos found drivers most alert when they were having SCEs and least alert in non-SCE control periods. One would expect the opposite if SCEs were valid fatigue surrogates.

Bad surrogate measures are not unique to traffic safety. In medicine, some congestive heart medications increase cardiac function without improving survival. Treatments for diabetes can lower blood sugar without reducing complications or death rates.

Moral: Any surrogate measure must be validated against the real problem.

Who should be concerned about SCE validity? Scientists? Industry? The government? Taxpayers? Answer: All of the above.

HOS rules affect millions and cost billions. Spurious science results in bad regulatory decisions with potentially adverse effects on national productivity and safety. To HOS researchers, I say: “Get real.” Study real crashes or, if you must study surrogates, make sure they represent real risk.

Knipling is the author of “Safety for the Long Haul; Large Truck Crash Risk, Causation, & Prevention.” Safety for the Long Haul Inc., based in Arlington, Virginia, provides safety research, training and management consultation.