Flawed Safety Data Penalizing Many Fleets Under CSA Program, ATA Report States
This story appears in the Dec. 16 print edition of Transport Topics.
Flawed data and methodology used in the Compliance, Safety, Accountability program have created “tens of thousands” of statistical anomalies, according to an American Trucking Associations white paper released last week.
“ATA continues to support the objectives of CSA and to call for improvements to the program,” ATA President Bill Graves said. “However, data and methodology problems continue to plague the system and the accuracy and reliability of companies’ scores.”
The paper indicated that the relationship between safety percentile scores and crash risk can be skewed by problems including:
• Lack of data, particularly on small carriers, which constitute the bulk of the industry.
• Several regional enforcement disparities.
• Questionable assignment of severity weights to individual violations.
• Underreporting of crashes by states.
• Inclusion of crashes that were not caused by carriers.
• Increased exposure to crashes experienced by carriers operating in urban environments.
The ATA white paper included some of its own observations but mostly cited past research compiled by the American Transportation Research Institute and the Federal Motor Carrier Safety Administration.
“Researchers have arrived at mixed conclusions with respect to the reliability of SMS scores in identifying unsafe [crash prone] motor carriers,” ATA said. “Some found virtually no correlation between scores and crash rates in any of the measurement categories.”
Rob Abbott, ATA’s vice president of safety policy, said the paper was intended to “enlighten discussions of CSA” and “alert folks to the scores’ relevance and utility in identifying individual carrier crash risk.”
ATA’s paper said ATRI’s statistical data analysis found a positive relationship between Behavioral Analysis Safety Improvement Categories scores and crashes in three of the publicly available measurement categories: the Unsafe Driving, Hours of Service Compliance and Vehicle Maintenance BASICs.
On the other hand, ATRI found a negative relationship between scores in the other two publicly available BASICs and crash involvement — the Driver Fitness and Controlled Substances and Alcohol BASICs. In those two BASICs, higher, or worse, percentile scores were found to be associated with lower crash risks.
Two other BASICs, the Crash Indicator and Hazardous Materials Compliance, are not available for public viewing.
However, the paper said scores in the Hazmat Compliance BASIC “reflect the likelihood of future hazardous materials violations but not the propensity to be involved in crashes.”
The paper also noted that, “FMCSA has consistently demonstrated that Crash Indicator BASIC scores are a strong predictor of future crash involvement.”
“It may make sense for FMCSA to use scores in those categories that correlate positively with crash risk to prioritize companies for enforcement review,” Graves said. “In the process, FMCSA can verify whether or not the scores paint an accurate picture. But third parties need to know that for the purposes of drawing conclusions about individual carriers, the scores are unreliable.”
ATA’s paper said that a fleet’s crash rate may be as much a “reflection of happenstance as their safety practices.”
David Parker, chairman of a specially appointed ongoing CSA subcommittee commissioned by FMCSA Administrator Anne Ferro, said the ATA white paper well summarizes some of the challenges facing the CSA program.
“The authors and researchers are seeing the same things I am seeing — and that other people are calling to our attention,” Parker said. “There’s nothing in the report that shocks me. We’ve diagnosed the symptoms, now we have to work on the cure.”
Parker, who calls CSA a “work in progress,” believes the program’s basic concepts are sound but expects that it could take years for the subcommittee to complete its work.
