GAO Critical of CSA Data
This story appears in the Feb. 10 print edition of Transport Topics.
The U.S. Government Accountability Office said last week that some data included in the federal government’s Compliance, Safety Accountability program do not have a “strong predictive relationship with crashes.”
The Feb. 3 report from GAO, a congressional investigation agency, also said the Safety Measurement System, a key part of the Federal Motor Carrier Safety Administration’s CSA program, faces challenges because “most regulations used to calculate SMS scores are not violated often enough to strongly associate them with crash risk for individual carriers.”
In addition, GAO said most carriers lack sufficient safety performance data to ensure that FMCSA can reliably compare them with other carriers.
Two days after the GAO report, FMCSA issued its study, claiming the CSA system is “working effectively as a prioritization tool” in identifying carriers that are at a high risk for crashes. FMCSA’s report said it is a better method of rating carriers than the previous SafeStat system.
SafeStat used safety performance information such as federal and state data on crashes and roadside inspections to measure a carrier’s safety fitness.
The GAO report was developed from an analysis of nearly 315,000 U.S.-based carriers. It concluded the CSA program is increasing industry interest in safety and is providing FMCSA with more tools to increase interventions with carriers.
But it was critical of the CSA data, claiming that for the estimated 800 violations included in the SMS, only two — speeding and failure to wear a seat belt while operating a commercial vehicle — consistently appeared as stable predictors of crashes.
American Trucking Associations commended GAO’s findings and urged FMCSA to remove all carriers’ scores from public view in light of GAO’s study.
“The GAO’s review of FMCSA’s Compliance, Safety, Accountability program was comprehensive, thoughtful and balanced,” ATA President Bill Graves said. “While ATA has long supported CSA’s objectives, we can’t help but agree with GAO’s findings that the scores produced by the program don’t present an accurate or precise assessment of the safety of many carriers.”
Graves also criticized FMCSA’s findings, saying that, “just because CSA is an improvement over previous programs does not make it a ‘good’ program for assessing the safety performance of individual carriers.”
The Owner-Operator Independent Drivers Association said the GAO study suggested that FMCSA revise its CSA methodology to account for limitations in analyzing roadside data and the lack of inspections.
“Since most of trucking is made up of small businesses, and the CSA system required a minimum level of information — based upon violations — for a carrier to be scored, there is a huge gap in making relevant conclusions about the vast majority of fleets since many carriers have no score at all,” Todd Spencer, executive vice president of OOIDA, said in a statement.
“In addition to misallocating enforcement resources, flaws in the CSA scoring system are having significant negative business impacts on small-business trucking companies operated by career-long, accident-free truckers,” he added.
Although FMCSA and DOT said they would consider the GAO recommendations, they expressed what GAO described as “significant and substantive disagreements with some aspects of our analysis and conclusions,” according to the study.
Meanwhile, the Feb. 5 FMCSA study concluded that the carriers identified for CSA intervention have a 79% higher future crash rate (4.82 crashes per 100 power units) than the group of carriers not identified for interventions (2.69 crashes per 100 power units).
“Results show that the companies the SMS would have identified for interventions, such as roadside inspections, warning letters and on-site investigations, had a future crash rate of more than double the national average,” FMCSA said.
Analysis of an “effectiveness test” by the Volpe Center, a DOT internal research agency, provided “solid evidence that the SMS as a tool is effectively
supporting FMCSA in its mission to reduce crashes, injuries and fatalities involving large trucks and buses by improving safety and compliance,” FMCSA said.
“These results also show that SMS is identifying carriers with higher future crash rates across the spectrum of the carrier sizes and over varying amounts of carrier safety data,” the FMCSA study said. “This allows the CSA program to hold a large portion of the motor carrier industry accountable.”
ATA said the FMCSA assessment mirrored GAO’s findings with respect to the limited amount of data available on smaller carriers.
“Despite the fact that fleets with five or fewer trucks represented 75% of the carriers in the study, the authors acknowledged that there is very little available safety information on these carriers to “make a meaningful safety assessment,” ATA said.
The Transportation Department’s inspector general also is analyzing CSA, and members of a special FMCSA advisory CSA subcommittee will discuss the two studies at their Feb. 12 meeting, said David Parker, the subcommittee’s chairman.
