DOT Inspector General Urges More SafeStat Improvements
By Sean McNally, Senior Reporter
This story appears in the July 2 print edition of Transport Topics.
WASHINGTON — The Federal Motor Carrier Safety Administration’s safety rating system has been improved but is still not ready to return to public view, said the Department of Transportation’s inspector general.
“We found that, although im-provements have been made, problems still exist with the reporting of crash data,” the inspector general said in a June 19 report.
As a result, the report said, the system needs more work before the agency gives the public access to its results.
The trucking industry has been critical of SafeStat, arguing its data, and the calculations used to tabulate safety rating scores were insufficient and inaccurate.
FMCSA Administrator John Hill told Transport Topics June 26 that, while the agency would review both the inspector general’s report and a recent Government Accountability Office study of the SafeStat program, he would like to repost the scores at some point.
FMCSA removed fleets’ SafeStat scores from its public Web site in 2004 after an earlier inspector general report highlighted inaccuracies. FMCSA still uses the data internally to initiate safety enforcement activities (2-23-04, p. 1).
According to the new report, “The effectiveness of the SafeStat scoring and ranking calculations is highly dependent on the quality of the crash-data file, which in the past was missing a substantial number of reportable crashes.”
The IG said in its most recent study that it found “anomalies that caused us to question the completeness of nonfatal crash reporting.”
“During our review, FMCSA acknowledged the need to develop a new, more reliable estimate and has begun work to implement the estimate,” the report said.
Hill said the agency was working with the University of Michigan Transportation Research Institute on a new way “to properly identify all crashes from states,” including the number of nonfatal truck crashes, and that data will be included in a revision of the FMCSA’s data-quality map.
FMCSA uses the map to highlight data-reporting problems from states, and Hill said once its work with the Michigan institute is complete in the fall, the agency is “going to start changing the metrics and elevating the standards for data quality.”
“Before FMCSA allows public access to SafeStat scores, it must improve its ability to measure the completeness of nonfatal crash reporting,” the IG’s report said, adding that FMCSA should complete its review and test the new nonfatal crash estimator “before making SafeStat scores available to the public.”
“What you’re going to eventually see is a measure for nonfatal crashes to be reported, whereas currently we use only fatal crashes,” Hill said.
Dave Osiecki, vice president of safety, security and operations for American Trucking Associations, said the report “confirms that many of the DOT-reportable crashes are not being reported by the states.”
“In combination, this report, with the recent GAO report, really lay out a road map for FMCSA to get a better, more reliable SafeStat,” Osiecki said.
On June 11, GAO said FMCSA’s current method for calculating safety scores could be improved by using a new mathematical formula (6-18, p. 5).
Both the GAO report and the recent IG report did say FMCSA has improved the quality of SafeStat data over the past three years.
“If you read the report from GAO and the inspector general, one resounding theme that comes out of those reports is that the data quality has improved significantly,” Hill said. “That’s a big step for us.”
In highlighting some of the reports’ findings, Hill said that, unlike the GAO, the IG offered no solutions to the agency.
“What is interesting is that they didn’t really have any good solutions for us. At least the GAO gave us a solution here,” he said. “I think what the IG is saying here is, ‘We realize you’ve got a big problem, and we don’t have any good ideas for helping you fix it, but we think it needs to be fixed.’ ”
Despite the problems highlighted by GAO and the IG, Hill said he was still “a believer in SafeStat.”
“My intent is to put the best data up as possible, and I want to see what GAO says about that and what the IG says about that,” he told TT.
However, Hill said, because of the review of nonfatal crash data and a second audit by GAO of FMCSA’s safety enforcement program, a final decision on SafeStat was not likely to come before 2008.
This story appears in the July 2 print edition of Transport Topics.
WASHINGTON — The Federal Motor Carrier Safety Administration’s safety rating system has been improved but is still not ready to return to public view, said the Department of Transportation’s inspector general.
“We found that, although im-provements have been made, problems still exist with the reporting of crash data,” the inspector general said in a June 19 report.
As a result, the report said, the system needs more work before the agency gives the public access to its results.
The trucking industry has been critical of SafeStat, arguing its data, and the calculations used to tabulate safety rating scores were insufficient and inaccurate.
FMCSA Administrator John Hill told Transport Topics June 26 that, while the agency would review both the inspector general’s report and a recent Government Accountability Office study of the SafeStat program, he would like to repost the scores at some point.
FMCSA removed fleets’ SafeStat scores from its public Web site in 2004 after an earlier inspector general report highlighted inaccuracies. FMCSA still uses the data internally to initiate safety enforcement activities (2-23-04, p. 1).
According to the new report, “The effectiveness of the SafeStat scoring and ranking calculations is highly dependent on the quality of the crash-data file, which in the past was missing a substantial number of reportable crashes.”
The IG said in its most recent study that it found “anomalies that caused us to question the completeness of nonfatal crash reporting.”
“During our review, FMCSA acknowledged the need to develop a new, more reliable estimate and has begun work to implement the estimate,” the report said.
Hill said the agency was working with the University of Michigan Transportation Research Institute on a new way “to properly identify all crashes from states,” including the number of nonfatal truck crashes, and that data will be included in a revision of the FMCSA’s data-quality map.
FMCSA uses the map to highlight data-reporting problems from states, and Hill said once its work with the Michigan institute is complete in the fall, the agency is “going to start changing the metrics and elevating the standards for data quality.”
“Before FMCSA allows public access to SafeStat scores, it must improve its ability to measure the completeness of nonfatal crash reporting,” the IG’s report said, adding that FMCSA should complete its review and test the new nonfatal crash estimator “before making SafeStat scores available to the public.”
“What you’re going to eventually see is a measure for nonfatal crashes to be reported, whereas currently we use only fatal crashes,” Hill said.
Dave Osiecki, vice president of safety, security and operations for American Trucking Associations, said the report “confirms that many of the DOT-reportable crashes are not being reported by the states.”
“In combination, this report, with the recent GAO report, really lay out a road map for FMCSA to get a better, more reliable SafeStat,” Osiecki said.
On June 11, GAO said FMCSA’s current method for calculating safety scores could be improved by using a new mathematical formula (6-18, p. 5).
Both the GAO report and the recent IG report did say FMCSA has improved the quality of SafeStat data over the past three years.
“If you read the report from GAO and the inspector general, one resounding theme that comes out of those reports is that the data quality has improved significantly,” Hill said. “That’s a big step for us.”
In highlighting some of the reports’ findings, Hill said that, unlike the GAO, the IG offered no solutions to the agency.
“What is interesting is that they didn’t really have any good solutions for us. At least the GAO gave us a solution here,” he said. “I think what the IG is saying here is, ‘We realize you’ve got a big problem, and we don’t have any good ideas for helping you fix it, but we think it needs to be fixed.’ ”
Despite the problems highlighted by GAO and the IG, Hill said he was still “a believer in SafeStat.”
“My intent is to put the best data up as possible, and I want to see what GAO says about that and what the IG says about that,” he told TT.
However, Hill said, because of the review of nonfatal crash data and a second audit by GAO of FMCSA’s safety enforcement program, a final decision on SafeStat was not likely to come before 2008.
