Panel: FMCSA Safety Scoring Needs More Science Behind It

An advisory committee to the Federal Motor Carrier Safety Administration is recommending the agency collect more data to ensure its safety measurement system is based on science and not on the intuition or opinions of experts.

FMCSA asked the Motor Carrier Safety Advisory Committee to evaluate whether the Compliance, Safety, Accountability program’s seven performance measurement areas, called BASICs, and severity weights for violations are accurate predictors of crash risk.

The committee said in its report, made public in late December, that members had not been supplied enough data to make sound recommendations as to whether the scoring system of 1 to 10 points, depending on the violation, was related to crash probability.

“During the course of its discussions, MCSAC learned that the initial severity weightings were not all based on data but rather, in part, on the opinion of experts and others with some knowledge of accident causation,” the report said.

Although it said it lacked sufficient data, the committee made some general observations with the program and pointed out potential problems.

The group suggested that FMCSA better instruct shippers, brokers, insurance companies and financial institutions how CSA ratings should be used in evaluating carriers.

In recent months, shippers and brokers have criticized CSA data because they can be misleading or inaccurate when choosing a carrier.

The committee also said that:

• FMCSA should consider putting hazardous materials violations in a separate BASIC.

• Roadside violations, including proper lubrication, excessive oil leaks and fuse problems, are suspiciously being cited in large numbers.

• The violation of failing to maintain cargo tanks appears to be missing from the violation groups.

• Some nonmoving violations should be removed from the dangerous driving category.

• FMCSA should better define “fatigue.”

• The agency should consider intent when assessing points for logbook violations.

The committee also questioned the differing violations within the “speeding related” group.

“We recognize that neither we nor the agency has at this time all of the data to finalize all those [severity weight] numbers,” committee chairman David Parker, senior legal counsel with Great West Casualty Company, told Transport Topics. “We also recognize it as being a work in progress.”

Rob Abbott, vice president of safety policy for American Trucking Associations and a member of the committee, cautioned that members shared a “common concern” that the agency would “over-assume the significance of the committee’s recommendations.”

“Really, they reflect sort of guesses at the relationship between violations and crashes in terms of severity,” Abbott said. “We really need to look at the data that shows a causal relationship.”

Although FMCSA is issuing percentile scores for carriers that have logged roadside inspections, the agency has not yet issued a new rule outlining how it will use those scores to compile safety fitness determinations.

The agency has said it plans to announce that proposed rule in early 2012.

Abbott said the most recent “op-model” evaluation of CSA revealed that scores in two of the BASICs don’t bear a correlation to crash risk.

“In another category, their relationship isn’t very strong,” Abbott said. “Knowing that, we think it’s important to be sure that the weights and the methodology are based on data-connecting violations and crashes.”

In particular, Abbott said FMCSA has not yet made public a 2007 violation severity assessment study done by Volpe Center, Cambridge, Mass.

“We requested that, but it was not provided to us,” Abbott said.

FMCSA did not return calls seeking additional comment.