FMCSA Outlines New Data Model to Be Tested in September
Federal Motor Carrier Safety Administration officials have withdrawn a slate of proposed “enhancements” to the Compliance, Safety, Accountability program’s Safety Management System to make way for a small-scale test in September of a new statistical data model intended to better identify high-risk carriers.
Those proposed enhancements eliminated by the agency included:
• Changing some of the SMS intervention thresholds to better reflect the Behavior Analysis and Safety Improvement Categories’ correlation to crash risk.
• Releasing motor carrier percentile rankings under the Hazardous Materials Compliance BASIC to the public.
• Reclassifying violations for operating while out of service under the Unsafe Driving BASIC rather than the BASIC of the underlying violation.
• Increasing the maximum vehicle miles traveled used to more accurately reflect the operations of high-utilization carriers.
FMCSA said the original enhancements represented a “continuation of the agency’s efforts to provide law enforcement, the motor carrier industry and motor carriers with more informative safety data.”
However, the future of the SMS will be based on a new statistical model being developed by FMCSA. The new model was outlined in a recent corrective action plan report to Congress in response to a slate of recommendations last year by an academic panel commissioned by the National Academy of Sciences.
The corrective action report, made public July 16, said FMCSA continues to develop cost estimates and benchmarks for its action plan and will work with the Department of Transportation Office of the Inspector General and plans to eventually implement the so-called Item Response Theory model recommended by the academic panel.
“The agency’s initial cost estimates, not including federal staff time, total approximately $2,500,000 over two years for implementation of the action plan,” FMCSA said. Included in that cost is a contract with the National Academy of Sciences for a new standing committee to assist in responding to the panel’s slate of sweeping recommendations. The standing committee includes subject matter experts on organization safety, statistics, research and evaluation methods, motor carrier operations, enforcement and IRT modeling.
Nas Correlation Study Corrective Action Plan Enclosure Final June 2018 by Transport Topics on Scribd
FMCSA’s benchmarks call for a small-scale test of the IRT model in September, a full-run scale test in April 2019 and evaluation of the effectiveness of the full-scale model by June. The agency said it is planning a public forum to discuss data issues and availability.
In response to the NAS study, FMCSA said it plans to improve registration data, consider external sources and voluntary reporting of data, and improve inspection software.
It also agrees with NAS that additional information about carrier operations might improve its analysis and identification of noncompliant motor carriers but that the collection of such data would come at a cost, and the benefits would be unknown.
In addition, FMCSA said it would develop a web page where researchers, carriers, safety consultants and the public can obtain simplified Motor Carrier Management Information System data snapshots. The agency also will be gathering public input from motor carriers, insurance companies and shippers regarding the ways in which the public uses CSA Safety Management System data.
The academy panel’s June 2017 study analyzed the ability of the SMS to discriminate between low- and high-risk carriers, assessed the public usage of SMS, reviewed the data and methodology used to calculate the measures and provided advice on additional data collection and safety assessment methodologies.
The 132-page, $1 million, 15-month study by the 12-member panel of academics — several who specialize in statistics and transportation safety policy — was funded by FMCSA and mandated by the FAST Act of 2015.
In an attempt to simplify the IRT concept, FMCSA officials and NAS academics often have compared it to SAT tests used in high schools. A student’s SAT score is not based on the sheer number of correct answers. Rather, those who have a mastery of a subject such as math or science are determined by how many of the difficult questions they can answer correctly.
Similarly, with the IRT model, the agency’s safety culture ranking could be restructured based on an analysis of violation numbers using the violation data to create rating categories.
