Whether you like it or not, CSA is here to stay. Their goal is “to prevent commercial motor vehicle (CMV) crashes, fatalities, and injuries.” However, there is a current debate on whether or not the scoring from CSA’s Safety Measurement System (SMS) is a valid and reliable predictor of safety. Is this system used to determine which carriers are safe and which are a risk to the industry legitimate? Do those scores really aid in the prevention of accidents? Can a score predict the risk of being involved in an accident?
CSA scores are made public which means there is a lot at stake for a carrier and a driver. Shippers, receivers, lawyers, insurance agents, and judges all have access. The reason scores were made public was to have the marketplace apply pressure for carriers to improve their safety. A carrier is assigned a score. A high score means that the motor carrier’s performance has proven it is an unsafe carrier. A low score shows that the carrier is doing everything right and is a “safe” carrier. Or does it? Are these scores reliable? The problem is whether or not the method used to assign scores and determine crash risk fits the legal rules of evidence; is the evidence brought against a carrier “trustworthy” and does it rest on “a reliable foundation.”
Can a relationship be found between a carrier’s SMS score and the carrier’s crash risk? According to a study conducted by the American Transportation Research Institute (ATRI), there is a correlation between carrier score’s and crash rates in three of the seven BASIC categories. Unsafe driving had the strongest relationship with crash risk. Hours of service compliance and vehicle maintenance BASICs are the other two categories that had a strong correlation. What about the other four categories? Two of them actually showed a negative relationship: driver fitness and controlled substances and alcohol. Higher scores in these two categories corresponded with lower crash risk.
Here are the issues that make the CSA’s SMS scores unreliable:
- First, there is a lack of sufficient data to reliably assess the performance of carriers. According to the CSA’s findings, they only had enough data to assign scores to three percent of active carriers in all seven of the BASICS.
- Second, small carrier’s scores are volatile. Because CSA has so little data to work with, one violation for a small carrier can cause the score to jump significantly. A large carrier who has more data may not even notice a score change from one offense. For example, one small carrier with 13 trucks and 15 drivers drove 1.3 million miles in 2011. In the past two years, this fleet was involved in only one crash, with no injuries or fatalities. During the same time, it had a total of 32 driver inspections with an out-of-service rate of just 3%, slightly below last year’s national average of 4%. Of 32 total driver inspections, only three of them resulted in a total of four violations. All four were hours of service regulations. Only one was severe – falsifying record of duty status. So 29 out of 32 inspections were clean. If you do the math, that's over 90%. In school that would likely mean an "A." But not in the CSA grading system. Instead the CSA SMS ranks this carrier in the hours-of-service compliance BASIC at just above the intervention threshold of 65%, at 67.2%. Its scores in all the other safety categories were low or there were no rankings. Yet according to FMCSA this carrier is in “alert status.”
- Third, a study conducted by the University of Michigan found that many states underreport accidents. Some states reported less than 20% of qualifying crashes. CSA claims they have information on the carriers involved in most crashes; this would be the large carriers who have dozens or crashes a year. However, these carriers only compromise 10% of the industry. The problem is that the CSA does not have data on most of the carriers who are involved in crashes, the other 90% of the industry.
- Fourth, states also have differences in their enforcement of certain laws and regulations. A driver is much more likely to get a moving violation in Indiana (29%) than Mississippi (1.4%). Vigillo Inc. found that about half of all speeding tickets were written in ten states. A driver operating in one of these states may have higher scores due to stricter enforcement.
- Fifth, all crashes are weighted equally to determine an SMS score. CSA looks at crash involvement as the measurement, not fault. Is it true that a driver is less safe because of involvement in crashes, even when the accident is not their fault?
- Sixth, how violations are weighted has significantly skewed carrier’s scores. CSA gives higher weight to violations, not because they are more likely to cause a crash, but because the violation may make the results of the crash more severe. For example, not wearing a seatbelt carriers a weight of seven points. Can not wearing a seatbelt increase crash risk? No. It increases the severity of the accident, not the risk.
It is one thing to compare scores amongst carriers to rate their performance. However, it is another to try to judge the potential safety of a carrier based on scores that may not be a reliable measure of crash risk. The lack of data on smaller carriers, the system used to assign scores, the disparity in enforcement or underreporting of accidents amongst states, and the severity of weights assignment to particular violation all hinder CSA’s ability to reliably predict performance and risk.
CSA by the Numbers
- 758,000 commercial fleets are covered by CSA
- 166,000 (21%) of those fleets are identified as for-hire, interstate freight carriers
- 76% of interstate freight carriers have CSA data in their records
- 90% of interstate freight carriers with more than 500 trucks have at least one BASIC score
- 25% of interstate freight carriers with five or fewer trucks have at least one BASIC score
- 34% of interstate freight carriers have a numeric score for at least one BASIC
- 0.3% of interstate freight carriers have a numeric score for every BASIC
