Body Armor and the Wrath of the GAO

I have a new favorite Federal Agency, and it's not the one that sends people to your home and takes everything you own away in a moving van when you forget to pay your taxes one year. It's the Government Accountability Office, or the "GAO" for short.

Our troops need the best body armor we can procure for them. The body armor we hand to our troops must be tested and rigorously evaluated. The GAO has issued a report that talks about how the military really didn't do a bang-up job of making sure that the body armor they were buying met the necessary standards [pdf alert].
During Preliminary Design Model testing the Army took significant steps to run a controlled test and maintain consistency throughout the process, but the Army did not always follow established testing protocols and, as a result, did not achieve its intended test objective of determining as a basis for awarding contracts which designs met performance requirements. In the most consequential of the Army’s deviations from testing protocols, the Army testers incorrectly measured the amount of force absorbed by the plate designs by measuring back-face deformation in the clay backing at the point of aim rather than at the deepest point of depression.

GAO, essentially then, caught the military administering and then believing in and citing fraudulent tests:
The following are examples of deviations from testing protocols and other issues that GAO observed:
  • The clay backing placed behind the plates during ballistics testing was not always calibrated in accordance with testing protocols and was exposed to rain on one day, potentially impacting test results.

  • Testers improperly rounded down back-face deformation measurements, which is not authorized in the established testing protocols and which resulted in two designs passing First Article Testing that otherwise would have failed. Army officials said rounding is a common practice; however, one private test facility that rounds told GAO that they round up, not down.

  • Testers used a new instrument to measure back-face deformation without adequately certifying that the instrument could function correctly and in conformance with established testing protocols. The impact of this issue on test results is uncertain, but it could call into question the reliability and accuracy of the measurements.

  • As a result, one design passed First Article Testing that would have otherwise failed. 
With respect to internal control issues, the Army did not consistently maintain adequate internal controls to ensure the integrity and reliability of test data. In one example, during ballistic testing, data were lost, and testing had to be repeated because an official accidentally pressed the delete button and software controls were not in place to protect the integrity of test data. Army officials acknowledged that before GAO’s review they were unaware of the specific internal control problems we identified.

Here's what the GAO is telling lawmakers, and, by and large, I don't expect our government to do backflips and panic over this. I expect them to hold the military accountable for administering bad tests, and here's a more detailed version of what I highlighted above:

During Preliminary Design Model testing the Army took significant steps to run a controlled test and maintain consistency throughout the process but did not always follow established testing protocols and, as a result, did not achieve the intended test objective of determining which designs met performance requirements as a basis for awarding contracts. The Army’s significant steps to run a controlled test included, for example, the consistent documentation of testing procedures using audio, video, and other electronic means and extensive efforts to maintain proper temperature and humidity in the test lanes. However, we identified several instances in which the Aberdeen Test Center deviated from testing protocols, including failing to test the ease of insertion of the plates into both pockets of the outer protective vest as required by the testing protocols; shooting several plates at the wrong velocity or location on the plate; and repeating failed clay calibration tests on the same block of clay—the latter having the potential to significantly affect test results. In the most consequential of the deviations from testing protocols, the Army testers incorrectly measured the amount of force absorbed by the designs tested by measuring back-face deformation at the point of aim rather than at the deepest point of depression.



Army testers recognized the error after completing about a third of the test and then changed the test plan to call for measuring at the point of aim and likewise issued a modification to the contract solicitation. At least two of the eight designs that passed Preliminary Design Model testing and were awarded contracts would have failed if measurements had been made to the deepest point of depression. The deviations from the testing protocols were the result of Aberdeen Test Center incorrectly interpreting the testing protocols.

Yes, that's the fraud here, and it's plain as day--the military was allowed to rig the tests in order to award contracts to body armor manufacturers without realizing that this could lead to the failure of the body armor in combat and the killing of our men and women in uniform. You can go read the 'he said, she said' section of the report, where the military and the GAO go back and forth on how the clay was subjected to calibration drop tests, and what you will find is that the GAO is essentially calling bullshit on a systematic effort to do slapdash testing in order to shut up Congress.

The controversy here comes when the GAO says, "here's what you need to do in order to get yourself in compliance with the expressed will of the U.S. Congress. One thing that the GAO would like to see happen is this:



To improve internal controls over the integrity and reliability of test data for future testing as well as provide for consistent test conditions and comparable data between tests, we recommend that the Secretary of Defense direct the Secretary of the Army to provide for an independent peer review of Aberdeen Test Center’s body armor testing protocols, facilities, and instrumentation to ensure that proper internal controls and sound management practices are in place. This peer review should be performed by testing experts external to the Army and DOD.



Fair enough. However, there's always pushback:



DOD did not concur with our recommendation for an independent evaluation of First Article Testing results and accordingly plans to take no action to provide such an assessment. DOD asserted that the issues we identified do not alter the effects of testing. However, based on our analysis and findings there is sufficient evidence to raise questions as to whether the issues we identified had an impact on testing results. As a result, we continue to believe it is necessary to have an independent external expert review these test results and the overall effect of the testing deviations we observed on those results before any armor is fielded to military personnel. Without such an independent review, the First Article Test results remain questionable, undermining the confidence of the public and those who might rely on the armor for protection. Consequently, Congress should consider directing the Office of the Secretary of Defense to either require that an independent external review of these body armor test results be conducted or that DOD officially amend its testing protocols to reflect any revised test procedures and repeat First Article Testing to ensure that only properly tested designs are fielded.



The GAO goes on to comment that:


In written comments on a draft of this report, DOD takes the position that our findings had no significant impact on the test results and on the subsequent contracting actions taken by the Army. DOD also does not concur with what it perceives as our two overarching conclusions: (1) that Preliminary Design Model testing did not achieve its intended objective of determining, as a basis for contract awards, which designs met performance requirements and (2) that First Article Testing may not have met its objective of determining whether each of the contracted plate designs met performance requirements.



Even though the GAO goes to great lengths to show the DOD that it screwed up, the DOD won't acknowledge it. Hence, U.S. troops are going into battle today with body armor that wasn't properly tested.We have bureaucrats being shown the error of their ways. If I'm missing something here, could someone please stop by and help this old man sit down in a chair and think about the error of his ways? I may need to go see someone about this problem I've been having, which is, I keep seeing blatant incompetence wherever someone does the legwork of pointing out that there is, umm, blatant incompetence here.

We've been involved in a continuous shooting war in two countries for over eight years, and the Poindexters can't get anything right. This is an absolute disgrace.

I don't know how anyone could keep their job after such a report, but nothing surprises me anymore. Our troops deserve the best equipment we can get them, and what they get instead is gear that is shoddy and poorly evaluated. It's not something you can just dismiss as being George W. Bush's fault, and even though the bulk of this testing happened on his watch, and on the watch of the current Secretary of Defense, Robert Gates, this is not a political issue. This is something that must be fixed as soon as possible. Congress needs to hold immediate and uncomfortable hearings over the matter. This is an incompetence and procurement issue, and it needs to be fixed because our troops deserve to have decent body armor.