Sustainment Testing

Developmental and operational testing continue after the system is produced and fielded. Such testing, planned for and executed in a combined manner whenever feasible, may be necessary when threat or user requirements change, modifications or upgrades become necessary, or even to investigate the impact of “aging” on the systems ability to perform its intended mission. In short, testing will likely continue as long as the system provides a viable capability in the joint combat environment, and perhaps even beyond that if it becomes necessary to test for the best manner of disposing of the system once its useful mission life is over.

Requirements

It is imperative that the Developmental Test (DT) and Operational Test (OT) communities participate in the requirements generation process. Their insights into the "testability" of performance requirements will not only facilitate successful test activities and the transition from development to production, but will also ensure the customer receives the mission-effective capability they need in the field.

Pre-Acquisition

During the Concept Refinement and Technology Development phases (that is prior to the start of a formal acquisition program) analysis of competing concepts and the technologies required to successfully implement a “concept of choice” are early activities that necessitate the involvement of the T&E community. A project which undergoes a Milestone A decision is required to prepare a document called the T&E Strategy. This strategy, while focused mainly on the application of modeling and simulation on the program, lays an important foundation for the formal T&E strategy document called the Test and Evaluation Master Plan (TEMP). The TEMP is a required Milestone B decision document which considers all elements of testing necessary during the life of an acquisition program. During Technology Development early prototypes should be evaluated from a combined developmental and operational test perspective as much as possible to best affect the eventual design and development of a system solution for the user.

The complexity of modern weapon systems demands that the discipline of Test and Evaluation (T&E) be integrated throughout the defense acquisition process. T&E experts must be involved as requirements are defined, concepts are considered, technologies are explored, designs are developed, systems are produced and fielded, and as these same systems are operated and sustained during their useful lifetime. T&E activities, whether conducted by the developing contractor, government acquisition team, or the operational community must be structured to maximize test planning and execution such that the system progresses from developmental to operational testing in a seamless manner. Testing in this way ensures the development test community certifies system performance such that the service Operational Test Agency (OTA) gets no surprises when it executes a dedicated operational test. A important tool that the DoD 5000 prescribes throughout the test [and overall acquisition] process is modeling and simulation. Properly applied modeling and simulation can reduce the amount of “live” testing required for a system during its development.

System Development

Early activity during System Development makes use of modeling and simulation to ensure full integration occurred during System Integration. It then capitalizes on that outcome to provide performance feedback which enables the system developer to finalize the system design and prove its performance in the intended operational environment. Combined developmental and operational testing of refined prototypes and engineering development models should be pursued wherever practical during this phase. Tests performed by the service Operational Test Agency (OTA) called an Operational Assessments (OA) may be accomplished during this phase, particularly if the program is required by law to conduct formal operational tests as part of the full production decision for a program. This formal test is called the Initial Operational Test and Evaluation (IOT&E).

Development Test

System Demonstration is the government's chance to ensure the system will operate in its intended environment. DT activities assess whether the Engineering Development Models (EDMs) satisfy the performance requirements specified in the development contract. The DT community is encouraged to use realistic test scenarios that give insight into how the system will perform in its operational environment.

Operational Test

Initial Operational Test and Evaluation (IOT&E) is conducted by the service Operational Test Agency, which for the Air Force is AFOTEC, the Air Force Operational Test and Evaluation Command. IOT&E is a mandatory requirement for selected acquisition programs and is accomplished to ensure a system is operationally effective and suitable in its intended mission environment; interoperability and safety are important elements of this crucial evaluation. The development community provides what is commonly referred to as “production representative system” (PRS) to the OTA for this test. The results of an IOT&E directly impact the full-rate production decision made by the system's Milestone Decision Authority. Combined developmental and operational testing increases the potential for successful IOT&E and can dramatically reduce the scope and expense of this mandated test.