Appendix E to GM 21.A.101 Procedure for evaluating material contribution to safety or impracticality of applying latest certification specifications to a changed product

CAA ORS9 Decision No. 1

E.1 Introduction.

E.1.1 The basic principle of enhancing the level of safety of changed aeronautical products is to apply the latest certification specifications for significant changes to the greatest extent practical. In certain cases, the cost of complying fully with a later certification specification may not be commensurate with the small safety benefit achieved. These factors form the basis where compliance with the latest standard may be considered impractical, thereby allowing compliance with an earlier certification specification. This Appendix gives one method of determining whether compliance with a later certification specification is impractical; however, it does not preclude the use of other methods for improving the safety of aeronautical products.

E.1.2 The CAA recognises that other procedures can be used and have historically been accepted on a case-by-case basis. The acceptance of results through the use of these procedures may vary from state to state. Consequently, they may not be accepted through all bilateral certification processes. Regardless of which method is used, the process must show that a proposed certification basis is able to achieve a positive safety benefit for the overall product.

E.1.3 Regarding impracticality, any method used must encourage the incorporation of safety enhancements that will have the most dramatic impact on the level of safety of the aircraft while considering the effective use of resources. This important point is illustrated graphically in Figure E-1 below. This Figure notionally shows the interrelation between the total resources required for incorporating each potential safety enhancement with the corresponding net increase in safety benefit.

Figure E-1. Safety Benefits versus Resources

E.1.4 Typically, it is found that, for impractical certification basis changes, there are proposals that can achieve a positive safety benefit that are resource-effective. Conversely, there are proposals that may achieve a small safety benefit at the expense of a large amount of resources to implement them. Clearly, there will be a point where a large percentage of the potential safety benefit can be achieved with a reasonable expenditure of resources. The focus of the methods used should be to determine the most appropriate certification standards relative to the respective incremental cost to reach this point.

E.1.5 This Appendix provides procedural guidance for determining the material contribution to the level of safety, or the practicality of applying a certification standard at a particular amendment level to a changed product. The procedure is generic in nature and describes the steps and necessary inputs that may be used on any project to develop a position.

E.1.6 The procedure is intended to be used, along with good engineering judgment, to evaluate the relative merits of a changed product complying with the latest certification standards. It provides a means, but not the only means, for applicants to present their position regarding an exception under point 21.A.101(b)(3).

E.1.7 The certification basis for a change to a product will not be at an amendment level earlier than the existing certification basis.

E.2 Procedure for evaluating the material contribution or impracticality of applying the latest certification specifications to a changed product.

The following are steps to determine the material contribution or impracticality of applying a certification specification at a particular amendment level.

E.2.1 Step 1: Identify the regulatory change being evaluated.

In this step, applicants should document:

E.2.1.1 The specific standard (e.g. CS 25.365),

E.2.1.2 The amendment level of the existing certification basis for the standards, and

E.2.1.3 The latest amendment level of the certification specification.

E.2.2 Step 2: Identify the specific hazard that the certification specification addresses.

E.2.2.1 Each certification specification and its subsequent amendments addresses a hazard or hazards. In this step, the specific hazard(s) is (are) identified. This identification will allow for a comparison of the effectiveness of the amendment levels of the certification specification in addressing the hazard.

E.2.2.2 In many cases, the hazard and the cause of the hazard will be obvious. When the hazard and its related cause are not immediately obvious, it may be necessary to review the explanatory note (EN) and/or the impact assessment (IA) in the ED Decision by which the certification specification or its amendment was adopted. It may also be helpful to discuss the hazard with the responsible CAA team.

E.2.3 Step 3: Review the consequences of the hazard(s).

E.2.3.1 Once the hazard is identified, it is possible to identify the types of consequences that may occur due to the hazard. More than one consequence can be attributed to the same hazard. Typical examples of consequences would include but are not limited to:

— incidents where only injuries occurred,

— accidents where a total hull loss occurred,

— accidents where less than 10 per cent of the passengers died,

— accidents where 10 per cent or more passengers died, and

— engine- and propeller-specific hazards.

E.2.3.2 The explanatory note (EN) and/or the impact assessment (IA) in the ED Decision may provide useful information regarding the consequences of the hazard that the certification specification addresses.

E.2.4 Step 4: Identify the historical and predicted frequency of each consequence.

E.2.4.1 Another source for determining impracticality is the historical record of the consequences of the hazard that led to a certification specification or an amendment to a certification specification. From these data, a frequency of occurrence for the hazard can be determined. It is important to recognise that the frequency of occurrence may be higher or lower in the future. Therefore, it also is necessary to predict the frequency of future occurrences.

E.2.4.2 More than one consequence can be attributed to the same hazard. Therefore, when applicable, the combination of consequences and frequencies of those consequences should be considered together.

E.2.4.3 The explanatory note (EN) and/or the impact assessment (IA) in the ED Decision may provide useful information regarding the frequency of an occurrence.

E.2.5 Step 5: Determine how effective full compliance with the latest amendment of the certification specification would be in addressing the hazard.

E.2.5.1 When each amendment is issued, it is usually expected that compliance with the certification specification would be completely effective in addressing the associated hazard for the designs and technology envisioned at the time. It is expected that the hazard would be eliminated, avoided, or mitigated. However, experience has shown that this may not always be the case. It is also possible that earlier amendment levels may have addressed the hazard but were not completely effective. A product may also contain a design feature(s) that provides a level of safety that approaches that of the latest certification specifications, yet is not fully compliant with the latest certification specifications. Therefore, in comparing the benefits of compliance with the existing certification basis to the latest amendment level, it is useful to estimate the effectiveness of both amendment levels in dealing with the hazard.

E.2.5.2 It is recognised that the determination of levels of effectiveness is normally of a subjective nature. Therefore, prudence should be exercised when making these determinations. In all cases, it is necessary to document the assumptions and data that support the determination.

E.2.5.3 The following five levels of effectiveness are provided as a guideline:

1. Fully effective in all cases. Compliance with the certification specification eliminates the hazard or provides a means to avoid the hazard completely.

2. Considerable potential for eliminating or avoiding the hazard. Compliance with the certification specification eliminates the hazard or provides a means to completely avoid the hazard for all probable or likely cases, but it does not cover all situations or scenarios.

3. Adequately mitigates the hazard. Compliance with the certification specification eliminates the hazard or provides a means to avoid the hazard completely in many cases. However, the hazard is not eliminated or avoided in all probable or likely cases. Usually this action only addresses a significant part of a larger or broader hazard.

4. Hazard only partly addressed. In some cases, compliance with the certification specification partly eliminates the hazard or does not completely avoid the hazard. The hazard is not eliminated or avoided in all probable or likely cases. Usually this action only addresses part of a hazard.

5. Hazard only partly addressed but action has a negative side effect. Compliance with the certification specification does not eliminate or avoid the hazard or may have negative safety side effects. The action is of questionable benefit.

E.2.5.4 If it is determined that compliance with the latest certification specifications does not contribute materially to the product’s level of safety, applicants should skip Step 6 of this Appendix and go directly to Step 7 to document the conclusion. If it is determined that complying with the latest amendment of the certification specification contributes materially to the product’s level of safety, applicants should continue to Step 6 of this Appendix.

E.2.6 Step 6: Determine the incremental resource costs and cost avoidance.

E.2.6.1 There is always cost associated with complying with a certification specification. This cost may range from minimal administrative efforts to the resource expenditures that support full-scale testing or the redesign of a large portion of an aircraft. However, there are also potential cost savings from compliance with a certification specification. For example, compliance with a certification specification may avoid aircraft damage or accidents and the associated costs to the manufacturer for investigating accidents. Compliance with the latest amendment of a certification specification may also help a foreign authority to certify a product.

E.2.6.2 When determining the impracticality of applying a certification specification at the latest amendment level, only the incremental costs and safety benefits from complying with the existing certification basis should be considered.

E.2.6.3 When evaluating the incremental cost, it may be beneficial for applicants to compare the increase in cost of complying with the latest certification specifications with the cost of incorporating the same design feature in a new aircraft. In many cases, an estimate for the cost of incorporation in a new aircraft is provided by the CAA in the regulatory impact assessment, which was presented when the corresponding certification specification was first issued. Incremental costs of retrofit/incorporation on existing designs may be higher than that for production. Examples of costs may include but are not limited to the following:

Costs

The accuracies of fleet size projections, utilisation, etc., may be different from those experienced for derived product designs and must be validated.

— Labour: work carried out in the design, fabrication, inspection, operation, or maintenance of a product for the purpose of incorporating or demonstrating compliance with a proposed action. Non-recurring labour certification specifications, including training, for the applicant supporting development and production of the product, should be considered.

— Capital: construction of new, modified, or temporary facilities for design, production, tooling, training, or maintenance.

— Material: costs associated with product materials, product components, inventory, kits, and spares.

— Operating costs: costs associated with fuel, oil, fees, training, and expendables.

— Revenue/utility loss: costs resulting from earning/usage capability reductions from departure delays, product downtime, and performance loss due to seats, cargo, range, or airport restrictions.

— The cost of changing compliance documentation and/or drawings in itself is not an acceptable reason for an exception.

Cost Avoidance.

— Avoiding costs of accidents, including investigation of accidents, lawsuits, public relations activities, insurance, and lost revenue.

— Foreign certification: conducting a single effort that would demonstrate compliance with the certification specifications of most certifying authorities, thus minimising certification costs.

E.2.7 Step 7: Document the conclusion.

With the information from the previous steps documented and reviewed, the applicant’s position and rationale regarding whether complying with the latest certification specifications contributes materially to the product’s level of safety or its practicality can be documented.

CAA records the determination of whether the conditions for the proposed exception were met. That determination is based on the information and analysis provided by the applicant in the preceding steps. If the determination to grant the exception is based on the product’s design features, those features are documented at a high level in the TCDS. Documentation in the TCDS is required so that the features are maintained during subsequent changes to the product, therefore, maintaining the product’s agreed level of safety. If the results of this analysis are inconclusive, then further discussions with the CAA are warranted.

E.3 Examples of how to certify changed aircraft.

The following examples illustrate the typical process an applicant follows. The process will be the same for all product types.

E.3.1 Example 1: FAR § 25.963, Fuel Tank Access Covers.

NOTE: This example is taken from the FAA’s certification experience, so references to FAR sections and amendments are kept.

This example is part of a significant change to a transport aeroplane that increases the passenger payload and gross weight by extending the fuselage by 20 feet (6.1 metres). To accommodate the higher design weights and increased braking requirements and to reduce the runway loading, the applicant will change the landing gear from a two-wheel to four-wheel configuration; this changes the debris scatter on the wing from the landing gear. The CAA will require the new model of the aeroplane to comply with the latest applicable certification specifications based on the date of application.

The wing will be strengthened locally at the side of the body and at the attachment points of the engines and the landing gear, but the applicant would not like to alter the wing access panels and the fuel tank access covers. Although the applicant recognises that the scatter pattern and impact loading on the wing from debris thrown from the landing gear will change, the applicant proposes that it would be impractical to redesign the fuel tank access covers.

Note: Points 21.B.107(a)(3) or 21.B.111(a)(3) may be an additional reason why CAA would require compliance with CS 25.963(e), regardless of the ‘significant’ determination.

E.3.1.1 Step 1: Identify the regulatory change being evaluated.

The existing certification basis of the aeroplane that is being changed is Part 25 prior to Amendment 25-69. Amendment 25-69 added the requirement that fuel tank access covers on transport category aeroplanes be designed to minimise penetration by likely foreign objects, and that they be fire-resistant.

E.3.1.2 Step 2: Identify the specific hazard that the certification specification addresses.

Fuel tank access covers have failed in service due to impact with high-energy objects, such as failed tire tread material and engine debris following engine failures. In one accident, debris from the runway impacted a fuel tank access cover, causing its failure and subsequent fire, which resulted in fatalities and loss of the aeroplane. Amendment 25-69 will ensure that all access covers on all fuel tanks are designed or located to minimise penetration by likely foreign objects, and that they are fire-resistant.

E.3.1.3 Step 3: Review the history of the consequences of the hazard(s).

There have been occurrences with injuries and with more than 10 per cent deaths.

E.3.1.4 Step 4: Identify the historical and predicted frequency of each consequence.

In 200 million departures of large jets, there was:

— 1 occurrence with more than 10 per cent deaths, and

— 1 occurrence with injuries.

There is no reason to believe that the future rate of accidents will be significantly different from the historical record.

E.3.1.5 Step 5: Determine how effective full compliance with the latest amendment of the certification specifications would be in addressing the hazard.

There is considerable potential for eliminating or avoiding the hazard. Compliance with Amendment 25-69 eliminates the hazard or provides a means to avoid the hazard completely for all probable or likely cases. However, it does not cover all situations or scenarios.

E.3.1.6 Step 6: Determine resource costs and cost avoidance. Costs.

— For a newly developed aeroplane, there would be minor increases in labour resulting from design and fabrication of new fuel tank access covers.

— There would be a negligible increase in costs related to materials, operating costs, and revenue utility loss.

Cost avoidance.

— There were 2 accidents in 200 million departures. The applicant believes that it will manufacture more than 2 000 of these aeroplanes. These aeroplanes would average 5 flights a day. Therefore, statistically there will be accidents in the future if the hazard is not alleviated. Compliance will provide cost benefits related to avoiding lawsuits, accident investigations, and public relations costs.

— There are cost savings associated with meeting a single certification basis for the CAA’s and foreign standards.

E.3.1.7 Step 7: Document the conclusion.

It is concluded that compliance with the latest certification specification increases the level of safety at a minimal cost to the applicant. Based on the arguments and information presented by the applicant through the certification review item (CRI) process, the CAA determined that meeting the latest amendment would be practical. The CAA has also found that fuel tank access covers that are not impact-resistant and fire-resistant, and which are located where a strike is likely, are unsafe features or characteristics which preclude the issue of a type certificate under 21.B.107(a)(3).

E.3.2 Example 2: FAR § 25.365, Pressurized Compartment Loads.

NOTE: This example is taken from the FAA’s certification experience, so references to FAR sections and amendments are kept.

This example is a passenger-to-freighter conversion STC. This change affects the floor loads on the aeroplane as well as the decompression venting.

E.3.2.1 Step 1: Identify the regulatory change being evaluated.

The existing certification basis of the aeroplane that is being changed includes § 25.365 at Amendment 25-00. The initial release of § 25.365 required the interior structure of passenger compartments to be designed to withstand the effects of a sudden release of pressure through an opening resulting from the failure or penetration of an external door, window, or windshield panel, or from structural fatigue or penetration of the fuselage, unless shown to be extremely remote.

Amendment 25-54 revised § 25.365 to require the interior structure to be designed for an opening resulting from penetration by a portion of an engine, an opening in any compartment of a size defined by § 25.365(e)(2), or the maximum opening caused by a failure that was not shown to be extremely improbable. The most significant change is the ‘formula hole size’ requirement introduced into § 25.365(e)(2) at Amendment 25-54.

Amendment 25-71/72 (Amendments 25-71 and 25-72 are identical) extended the regulation to all pressurised compartments, not just passenger compartments, and to the pressurisation of unpressurised areas. Pressurisation of unpressurised areas had previously been identified as an unsafe feature under § 21.B.111(a)(3).

Amendment 25-87 redefined the pressure differential load factor that applies above an altitude of 45 000 feet. Compliance with Amendment 25-87 is not affected since the aeroplane does not operate above an altitude of 45 000 feet. The applicant proposes to meet the ‘pressurisation into unpressurised areas’ requirement introduced in Amendment 25-71/72. The applicant does not propose to comply with the ‘formula hole size’ requirement introduced in § 25.365(e)(2) at Amendment 25-54.

E.3.2.2 Step 2: Identify the specific hazard that the certification specification addresses.

The hazard is a catastrophic structure and/or system failure produced by a sudden release of pressure through an opening in any compartment in flight. This opening could be caused by an uncontained engine failure, an opening of a prescribed size due to the inadvertent opening of an external door in flight, or an opening caused by a failure not shown to be extremely improbable. The opening could be caused by an event that has yet to be identified.

E.3.2.3 Step 3: Review the history of the consequences of the hazard(s).

There have been occurrences with injuries, with less than 10 per cent deaths and with more than 10 per cent deaths.

E.3.2.4 Step 4: Identify the historical and predicted frequency of each consequence. In 200 million departures of large jets, there were:

— 2 occurrences with more than 10 per cent deaths,

— 1 occurrence with less than 10 per cent deaths, and

— 1 occurrence with injuries.

— There is no reason to believe that the future rate of accidents will be significantly different from the historical record.

E.3.2.5 Step 5: Determine how effective full compliance with the latest amendment of the certification specifications would be at addressing the hazard.

Compliance with the latest amendment eliminates the hazard or provides a means to avoid the hazard completely.

Design changes made to the proposed aeroplane bring it closer to full compliance with

§ 25.365 at Amendment 25-54. The original aeroplane was shown to meet the requirements for a hole size of 1.1 square feet. Amendment 25-54 would require a hole size of 5.74 square feet, and the current reinforcements for the converted aeroplane can sustain a hole size of 3.65 square feet in the forward area and 2.65 square feet at the aft area. This is 3.1 and 2.4 times, respectively, better than the original design condition of Amendment 25-0 and is a significant improvement over the worldwide passenger fleet in service.

E.3.2.6 Step 6: Determine resource costs and cost avoidance. Costs.

There would be savings in both labour and capital costs if compliance were shown to Amendment 25-0 instead of Amendment 25-54. Major modifications to the floor beams would be necessary to meet the ‘formula hole size’ requirement in Amendment 25-54.

Cost avoidance.

There were 4 accidents in 200 million departures. The applicant believes that it will manufacture more than 2 000 of these aeroplanes. These aeroplanes would average 2 flights a day. Therefore, statistically there will be accidents in the future if the hazard is not alleviated. Compliance will provide cost benefits related to avoiding lawsuits, accident investigations, and public relations costs.

There are cost savings associated with meeting a single certification basis for FAA and foreign regulations.

E.3.2.7 Step 7: Document the conclusion regarding practicality.

The design complies with § 25.365 at Amendments 25-0, 25-71/72, and 25-87, and it is nearly in full compliance with Amendment 25-54. The design would adequately address the hazard at an acceptable cost. Therefore, based on arguments of impracticality discussed in an issue paper, the FAA accepts the applicant’s proposal to comply with

§ 25.365 at Amendment 25-0.

E.3.3 Example 3: FAR § 25.981, Fuel Tank Ignition Prevention.

NOTE: This example is taken from the FAA’s certification experience, so references to FAR sections and amendments are kept.

This example is part of a significant change to a transport aeroplane that increases passenger payload and gross weight by extending the fuselage by 20 feet (6.1 metres). To accommodate the longer fuselage, the applicant will modify systems wiring installations; this includes changing fuel tank system wiring. The new model of the aeroplane will be required to comply with the latest applicable certification specifications based on the date of application.

E.3.3.1 Step 1: Identify the regulatory change being evaluated.

The existing certification basis of the aeroplane that is being changed is Part 25 prior to Amendment 25-102 but includes Amendment 25-40.

Note: If the original certification basis does not include Amendment 25-40, the certification basis should be considered not adequate for fuel tank ignition prevention.

The 2001 Fuel Tank Safety (FTS) rule adopted Amendment 25-102 to add explicit requirements in § 25.981(a)(3) for demonstrating that the design precludes fuel tank ignition sources. This was required, but had in several cases not been properly applied in demonstrating compliance with §§ 25.901 and 25.1309. Amendment 25-102, § 25.981(b), added a requirement to develop fuel tank system airworthiness limitations to maintain the ignition prevention features of the design. Section H25.4, Amendment 25-102, requires the inclusion of those fuel tank system airworthiness limitations in the Airworthiness Limitations section of the Instructions for Continued Airworthiness (ICA).

Since the FAA policy for performing the failure analysis to demonstrate compliance with

§§ 25.901 and 25.1309 at Amendment 25-40 and 25-46 was adopted in the explicit fuel tank ignition prevention failure analysis requirements of § 25.981(a)(3), the incremental requirement for demonstrating compliance with the ignition prevention requirements of Amendment 25-102 is to develop and implement the fuel tank system airworthiness limitations instead of developing Certification Maintenance Requirements in accordance with § 25.901(b)(2) at Amendments 25-40 through 25-46 and AC 25-19A.

E.3.3.2 Step 2: Identify the specific hazard that the certification specification addresses.

The FAA issued the 2001 FTS rule to preclude fuel tank ignition sources because of a history of fuel tank explosions. The catastrophic TWA Flight 800 in-flight fuel tank explosion on July 17, 1996, caused the death of all 230 people on board.

E.3.3.3 Step 3: Review the history of the consequences of the hazard(s).

There have been occurrences with injuries, with more than 10 per cent deaths, less than 10 per cent deaths, and no deaths.

E.3.3.4 Step 4: Identify the historical and predicted frequency of each consequence.

The 1998 Aviation Rulemaking Advisory Committee Fuel Tank Harmonisation Working Group report documented the number of historical fuel tank explosions as 16, which caused a total of 539 fatalities.

There have been 2 additional fuel tank explosions since that report was issued:

— March 3, 2001: Thai Airways International Flight 114 experienced a fuel tank explosion on the ground that caused 1 fatality and 3 serious injuries. The explosion and subsequent fire destroyed the aeroplane.

— May 4, 2006: A Malaysia Airlines Boeing 727 experienced a wing tank low pressure explosion during ground operations. There was no fire and no injuries. The wing structure suffered significant damage.

There is no reason to believe that the future rate of accidents will be significantly different from the historical record if fuel tank system airworthiness limitations are not included in the ICA as is permitted in earlier amendment levels.

E.3.3.5 Step 5: Determine how effective full compliance with the latest amendment of the certification specifications would be at addressing the hazard.

There is considerable potential for eliminating or avoiding the hazard.

In the 2008 Fuel Tank Flammability Reduction (FTFR) rule, the FAA estimated that compliance with the ignition prevention requirements of Amendment 25-102, together with the fuel tank ignition prevention airworthiness directives issued as a result of the Special Federal Aviation Regulation number 88 reviews, resulted in the range of effectiveness in preventing fuel tank explosions between 25 to 75 per cent with a median value of 50 per cent (73 FR 42449).

E.3.3.6 Step 6: Determine resource costs and cost avoidance. Costs.

— For newly developed designs, there would be minor increases in costs resulting from the identification and implementation of fuel tank system airworthiness limitations.

— There would be no increase in costs related to materials, operating costs, and revenue utility loss.

Cost avoidance.

There were 18 accidents in 200 million departures. The applicant believes that it will manufacture more than 2 000 of these aeroplanes or derivatives of these aeroplanes. These aeroplanes would average 5 flights a day. Therefore, statistically there will be accidents in the future if the hazard is not alleviated. Compliance will provide cost benefits related to avoiding fatalities and injuries.

E.3.3.7 Step 7: Document the conclusion.

The following is additional background on the specific hazard that the certification specification addresses:

As stated in the 2001 FTS rule under ‘Changes to Part 25’, § 25.981(a)(3) was adopted because the previous regulations (§§ 25.901 and 25.1309) were not always properly applied.

Section 25.901(b)(2), Amendments 25-40 through 46, requires in part preventative maintenance as necessary to ensure that components of the powerplant installation, which includes the fuel tank system, will safely perform their intended function between inspections and overhauls defined in the maintenance instructions. When demonstrating compliance with the requirements of § 25.901(b) for maintenance of fuel tank ignition prevention features, the policy has been that the applicant identify critical features as critical maintenance requirements using the guidance in AC 25-19A.