AMC No 2 to 21.B.100(b) Level of involvement (LoI) in UK technical standard order authorisation (UKTSOA) projects

CAA ORS9 Decision No. 1

The applicant for an UKTSOA is required to demonstrate its capability by obtaining the CAA’s agreement for the use of procedures that incorporate its specific design practices.

The assessment by the CAA that these procedures are properly applied is performed solely through the various UKTSOA projects of the applicant. No regular audits of the organisation are performed by the CAA outside the UKTSOA projects.

A properly completed Form 34 and the certification programme, including a technical description of the proposed design of the UKTSO article, are the basis for the determination of the CAA’s initial LoI.

The CAA assesses the compliance of the proposed UKTSO article with the UKTSO requirements as defined in the applicable CS-UKTSO standards, as well as compliance with Part 21 Subpart O (e.g. the declaration of design and performance (DDP), UKTSO marking, rating of performance, etc.). The UKTSOA applicant should deliver a complete data package per point 21.A.605.

The CAA’s LoI is further reassessed and adapted throughout the certification project until the UKTSOA is issued, depending on the applicant’s data, as well as on the UKTSO project changes regarding the applicant’s compliance demonstration (e.g. methods, design changes, deviations, limitations, problem reports, etc.).

1. Principles

The CAA’s LoI in UKTSO projects is defined based both on the responsibility of the CAA to assess the applicant’s demonstration of compliance, and on the risk evaluated, according to the following criteria:

— the applicant’s level of experience in the UKTSO process and scope of work;

— the applicant’s level of performance in the UKTSO scope of work;

— the use of novelties in the technology/design or in the means of compliance; and

— the complexity of the UKTSO article.

1.1. Applicant’s experience in the UKTSOA process and scope of work

This Section addresses the experience of the applicant’s organisation in the UKTSOA process, as well as in the scope of the certification basis of the UKTSO article, and of the related requirements. The presence of any of the following aspects contributes to the CAA’s identification of the risk related to the level of experience of the applicant in the UKTSOA process, or to the scope of work of the article:

— the applicant is new and has just applied for the acceptance of its procedures by the CAA, or it is the first project of the applicant after the CAA has accepted such procedures;

— the organisation has changed significantly the agreed procedures; and

— the scope of work of the UKTSOA project (UKTSO standards) is new to the applicant.

1.2. UKTSOA applicant’s performance within its scope of work

The UKTSOA applicant’s level of performance within its scope of work is evaluated using criteria that enable the CAA to identify risks in the applicant’s performance due to the following situations:

— the applicant has deficiencies in the procedures that it uses to demonstrate compliance with the certification requirements;

— the applicant has changed its methods or procedures to demonstrate compliance with the certification requirements;

— the assessment of the applicant’s compliance on previous projects in the same UKTSO scope of work has revealed significant issues in complying with the certification requirements, in the completion of data, or in the repetition of errors;

— the scope of work is new to the applicant’s team at the facilities where the project is developed, or the team had significant issues on preceding projects;

— the CAA has not conducted an UKTSOA project assessment of the applicant in the same UKTSO scope of work for a long period (i.e. 2 or 3 years); and

— the applicant did not regularly report minor changes or occurrences in a timely manner.

1.3. Novelty in the technology or in the means of compliance

A ‘novelty’ is understood to be the use of new technology, new sensors, new material, the use of new requirements or the use of new means of compliance. When an applicant is faced with a technology for the first time, or when that applicant is relatively unfamiliar with the technology, this is considered to be ‘novel’ even if other applicants may be already familiar with that technology.

Also related to novelty is the extent to which requirements, means of compliance or guidance need to be adapted due to particular novel features of the design. The following list includes some examples:

— recently issued CS-UKTSO standards, with which the applicant has limited experience;

— novel deviations;

— new guidance;

— new means of compliance (i.e. other than those previously applied by the applicant) or unusual means of compliance (different from the existing guidance material and/or different from industry standard practices);

— the use of new industry standards or new in-house methods, as well as the CAA’s familiarity with these new standards and methods;

— changes in methodology, tools or assumptions (compared with those previously applied by the applicant), including changes in software tools/programs.

Technology or means of compliance may be new/novel either from a global industry, applicant or the CAA perspective.

1.4. Complexity

Complexity may result from the design, technology, associated manufacturing process, compliance demonstration (including test set-ups or analysis), as well as from the variety of UKTSOs with which the applicant intends to comply, and their possible interactions.

The demonstration of compliance may be ‘complex’ for complex (or highly integrated) equipment, so it typically requires more effort from the applicant.

1.5. Criticality of the design and of the technology

The criticality levels of the design and of the technology of the UKTSO article are considered, but have a minor impact on the definition of the CAA’s LoI. The main reasons are:

— the assessment of UKTSO compliance is as important for an UKTSO article that hosts a critical function as it is for equipment that host less critical functions (e.g. flight data recorders); and

— the criticality of the design or technology is not always defined for an UKTSO article, and it may depend on the installation of the design or technology (e.g. a multifunction display), which may only occur later.

2. Determination of the CAA’s LoI

The CAA’s LoI in the assessment of the applicant’s compliance demonstration is determined by the CAA on the basis of the qualitative risk class and the CAA’s responsibilities in assessing the UKTSO project certification data package, together with the procedures for compliance with the UKTSO requirements (Part 21 Subpart O, and CS-UKTSO).

The CAA’s LoI is defined in the following paragraph 2.1 and, as per point 21.B.100(c), the the CAA’s LoI that is applicable to each project is notified to the applicant.

To every LoI class corresponds a list of activities that govern the CAA’s involvement. By means of these activities, the CAA verifies the demonstration of compliance (e.g. by document review and acceptance, test witnessing, sampling on the applicant’s site, desktop assessments, etc.).

The UKTSO applicant is responsible for providing a complete UKTSO certification data package.

2.1. Definition of the LoI classes

The CAA’s LoI for an UKTSO certification project is classified as one of the following:

— class high,

— class high reduced,

— class medium, or

— class basic.

Class ‘high reduced’ is, by default, the CAA’s initial LoI in an UKTSO project. The following is a description of each LoI class:

— High

The CAA evaluates and samples/checks in an extensive manner all the compliance data to assess the applicant’s demonstration of compliance with the applicable UKTSO standards. The CAA assesses the applicant’s DDP and general compliance with Part 21 Subpart O. the CAA performs desktop reviews, as well as on-site assessments of compliance demonstrations. This occurs when design and verification evidence is available.

— High reduced

The CAA assesses all the compliance data; sampling/checking is significant and adapted to the likelihood of an unidentified non-compliance. The sampling rate may be reduced if the content of the life cycle data provides confidence in compliance and is focused in the area where confidence needs to be gained. The CAA assesses the DDP and general compliance with Part 21 Subpart O. the CAA performs desktop reviews, as well as an on-site assessment of the applicant’s compliance demonstration. This occurs when design and verification evidence is available.

— Medium

The CAA assesses all the compliance data, but for some compliance data, it performs no or limited sampling/checking. The CAA adapts its sampling and focuses on the likelihood of an unidentified non-compliance, taking into account the level of complexity and novelty of the project. The CAA assesses the DDP and general compliance with Part 21 Subpart O. the CAA performs desktop reviews and may perform an on-site assessment of the applicant’s compliance demonstration.

— Basic

The CAA assesses the DDP and general compliance with Part 21 Subpart O, and verifies the completeness of the data package.

Generally, the CAA performs a desktop assessment.

3. The process of determining the CAA’s LoI

The determination of the CAA’s LoI is captured as a process. This process is performed mainly in three steps and is illustrated in the following figure:

Figure 1: Process of determination of CAA’s LoI in UKTSO certification projects

Step 1 consists of the initial LoI determination which the CAA evaluates by assessing:

— the applicant’s experience in the UKTSOA process and scope of work according to Section 1.1 above, and

— the UKTSOA applicant’s performance within its scope of work according to Section 1.2 above.

The result of this determination of the CAA’s initial LoI is either high or high reduced.

Step 2 consists of reassessing the CAA’s LoI. Throughout the UKTSO project, the CAA receives project deliverables (e.g. plans, reports), means of compliance, requests for deviations, limitations, etc., and interacts with the applicant.

If the CAA’s LoI has been initially set to high reduced, the CAA re-evaluates it considering:

— the novelty in the technology or in the means of compliance according to Section 1.3 above, and

— the complexity of the UKTSO project according to Section 1.4 above.

The result of this reassessment may vary from high to medium according to the following table:

Assessment results

LoI adaptation

The UKTSO article is novel and complex or a significant issue is detected during the compliance demonstration.

LoI is increased to high.

The UKTSO article is novel or complex or a new deviation is requested (1).

LoI is confirmed as high reduced.

The UKTSO article is non-novel and non-complex, no issue is detected during the compliance demonstration or method, and no novel deviation or new limitation is requested.

LoI is decreased to medium.

There is a major change with straightforward redemonstration of the UKTSO compliance (2).

LoI is reduced to basic.

1    It refers to deviations from UKTSO minimum operational performance standards (MOPSs), excluding deviations for requesting compliance with a new revision of an industry MOPS standard.

2    When the CAA agrees that a major change only requires a straightforward redemonstration of the UKTSO compliance using previous methods, without any identified risk, then the CAA’s LoI is reduced to basic. Please note that this may only be defined after a minimum assessment of the applicant’s compliance demonstration methods.

Note: For a minor change, this process does not apply; in that case, the CAA’s LoI consists of an assessment of the minor change classification, an update of the certificate, and, when needed, an assessment of the DDP.