GM 66.B.200 Examination by the CAA
CAA ORS9 Decision No. 1
1. Questions may be prepared in the national language but the use of aviation English is recommended wherever possible.
2. The primary purpose of essay questions is to determine that the candidate can express themselves in a clear and concise manner and can prepare a concise technical report for the maintenance record, which is why only a few essay questions are required.
3. Oral type questions may not be used as the primary means of examination because of the difficulty in establishing consistency of standards between examiners or day-to-day.
However, nothing prevents the CAA from meeting potential certifying staff for the purpose of ensuring they understand their obligations and responsibilities in the application of maintenance Parts.
4. For pass mark purposes, the essay questions should be considered as separate from the multiple choice questions.
5. Multiple choice question (MCQ) generation.
The following principles should be observed when developing multiple choice question:
(a) The examination should measure clearly formulated goals. Therefore the field and depth of knowledge to be measured by each question should be fully identified.
(b) All the questions should be of the multiple choice type with three alternative answers.
(c) Questions that require specialised knowledge of specific aircraft types should not be asked in a basic licence examination.
(d) The use of abbreviations and acronyms should generally be avoided. However where needed, only internationally recognised abbreviations and acronyms should be used. In case of doubt use the full form, e.g. angle of attack = 12 degrees instead of a= 12°.
(e) Questions and answers should be formulated as simply as possible: the examination is not a test of language. Complex sentences, unusual grammar and double negatives should be avoided.
(f) A question should comprise one complete positive proposition. No more than 3 different statements should appear among the suggested responses otherwise the candidate may be able to deduce the correct answer by eliminating the unlikely combinations of statements.
(g) Questions should have only one true answer.
(h) The correct answer should be absolutely correct and complete or, without doubt, the most preferable. Responses that are so essentially similar that the choice is a matter of opinion rather than a matter of fact should be avoided. The main interest in MCQs is that they can be quickly performed: this is not achieved if doubt exists about the correct answer.
(i) The incorrect alternatives should seem equally plausible to anyone ignorant of the subject. All alternatives should be clearly related to the question and of similar vocabulary, grammatical structure and length. In numerical questions, the incorrect answers should correspond to procedural errors such as corrections applied in the wrong sense or incorrect unit conversions: they should not be mere random numbers.
(j) Calculators are not allowed during examination. Therefore all calculations should be feasible without a calculator. Where a question involves calculations not feasible without a calculator, such as 
, then the question should specify the approximate value of .
(k) Questions should be in accordance with Part-66 examination syllabus (Appendix I and Appendix VII).
6. Essay question generation
(a) The purpose of the essay is to allow the CAA to determine if candidates can express themselves in a clear and concise manner in the form of a written response, in a technical report format using the technical language of the aviation industry. The essay examination also allows to assess, in part, the technical knowledge retained by the individual and with a practical application relevant to a maintenance scenario.
(b) Questions should be written so as to be broad enough to be answered by candidates for any A or B licence category or subcategories and comply with the following general guidelines:
— the question topic selected should be generic, applicable to mechanical as well as avionic licence categories and have a common technical difficulty level as indicated in Part-66 Appendix I or Appendix VII;
— cover technology applicable to most areas of aircraft maintenance;
— reflects common working practices;
— it is not type- or manufacturer-specific and avoids subjects which are rarely found in practice;
— when drafting a question, there is need to ensure consideration is given to the limited practical experience that most candidates will have.
(c) To make the questions and the marking procedures are as consistent as possible, each question and model answer, with the required key areas required (see below), should be reviewed independently by at least 2 technical staff members.
(d) When raising questions the following should be considered:
— Each essay question will have a time allowance of 20 minutes.
— A complete A4 side is provided for each question and answer, if required the answer can be extended onto the reverse side of the page.
— The question should be such that the answer expected will be at the level shown for that subject in the module syllabus.
— The question should not be ambiguous but should seek a broad reply rather than be limited in scope for answer.
— The question should lend itself to be written in a technical report style, in a logical sequence (beginning, middle and end), containing the applicable and relevant technical words needed in the answer.
— Do not ask for drawings/sketches to support the essay.
— The question should be relevant to the category and level of difficulty listed in the syllabus, e.g. a description of a typical general aviation system may not be acceptable for a typical commercial aeroplane.
— Subject to obvious constraints in relation to the topic being addressed the question should have a strong bias towards the practical maintenance of a system/component and the answer should show an understanding of normal and deteriorated conditions of an aircraft and its systems.
Variations on alternative possible answers which have not been thought of, may have to be taken into account to aid the examiner when marking. If considered relevant, the model answer should be amended to include these new points.
(e) Because of the difficulty in marking an essay answer using key points only, there is a need for the way in which the report was written to be assessed and taken into consideration.
(f) The total points for each question will add up to 100 and will need to reflect both the combination of the technical (key point) element and the report style element.
(g) Each key point will be graded upon its importance and have point weighting allocated to it. The total weight will represent 60% of the mark.
(h) Key points are the ‘important elements’ that may be knowledge or experience-based and will include other maintenance orientated factors such as relevant safety precautions or legislative practices if applicable. Excessive reference to the need for MM referral or safety checks may be considered wasteful.
(i) The question answer will be analysed for the clarity and manner in which the essay report is presented and have a weighting allocated to it which will represent 40% of the mark.
(j) The answer should show the candidate's ability to express himself in technical language. This includes readability of the language, basic grammar and use of terminology.
(k) The report starts in the beginning and has logical process to reach a conclusion.
(l) Supporting diagrams should not be encouraged but if used should supplement the answer and not replace the need for a broad text answer.
(m) The report should not be indexed, itemised or listed.
(n) Within reason the candidate should not be penalised for incorrect spelling.
(o) A zero mark should only be given in exceptional circumstances. Even if the student misunderstands the question and gives an answer to a different question, a sympathetic mark even if only for the report style should be given, this could up to the maximum percentage allowed.
(p) The two allocated marks should be added together and written into the answer paper.
(q) If a question answer resulting in a borderline failure is principally due to ‘written report errors,’ the paper should be discussed and the mark agreed if possible with another examiner.