Procedures / Technical Documentation

Guidelines

Designing procedures and documentation is not different in concept from the design of other tools and systems for human use: it all requires the application of good human factors knowledge. It needs to use human factors principles of how to involve the user and how to incorporate knowledge about human performance and behavior.

Procedures overview

Design proceeds from the user outwards, not from the engineer downwards. Leaving the documentation design until the last part of the job, assigning it to the newest/rawest engineer, and imposing a rush deadline sets up even the most conscientious and capable AMT to make errors. In this section, we present a list of the steps needed to design and test a document for a procedure. These steps are a typical example of the user-centered design (UCD) process.

Note in the following guidelines, that the particular technology used to present the task cards to the AMT is not the main issue. These principles apply equally well to computer-based documents as to paper-based ones. A study comparing standard paper-based task cards with both well-designed task cards and computer-based task cards showed that 84% of the improvement was due to task card design, not to computerization. This should help you counter the argument that there is no need to redesign a task card as "we will soon be scrapping them and getting a computer-based system", which is often used as justification for putting off good design.

If you design the task cards correctly, then all of the effort is easily ported to the computer. In addition, you start reaping the reduced error benefits now instead of waiting for the inevitable changes and delays associated with any new software or hardware system. These guidelines will show where (and why) computer-based documents can have significant advantages, but none of these are worth delaying the design effort.

Collecting information for the procedure

A procedure can be no better than the information upon which it is based. The information must be accurate, specific and up-to-date. Modern aircraft come from the manufacturer supported by task details, diagrams and logic trees for performing maintenance and inspection tasks. These materials are updated frequently, indeed from an operator's perspective seemingly too frequently.

Collecting information

Obviously this information needs to be used, but it is not the only source of information. There are regulatory documents (e.g., ADs and ACs) that are used to identify tasks that apply to specific aircraft tail numbers. One must also take into account the training of AMTs. If the procedure is not consistent with the training, then errors are likely to occur. The solution is to change one to fit the other, not to expect the AMT to perform this reconciliation on the fly.

This consistency also applies in document wording, point of view in diagrams, ordering of procedure steps, and even to seemingly small changes such as where to place the sign-off boxes.

We need to make special mention of the specificity of each procedure. Each company's maintenance database contains lists of which equipment is on which aircraft in the fleet. There is no longer an excuse for written procedures that have several different sets of steps for different tail numbers. Again, if we do the work of sorting out the applicability / effectivity at the procedure design stage, we prevent confusion and errors by an AMT trying to determine which of several procedure steps to follow on a rainy midnight shift.

Finally, remember participative design. There are no reasons not to involve the AMT in the design of the procedure apart from time pressure and inconvenience. We have found procedures written for routine check inspections (e.g., overnight checks) that looked ridiculous to the AMT trying to follow them on the aircraft (see 2.2 Ordering the Activities below). A few minutes with an AMT who performs these tasks would have saved much effort, confusion and error potential.

Ordering the activities in the procedure

Some activities have only one logical order. For instance, you cannot access a control run until you have removed the access panel. But many tasks are composed of activities that can be re-ordered without problems. For example, an overnight check on an aircraft comprises many individual inspections of many systems. There are all of the tires to check, all of the brakes, all of the interior seats, all of the fluid levels.

Ordering the activities

The natural way for an engineer to order these activities is by system, asking for all tire checks then all brake checks. But AMTs know this is not the most efficient way for them to perform the procedure, so they typically use a spatial sequence rather than a functional sequence. This means performing all checks on each landing gear unit together (tires, brakes, any fluid levels) before moving to the next unit. The illustrations below show this example.

Sequence

Functional Sequence

Step #TaskChecked
4.0 Check Tire Pressures  
4.1 Check tire pressure L Main Landing Gear  
4.1 Check tire pressure R Main Landing Gear  
4.1 Check tire pressure Nose Landing Gear  
5.0 Check Brakes for condition  
5.1 Check brakes for condition L Main Landing Gear  
5.1 Check brakes for condition R Main Landing Gear  
5.1 Check brakes for condition Nose Landing Gear  
6.0 Check Hydraulic Fluid levels  
6.1 Check hydraulic fluid level L Main Landing Gear  
6.1 Check hydraulic fluid level R Main Landing Gear  
6.1 Check hydraulic fluid level Nose Landing Gear  

 

Spatial Sequence

Step #TaskChecked
4.0 Check Left Main Landing Gear  
4.1 Check tire pressure  
4.1 Check brakes for condition  
4.1 Check hydraulic fluid level  
5.0 Check Right Main Landing Gear  
5.1 Check tire pressure  
5.1 Check brakes for condition   
5.1 Check hydraulic fluid level  
6.0 Check Nose Landing Gear  
6.1 Check tire pressure  
6.1 Check brakes for condition  
6.1 Check hydraulic fluid level  

 

Writing the task card in the sequence that mechanics find natural prevents a mismatch between the task card and the AMT, thus preventing errors. Note also in the illustration that the number of sign-offs has been reduced (see Determining how many sign-offs are best later) In a study of an overnight check, AMTs much preferred spatial layout, and in controlled trials participants made fewer errors and used less time with a spatial layout.

Laying out the document

The overall goal when laying out a task card or an entire maintenance manual should NOT be to minimize the page count. Rather, the goal should be to make the procedure or document clear, unambiguous, easy to read, and easy to follow. A few pages less more is not going to bankrupt the company or save destroy the planet, but it could easily cause errors or lost time.

When using a task card, the AMT must be able to return to the correct point in the procedure after looking away, e.g., to perform a task step or answer a question. Anything that interferes with this ability to return to the proper location will increase the likelihood of errors, or at the very least add unnecessary delay.

Laying out the document

The best layouts have considerable white space and short paragraphs, like newspapers rather than novels. Check a newspaper (e.g., USA Today) to see how information is broken into small, self-contained pieces with frequent call-out headings and sub-headings so that readers do not lose their place after any interruption.

Example card layouts

A good technique is to start the procedure with a flow chart so the user can understand the whole procedure before starting to perform it. When AMTs pick up a task card, they typically scan through the whole procedure to get this overall mental model in place. Providing a flow chart can help them do this better and more rapidly.

When a procedure is at all complex, such as an inspection task with different paths depending on whether a defect is found, a flow chart is particularly important. It shows the overall logic of the procedure at a glance. Tests of well-designed documents with flowcharts have shown a spectacular reduction in errors compared to good current airline practice of task cards.

Flow chart

Within the procedure, clear indications of branching points are important. The convention in logic or computer programming is useful here with each branch starting with IF and the two outcomes starting with THEN and ELSE. Thus:

IF crack larger than 3mm is found on inner radius

THEN report crack and examine outer radius closely for evidence of corrosion
ELSE continue to next component and inspect inner radius

Note also the use of indentation to make the choice point even clearer.

Paginating the document

Pagination means figuring out where page breaks occur in a document. Something as simple as pagination can make a difference in error rates. The concept here is that there should be a minimum need for page turning at critical points in the procedure. When the written instructions are on one page and the associated diagram on the next page, this put unnecessary load on "working memory" and can lead to errors.

Paginating the document

Think through how the AMT will use the task card and paginate to avoid flipping backwards and forwards. When this was done for pilot procedures in Army helicopter actions, performance improved. This may mean that some space is "wasted" at the bottom of a page where an engineer concerned with minimizing paper costs could squeeze in a few extra lines of text. As noted above, however, this should not be an issue if the aim is error-proof procedures. In computer-based procedures, this is never an issue: one complete step per screen is an excellent concept working rule.

Using illustrations and call-outs

Procedures usually specify tasks that are inherently spatial in nature. For example, the instruction to "inspect the fuselage crown from station BS 83 to BS 85 between stringers 5L and 5R" is typical. The human brain has two distinct modes of operation - spatial and verbal - and moving between the two is inherently difficult and error-prone. We all know this from trying to receive driving directions, where hand gestures are a great addition to the verbal "turn the first left on toafter Main Street" directions.

In procedures, diagrams and illustrations play the same role as hand gestures in driving directions. They prevent the error-prone translation of spatial into text instructions (by the procedure writer) and the equally error-prone translation (by the AMT) of text back into spatial positions. For this reason, diagrams and illustrations should be plentiful and well designed. We are long past the era of computers, programs and printers that were suited only to text so this principle should be standard procedure.

The design of the diagrams themselves needs to be carefully done, again to reduce errors. Any illustration of aircraft structure is best done in two parts: a call-out from an overall picture of the aircraft structure (e.g. left wing, empennage or fuselage) to a larger view of the particular structure referenced in the text. The more detailed diagram should be shown from the viewpoint of the AMT performing the task, rather than from some standard or conventional viewpoint. Trying to puzzle out the relationship between the diagram in the task card and the actual structure visible to the AMT is again an error-prone activity.

Callout

Where there are specific tie-ins between the text and a diagram, these should be indicated by arrows from the text to the appropriate point on the diagram. In the helicopter example given above, each text step was placed in a box in order of performance down the left side of each page, with steps keyed to the diagram with arrows. This may not be possible within the software used to produce task cards for maintenance, so that arrows will typically be numbered with the step number of the name of the structure element or part. Note that these names need to be chosen carefully to be consistent throughout the procedure, and between procedure and training.

Illustration of callouts

The same name needs to be used each time for each part. Using different names for the same part sets up the AMT for errors. If the diagram is from a source that uses a different name for a part, it needs to be edited to be consistent.

Finally, do not be tempted to save space and effort by including a diagram of only one side of the aircraft. Too often, the right wing or stabilizer will be shown even though both right and left sides need the AMT's attention. A sure giveaway line for lazy procedure writing is "left side similar" written under the illustration of the right side. We can easily transpose left to right on a computer, but having the AMT transpose manually while performing the task is introducing another easily-avoidable and well-known human error source.

Writing task instructions

The language of instructions is as important as their accuracy and layout. Poorly worded task cards create confusion and many avoidable errors for the AMT. The language of task cards is English for all US-based airlines, even if the actual maintenance work is performed overseas or by non-native English speakers inside the USA. Any improvement in the intelligibility of the written English will be an improvement for native English speakers just as much as for their non-native counterparts. Simplified English was developed for just this purpose.

Writing task instructions

Simplified Technical English. Simplified English (now called Simplified Technical English, or STE) was developed by The European Association of Aerospace Industries (AECMA) as a standard way to write instructions. The idea behind Simplified English was a specialized language for aviation maintenance, not just a simplified version of English. The goal is to make the reading task easier for AMTs under often-difficult environmental conditions, and also to be much easier to translate into other languages.

Simplified Technical English consists of a set of writing rules, for example starting each procedure step with an action verb (check, tighten, install…), and a limited set of words for many of the verbs, nouns and adjectives used in maintenance instructions. Words in this set are the only ones allowed except for technical nouns such as part names (e.g., bracket) or equipment (e.g., eddy-current device)". There are no synonyms, so that "start" is always used while "begin, commence, etc" are never used.

Table 6-2 A: Example of Non-Simplified Technical English
B. Remove Filter Element
(1) Provide electrical power (Ref 24-22-00)
(2) Remove pneumatic power (Ref 36-00-00)
(3) Open POT WATER CPRSR (6K21) circuit breaker on main distribution panel, P6.
Attach DO-NOT-CLOSE identifier
(4) Open fill/overflow valve to depressurize water system, then close valve
Table 6-2 B: Simplified Technical English
B. Remove the Air-Filter Element
(1) Supply electrical power (Ref 24-22-00)
(2) Remove pneumatic power (Ref 36-00-00)
(3) Open this circuit breaker on main distribution panel, P6.
(a) POT WATER CPRSR (6K21)
(4) Attach DO-NOT-CLOSE identifier
(5) Open fill/overflow valve to depressurize water system,
(6) Close then close the fill/overflow valve after the pressure is released

 

Tests of STE using AMTs, inspectors and even engineers have shown it to lead to drastically reduced errors related to task card comprehension. Other tests have shown that it is easy for engineers to learn and use STE. The only small downside is a slight increase in the length of task cards when procedures are spelled out fully, e.g., instructions for the left and right side are included in their entirety rather than using "left side similar" or "repeat for left side".

Writing in Simplified Technical English should result in more readable text and lower text complexity scores. The usual recommendation is to use words with an equivalent reading level of 6th grade. This ensures that reading and comprehension are not compromised by poor environmental conditions, such as noise, interruptions, or restricted spaces within aircraft structures. AMT reading levels in the USA average between undergraduate and graduate student levels, but having a much lower reading level on task cards helps ensure low error rates under real-world conditions.

Fonts. An often neglected, but important part of procedure writing is the text formatting. Use upper case and lower case throughout rather than all upper case to imply emphasis. Readers benefit from rapid perception of the shape of each word, which is much more salient in lower case or mixed case than in upper case. The poor lighting conditions encountered during maintenance tasks (night work, work in restricted spaces) make the ability to recognize word shapes a potent error reduction tool.

Where emphasis is needed, bold, italics, a larger font, or even a small number of words in UPPER CASE can help. Indentation also helps AMTs keep their place in the document. The font itself should be easily readable, either a serif font such as Times Roman or a sans-serif font such as Arial (used in this chapter). Make the text and diagram fonts large enough to be easily read, typically 12 point but certainly no smaller than 10 point. A "point" is 1/72 of an inch.

Handling feed-forward information

AMTs and inspectors share their expert knowledge widely, even with those in other companies where the matter concerns safety. This knowledge, known as feed-forward information, helps all to do a better job, e.g., by alerting inspectors to new places where cracks have been found or warning AMTs of hidden hazards within a task. This type of information should be formalized into the task cards and other documents so that it is captured and its lessons can be learned. One airline has a computer-based task card system for its NDI shops where best practices are shared by inspectors, allowing those new to each specific inspected part to benefit from accumulated information.

Alert!

One concern regarding feed-forward information is that alerting an inspector to specific types of defects and specific locations may cause them to go straight to those points and inadvertently skimp on the rest of the inspection task. Controlled studies have shown just the opposite to be true. One study, at Sandia National Laboratories, tested this concern with airline inspectors on a B-737 where there were known defects in known locations. The overall result was that inspectors found more defects that were alerted, but also did better rather than worse on non-alerted defects.

Determining how many sign-offs are best

AMTs and inspectors must certify their work for legal and production control purposes, typically by signing off each part section of a procedure. Most procedures involve several steps, so an issue is how to assign sign-offs to steps? At one extreme, we have encountered a one-line procedure for replacing a wiring harness on a jet fighter that merely stated "Accomplish Task 33-649 per manual". This procedure took several days, often handing off the work across shifts. How the steps were supposed to be accomplished without error is beyond the understanding of human factors engineers!

Sign-offs

At the other extreme are tasks where past errors have resulted in the insertion of more and more sign-offs to "make sure they don't get omitted again". The danger here is that the AMT does not interrupt their work frequently to perform the sign-offs but waits until a convenient point in the procedure before signing off many steps already accomplished. Neither extreme appears sensible as both task AMTs' memory for where they are in the sequence.

A direct test of number of sign-offs on a simulated overnight check of over 100 steps showed that signing off logical groups of steps was just as safe in terms of correctly executed procedures and was both slightly faster and preferred by participants. The concept is to use AMTs' knowledge to find sensible related groups of task steps that are amenable to a single sign-off. This also related back to the section 3.2Ordering the Activities section. on how steps are ordered: if the steps are not sequenced in the order the AMT finds logical, then sensibly related groups will be impossible to find. The illustration of Functional vs. Spatial layout in that section illustrates the use of sensible grouping of sign-offs.

Validating the document

Until a task card or manual has been tested under realistic conditions it is only a theory not a valid procedure. The draft procedure (i.e., before validation) needs to be tested by real AMTs on the real aircraft, or for new aircraft on the virtual mock-up. Often the validation is skipped entirely as engineers are convinced they "know the 'plane", when in fact they know the OEM documentation much better than the plane itself. Or there is too little time to validate, or (your favorite excuse here). Any excuse for not validating is equivalent to selling airplanes to customers without a test flight. Everything should be OK, but there are so many instances where errors have occurred that we know we are setting ourselves up for error if validation is omitted.

Overview

Table 6-3, below, presents a condensed set of the guidelines described above. These are presented in the same order as in this section to demonstrate internal consistency. Many of the guidelines can be incorporated into standard templates as noted above. These guidelines plus Simplified Technical English (STE) have been codified into a Documentation Design Aid (DDA), available at http://hfskyway.faa.gov. This computer program operates in a window, illustrating each good practice by an example, and detailing why the practice should be followed. The DDA also includes an STE checker to determine whether a word is allowed in STE, or has a synonym in STE.

Validation

The DDA was tested using six technical writers, who were able to make significant improvements to test documents within one hour of first trying the program. In a direct test of the effectiveness of documents designed using the DDA, researchers found substantial and statistically significant improvements in comprehension errors when the DDA was used. The error rates on a comprehension test for the two carriers' versions were 51% and 35%, but only 4% for the DDA version.

Table 6-3. Guidelines for Procedeures and Technical Documentation
Collect Information
Get all documentation users involved in the design procedure.
Use the existing documentation and AMTs' knowledge as the basis for a task analysis
Find an agreed terminology
Ensure effectivity is specific to each tail number
Use participative design
Order Activities
Use a fixed logical sequence for multiple activities that require this sequence
Use a spatial rather than a functional sequence for multiple activities that have many logical sequences
Lay out the document
Use a flowchart at the start of the procedure
Use IF... THEN... ELSE Where there are choice points or branches in the sequence
Do not try to minimize page count in the document
Use short rather than long steps for instructions
Put Warnings and Cautions in text boxes
Do not put instructions in Warning or Caution boxes
Paginate the document
Use document users to help find best pagination
Check that each diagram is on the same page as its accompanying text
Do not try to minimize page count in the document
Use Illustrations
Use illustrations freely to minimize text/spatial errors
Use an overall aircraft diagram to locate the specific area of the task
Show the main illustration from the same viewpoint as the AMT performing the task
Indicate forward, up and right on each diagram
Change all text on the diagram, such as part names, to match that in the main text
Use separate diagrams for each side of the aircraft, not just "Left Side Similar" with a Right Side diagram
Write Instructions
Use Simplified Technical English for all instructions
Check the readability score of each document to ensure that it is Grade 6
Format text in easily readable font such as Calibri, Ariel or Times Roman
Keep text in 12 point. 10 point is an absolute minimum.
Use Upper Case and Lower Case letters throughout
Add emphasis with bold, italic, larger font
Provide feedforward information
Give specific up-to-date information to keep AMTs safe and help inspectors locate defects
Alerting inspectors to specific defects / locations will not harm performance on non-alerted items
Sign-off Logical groups of task steps
Provide sign-off boxes for logical groups of task steps
Use AMTs to help determine these logical groups
Too few sign-offs risk missed steps: too many risk working from memory rather than the task card
Validate the Document
Validate all documents on actual task with actual AMTs, even if AMTs have participated in the design
Validate by having the AMT perform the task literally from the task card, noting any implicit steps or missing knowledge