Field Guides: The Good the Bad and the Ugly

For the past four years I’ve been reviewing field guides used by grantees delivering weatherization services throughout the country. With 58 grantees and up to three housing types per grantee, that’s a lot of field guides, and I’ve seen the full range of the good, the bad, and the ugly (cue Western whistle and tumbleweed).

The Ugly: Try to be everything to everybody and end up not being very useful to anybody. These are the 500-page guides that bury references to specific material requirements in paragraphs of policy or billing rules.

The Bad: Focus specifically on installations, but dump technical standards into a guide format with no interpretation. Picture lines and lines of text where it’s hard for installers to find what they need. These guides are often redundant, yet leave out helpful how-to information.

The Good: Include only the need-to-know information for specific installations, with lots of graphics and few paragraphs. This makes it easy for installers to find what they need to do the job right the first time.

One of several checklists included in the Arizona field guide.

What Makes a Good Field Guide?

First, identify your audience as narrowly as possible. The narrower you can be, the more useful your guide will be to that audience. The very name field guide implies that it is used in the field—which means it probably lives in a truck. That means it is referenced between trips to hot attics or spider-infested crawl spaces. When an installer finds an unexpected dropped soffit at the far end of the attic and needs to air seal it, he or she doesn’t care about how the measure is charged, or about the building science behind moisture dynamics.

The field guide will also be the tool most used by the inspectors who determine the pass/fail of the work. The installers and the inspectors should both be working from the same playbook, after all.

Once you identify the audience, speak directly to them. Put yourself in their shoes, or coveralls. If you are writing or revising a field guide, look at every single line and ask yourself, “If I were doing their job, would I need this information? Am I using plain language that these readers would easily understand?”

Now provide the minimum information that these readers need to do the job correctly. This includes three things:

Objective of the measure. What is this work supposed to accomplish? A recognized principle in adult learning is that adults are motivated by information or tasks that they find meaningful. If they know why they are air sealing those ducts or insulating that attic, they are more likely to tackle the task with a versatile, productive approach aimed at achieving the stated objective and solving any problems that arise.

Materials and step-by-step instructions. How can installers best accomplish each task? List the steps; then for each one ask, “Is this step critical to the success of this measure? Is this step not captured anywhere else?” If the answer to both questions is not a resounding No—nix it. No one will use a checklist that is 200 items long.

Evaluation parameters. How will the accomplishment of this job be verified? Parameters should be objective and measurable. Most people want to do a good job, but they often aren’t given the resources to determine if they are succeeding. They guess and hope for the best, and when a job fails, it’s demoralizing.

Reimagine the Field Guide as a Set of Job Aids

Job aids are tools that provide workers with the information and guidance they need at the moment in order to be productive, accurate, and efficient on the job. A good field guide should be a set of job aids.

The best sets of job aids are

Step-by-step illustrated guides. These are most useful when steps must be completed in a certain order to achieve the desired outcome. They are especially helpful for the common, linear tasks we complete in home performance, such as air sealing an attic floor or sealing and insulating ductwork.

Checklists. These are most useful for tasks involving inspection or observation where attention to detail and comprehensive evaluation is required to determine the pass/fail of the installation. The inspection doesn’t need to be completed in a specific order, but it needs to cover all the elements of a successful, effective installation.

A Field Guide Is Not Training

While job aids are often used as part of training, training and job aids fulfill different needs. Training is when workers learn the information for the first time or are adding to their existing knowledge. Job aids are used when workers are trying to apply that knowledge to a given job, remember something they’ve learned, solve a problem, or adapt to a unique situation.

Job aids are used extensively in fields that range from commercial cleaning to health care to fast food. (Ever wonder why that Big Mac you got in Albuquerque looked and tasted the same as the one you got in Akron? Good job aids, that’s why.) Adults are independent, self-directed learners, which makes providing the information they need in an accessible reference like a job aid very effective. Job aid effectiveness has been proven by industries with far more resources than we have available in home performance. One small study looked at the introduction of illustrated job aids in a malaria-testing program in Zambia. Researchers found that practitioners who were given only the instruction packet successfully completed the critical steps 57% of the time. Workers given illustrated job aids successfully completed the critical steps 80% of the time, and workers who received the job aids and a three-hour training successfully completed the steps 92% of the time. Training and job aids together are a winning combination.

The benefits of good job aids include

• shorter training time with less retraining;
• better performance than can be achieved with training alone;
• faster achievement of worker competence;
• lower cost of achieving worker competence;
• less waste and fewer errors and omissions;
• increased productivity and efficiency; and
• improved employee satisfaction—everyone likes to get it right the first time.

Of the hundreds of field guides that I’ve reviewed, the best I have seen is the Success with Weatherization Arizona guide developed by the Foundation for Senior Living (FSL) Southwest Building Science Training Center and Advanced Energy in 2012. This set of checklists and step-by-step illustrated installation instructions has borne out the benefits of job aids described above. Since the guide was introduced in 2012 (in conjunction with other program enhancements), program quality assurance professionals have seen average measure installation scores steadily improve. (See Figure 1.)

Figure 1. Quality assurance professionals rank installations on a score of 1–3. Introduction of the new field guide was followed by steady improvement in scores.

Illustrated step-by-step instructions from the Arizona field guide.

How Do You Get There?

Establish clarity. This involves identifying your audience, but extends to defining the purpose and objectives of the guide. Picture how the guide will be used and how it should be structured. Determine the goals of the guide and the responsibilities of each team member. Determine how you will evaluate the effectiveness of the guide. The roles of the team members can be broken out into Management; Materials Development, including trainers, contractors, and other subject matter experts (SMEs); and Training/Deployment of the guide.

Put together the right team and give them resources. Adequate resources must include dedicated time to work on the field guide. Asking technical monitors or final inspectors to carve time out of their existing schedules will make for a lackluster guide. Give your team enough time to develop a useful, effective guide. For a list of free resources, see “learn more.”

Stay focused. Focus must be on the end user. Resist including anything that doesn’t support the goals defined under “Establish clarity” above. If it doesn’t directly support achieving those goals, no matter how interesting it is, it doesn’t belong here.

Commit. Stakeholders must support the project from beginning to end. Even with the best team and the best of intentions, any project can fail if the upper management or the state or utility office stops supporting the project at any stage.

Criteria	Points Scored
	1	2	3	4	5
How critical is this step? (Will the task fail if this step is not completed?)	Can easily pass if step not completed		May fail if step not completed		Will fail if step not completed
How frequently do we see this specific scenario in our housing stock? (This is helpful when cutting through “what if” or “one time I saw” discussions among the team.)	Seen in 1 out of 10 jobs or less		Seen in 1 out of 5 jobs		Seen in nearly every job
Is this step likely to be overlooked if it is not included in the field guide? (For example, the actual blowing of the insulation for an attic is unlikely to be overlooked, so focus on preparation and installation requirements.)	Definitely not	Unlikely	Not sure	Likely	Yes, definitely

 

Basic Tips for Good Job Aids Use short sentences with short words. Use simple and concise language. In step-by-step aids, begin sentences with verbs whenever possible. Use visuals consistently. If one series starts with a photo, illustrate all following steps in that series with photos. If you use illustrations, use them for every step. Leave out all the nice-to-know information. That just muddies the waters and makes the reference less accessible.

 

Identifying the Need-to-Know Information

One of the most common problems your development team will face is the challenge of cutting through all of the nice-to-know information and homing in on the need-to-know information. Here are some tips:

Use a ranking system. For any given step or scenario, assign points based on the three criteria listed in Table 1 to help separate the wheat from the chaff. The higher the score, the more likely that step or scenario belongs in the guide.

Still having trouble narrowing down the need-to-know information? Have your team of SMEs complete the task in question, but complete it without using their dominant hand, or remember what it was like the first time they did it. Research shows that when determining what information is useful to newcomers in a given field, experts offered more useful advice after they had been asked to recall what it was like to complete the task when they were new to the job.

Wrap-Up

Producing a field guide as a series of job aids makes the task easier than thinking of the guide as a unit that must be finished in one attempt. Start by identifying your audience and the objectives of the guide. Get all stakeholders to commit to seeing the project through. Put together a good team, assign responsibilities, and give them the resources they need (see “Choosing the Right Team—Notes from the Field”). Have the team list the specific tasks you want to cover and tackle them one by one. Develop step-by-step illustrated guides for each task, whittling information down to need-to-know only. Have the team determine the objective criteria that inspectors will use to determine the pass/fail of the work and make those criteria into a checklist.

For more tips on developing an effective field guide, see “Basic Tips for Good Job Aids.”

It takes a lot of time and effort to create a great field guide, but start now and you will see improved performance, reduced rework, and increased worker satisfaction, which will save time and effort down the road.

Kelly Cutchin has over 10 years’ experience in the buildings energy efficiency industry, with most of that being as a technical consultant to DOE’s Weatherization Assistance Program (WAP). Highlights include crawling through attics and crawlspaces in over two dozen states and territories, working with the WAP Trainers’ Consortium to develop the Standardized Training Curricula, and seeing first-hand how the work this industry performs improves the lives of those we serve.

 

Choosing the Right Team—Notes from the Field The Arizona field guide team was a collaboration between FSL Home Improvements and Advanced Energy. The FSL team consisted of three members, all of whom had extensive installation experience, and each of whom brought a different perspective to the table. Charlie Gohman is FSL’s quality assurance manager for Home Performance with Energy Star (HPwES). He worked for over 20 years at the Arizona Department of Energy during which time he managed the Arizona Weatherization Assistance Program (WAP). He provided the field inspector’s point of view. The field inspector may be more focused on the final product and standards than on the process necessary to achieve them. Chris Baker is the manager of FSL’s Southwest Building Science Training Center. He previously worked his way up from installer to manager of a WAP agency in northern Wyoming. He provided the trainer’s point of view. The trainer is likely to pay more attention to the process itself, and to best practices that can be used to improve that process. Ken Pancost is the owner of Arizona Energy Management, a contracting company that provides WAP/HPwES services. He is a sub-contractor for FSL weatherization work. He previously worked under Charlie Gohman at the Arizona Energy Office. Ken provided the contractor’s point of view. The contractor wants to see cost control across all the steps of a process.The Advanced Energy team consisted of two building science people including John Tooley. A scribe took all the notes, and a couple of creative people put the final product together. The team was led by John, a nationally respected building science consultant. We had all worked together at DOE on the creation of the standard work specification (SWS) in 2010. The SWS was the foundation for the Arizona field guide. This was collaborative. FSL and AE both decided which steps were needed, and all the text for each step. Our job was to bring those standards to life with pictures and very simple step-by-step explanations of how to do the work. This involved several all-day conference calls that were held to strict agendas. Here is a very brief step-by-step of what we did over the course of months to make our guide. Decide which measures would make the guide. Which measures were most common in our housing stock? There are very few crawl spaces in Arizona. So measures concerning crawl spaces did not make the guide. We do come across radiant barriers frequently. So a measure concerning radiant barriers was added. Pare down the obvious measures to what made sense for our housing stock. The process of attaching and sealing flex duct to a boot, start collar, or exhaust fan is the same in all three cases. One process covered several variations. Blowing fiberglass or cellulose is the same process from an installer’s perspective, so only one measure was added. Determine the critical steps within the process for each measure. Critical step was defined as “if this particular step is missed, this measure will not meet the SWS standard.” Determine the text that would describe each step within the process. Use short explanations with short words. Think in terms of tweets. Determine the desired outcome. The desired outcome is a simplified, shortened explanation of the objective as shown in the SWS. It enables the installer to understand what he or she is trying to accomplish. Determine what needs to be illustrated in every photo. Include tools when possible. Illustrate techniques. Illustrate personal protective equipment (PPE). Lead by example. FSL and AE worked together. There were several 8 hour long conference calls where we all discussed every aspect of the steps listed above. We all had equal input not just into which measures became critical details, but every word of every bit of text explaining those steps. Ken the contractor was not available to participate in conference calls and the planning parts, but was brought in when we felt a contractor’s opinion was needed. Chris and Charlie locked themselves in a small office planning, writing and staging. Charlie and I sometimes consulted with Ken and our internal installers when we had questions or wanted their opinions on the processes we were defining within the guide. This was a very collaborative effort. I believe that is why it turned out as well as it did. The critical details in the Arizona field guide continue to be used by contractors, trainers, and inspectors in the Arizona weatherization and HPwES programs to date. Chris Baker has over 20 years’ experience in the residential energy efficiency industry. Highlights include managing a WAP agency, and managing a building science training center. He is currently the program manager for the Home Performance with Energy Star, Limited Income Weatherization and Residential HVAC programs for Arizona Public Service in Phoenix, Arizona.

• 1
• FIRST PAGE
• PREVIOUS PAGE
• NEXT
• LAST