Safety Training: Tell Me a Story

| |
National Institute for Occupational Safety and Health (NIOSH)

Summary Statement

This article discusses the social and cultural power of stories, and how they have been used to develop training for skilled blue-collar workers. It includes examples as well as suggestions for trainers on where to find stories, what types of stories to pay attention to, and how to use them to improve occupational safety training in the mining industry. The same principles apply in construction. The research findings were also published in a peer-reviewed journal (Reference: Cullen ET, Prof Saf 2008 Jul; 53(7):20-27).
2008

Picture of a male miner: Using stories to improve occupational safety training

HUMANS ARE NATURAL STORYTELLERS and story listeners. People learn very early that stories not only are entertaining, but also provide the rules avout what is expected by society and how to make meaning of experiences. Stories are integral to human existence and have great power to change or influence how people think or react. They are an integral part of how learning takes place, and have obvious benefits to trainers and educators who want to impact what and how students learn.

This article discusses the social and cultural power of stories, and how they have been used to develop training for skilled blue-collar workers. It includes examples as well as suggestions for trainers on where to find stories, what types of stories to pay attention to, and how to use them to improve occupational safety training.

Background

The research behind this work takes place within the framework of the U.S. mining industry. In the late 1990s, the Spokane Research Lab (SRL), one of two NIOSH mining offices, held a series of stakeholder meetings in the West to gain input on issues and concerns that would benefit from NIOSH safety and health research. SRL held these meetings in different mining areas to meet with representatives of the many unique segments of the industry. Engineers, geologists, supervisors, union representatives and safety specialists were invited to share their ideas and help SRL create a research program that was responsive to industry needs. The issue of training was raised at each meeting. Training materials were generally perceived to be outdated and boring, or absent altogether.

Federal law—specifically 30 CFR parts 46 and 48—requires safety and health training for all miners (MSHA, 2002). This includes new miner training, specified as at least 40 hours of training for new underground miners and 24 hours for new surface miners, and annual refresher training for every miner, which is mandated to be at least 8 hours each year. Mining is one of the few industries that has such a stringent federally required training component.

However, instructors were not sure the required training was making a difference. They were, in their opinion, meeting the letter of the law, but not necessarily meeting the intent. In fact, according to several instructors, miners dreaded refresher training, referring to it as “safety jail,” and considered it little more than an opportunity for a free donut and a nap. According to trainers, miners begrudgingly spent the required 8 hours of “seat time,” but getting them to pay attention was another matter. Trainers believed that there was a need for more and better training materials that would get and keep the attention of workers, and that would be remembered after the sessions were over. They asked NIOSH to help by creating effective training materials for the industry.

An ambitious project emerged as a result of these meetings. Its objective was to develop and evaluate effective safety training for miners. One major obstacle was that no one could say what “effective” looked like or how to measure it, although everyone knew it was the goal. Safety professionals, however, offered ideas, technical advice and access to work sites, and readily offered critical opinions if they thought the project was off-track. The industry’s mandate for training was an asset in that trainers were required to train every miner each year, and would be grateful, if not eager, for any new materials.

female miner talking

At the start of the project, an informal group of safety trainers and directors was organized to help create a prioritized list of topics that needed new or improved training materials, and to act as technical advisors. This group was crucial to the project’s success. Members not only made suggestions, they also acted as subject-matter experts and became, in a sense, cocreators of the materials that were developed. Membership in the group was fluid as the topics under development changed, but the safety experts consistently ensured that created materials met their needs and those of workers.

Before developing training materials, it was necessary to spend time studying theories of adult learning (Knowles, Holton & Swanson, 1998; Lave & Wenger, 1991; Kowalski & Vaught, 2002; Wlodowski, 1985), training models (Owenby, 1992; Pegg, 1999; Billett, 1994) and regulations that controlled, to some degree, what was included in safety training. Currently available materials were examined as well. The team had to find a different way of presenting information to fulfill the initial request to make training more interesting and effective. Two dominant characteristics of the mining industry became the underlying framework for the new videos: mining’s strong occupational culture and the miners’ love for stories.

The Role of Culture

Every organization has its own culture. Culture is not something an organization has, it is something the organization is (Lewis & Thornhill, 1994). It has been described as a social roadmap for its members, providing essential information on how to survive and be successful within its boundaries. Patton (2002) defines culture as “that collection of behavior patterns and beliefs that constitutes:

  • standards for deciding what is;
  • standards for deciding how one feels about it;
  • standards for deciding what to do about it;
  • standards for deciding how to go about doing it” (p. 81).

Culture is inherently social. It develops over time through the shared experiences of its members who then teach new members its rules, perpetuating the culture. It can be a difficult thing to define, but it is readily apparent to those inside and, often, to those outside. Simply put, it is “the way we do things around here.”

Occupations also have cultures, particularly those with a shared sense of danger or whose members believe they are different from other work cultures. Occupational cultures have been defined as “a group of people who consider themselves to be engaged in the same sort of work; whose identity is drawn from the work; who share with one another a set of values, norms and perspectives” (Van Maanen & Barley, 1984, p. 287). These cultures are also seen as relationships “that apply to but extend beyond work-related matters . . . whose social relationships meld work and leisure” (p. 287).

According to Van Maanen and Barley, occupational cultures that are particularly strong because of shared dangers faced by members will be resistant to any changes suggested or mandated by outsiders. Examples include the military, loggers, firefighters, police, commercial fishermen and miners. If these same changes are brought forward by insiders, however, they are much more likely to be viewed positively and eventually adopted. According to Van Maanen and Barley:

Danger . . . invites work involvement and a sense of fraternity. . . . Recognition that one’s work entails danger heightens the contrast between one’s own work and the work of others, and encourages comparison of self with those who share one’s work situation. Attitudes, behaviors and self-images for coping physically and psychologically with threat become part of an occupational role appreciated best, it is thought, only by one’s fellow workers (p. 301).

The lesson is that members of these work cultures view outsiders as incapable of understanding the dangers they face, and are likely to place more trust in those who are perceived to be part of the occupational culture. Credible insiders have much more power to change behaviors within a culture than do outsiders.

Occupational cultures can often be identified by their tribal language (Neuhauser, 1988). This language is understood by members and can be used as shorthand, while maintaining the added benefit of being somewhat incomprehensible to outsiders. It serves as a communication tool, but also helps identify who is inside the culture and who is out, who is good and who is bad. It may be jargon specific to the industry, geographic idioms or phrases that come from languages other than English. Whatever the origin, it is the common language that has the power to convey meaning.

Using language in training programs that is not common to the culture will not be effective. While it may be understood, it will clearly brand the communicator as an outsider, and will create one more barrier between the speaker and the members (Hansen, 1995). In studying existing mine safety training materials, it became obvious that many of the materials available were presented in highly technical or regulatory language that was not commonly spoken in the industry. This may have been the primary reason the materials were considered ineffective. The products were full of information, but as Bruner (1990) points out, “Information is indifferent with respect to meaning” (p. 4). If trainees cannot use the information provided to construct meaning for their own work experience, then it cannot be effective.

Organizations are often made up of many subcultures, all of which have their own way of doing things and their own internal language. In a large mining operation, for example, the mechanics may not believe they have much in common with the geologists. Environmental engineers probably have a whole different set of concerns as compared to truck drivers. Shovel operators may not believe that company managers understand what they do or what their problems are. A safety trainer needs to be aware of the differences between organizational and occupational culture in order to be truly effective and to realize that with respect to safety training one size does not fit all.

Hansen (1995) believes that new information such as training will always be filtered through the beliefs, experiences and norms provided by the learners’ occupational culture. Members of these work cultures share a belief that “members have the exclusive right to perform a given set of related tasks” (p. 60), and this can make them quite resistant to outside influences. Hansen further says that “information communicated in a manner greatly different from what is culturally common requires a longer learning period and often leads to a lack of comprehension or misunderstanding” (p. 61). Learners are much quicker to pay attention to and pick up the meaning of new information if the teacher looks, walks and talks like they do.

Two characteristics of strong occupational cultures such as mining, logging or fishing are workers’ shared belief in their own competence, and their reliance on the ability and willingness of others to perform tasks without putting coworkers in danger (McCarl, 1997). This means that experienced workers believe no one knows the job better than they do, and they can become resentful of someone else, particularly an outsider, telling them how to do it better (Voynick, 1978). There is also a strong sense of independence among these workers—what experts would call an internal locus of control (Wyman, 1979). Because they have a cultural resistance to outsiders trying to control them, they will generally do something because they choose to do it, not necessarily because someone else tells them to (Schein, 1996). Trying to permanently change behaviors in this type of culture without obtaining the buy-in of the workers is impossible.

People will change their behavior to comply with mandated rules when they must (when an external locus of control is present such as the supervisor or inspector) but when no one is monitoring their actions, they will generally revert to how they have always done things, and how the occupational (not organizational) culture expects things to be done. To openly and willingly go against a traditional norm, workers must be convinced that the new behavior is a better choice—and that it is their choice. If this is done successfully, the new behavior becomes part of the cultural norms and a safety trainer or director does not need to monitor it.

The Value of Storytelling

The question is, How do you convince people— especially people with experience in an industry and perhaps a long history of doing things unsafely—to do things differently? The first and most obvious step is to get their attention. The safety trainer needs to find the internal switch that responds to the question, Why should I care about this information? and answers, Because it makes sense for me to care. It may save my life someday. Stories have the ability to do this.

Stories have been used to entertain ever since humans gathered together in groups. They have other functions, however, such as to help us make sense of what is happening in our lives. “Storytelling is a natural way of recounting experience, a practical solution to a fundamental problem in life, creating reasonable order out of experience” (Moen, 2006). Simmons (2001) believes that stories are “the oldest tool of influence in human history” (p. xvii) and that they effectively “connect people to what’s important and to help them make sense of their world” (p. 29). Slater (2002) further explains the power stories have to influence behavior. “It is difficult to consider another communication genre that can communicate beliefs, model behavior, teach skills, provide behavioral cues, and simulate consequences of behaviors over time in as compelling a fashion” (p. 16).

picture of miners talking

For a trainer, one of the most valuable characteristics of stories is their ability to teach vicariously. One need not actually be part of the story to learn from it. Because stories engage both the thinking and feeling sides of the brain, learners can place themselves inside the story and think about what they might have done in the same circumstances, while at the same time feeling the anxiety caused by the problem. Storytellers can elicit the fear, confusion or heightened awareness common to disaster stories without ever placing the learners in danger. This situation greatly increases the likelihood that listeners will remember both the story and the lessons it taught. Livo and Rietz (1986) claim:

“Story” is a way of knowing and remembering information—a shape or pattern into which information can be arranged. It serves a very basic purpose; it restructures experiences for the purpose of “saving” them. And it is an ancient, perhaps natural order of the mind. . . . By imposing the structure of a story onto some circumstance or happening, greater coherence and sensibility are achieved within the event itself, and otherwise isolated and disconnected scraps are bound up into something whole and meaningful (p. 5).

The facility with which stories organize random or unfamiliar information makes them valuable tools to train new workers. People entering a new work environment can be overwhelmed by the unknown. Not only do new employees not know the tribal language, they may have no point of reference to connect unfamiliar tools or environmental features with something they do understand. It is difficult to make sense of so much new information, even more so to make use of it.

two miners talking

Behind every regulation and statistical chart is a good teaching
story. These stories can give a face to the impersonal and make
it interesting- the people in the stories are just like the listeners,
and the listeners understand them. They share a sense of
cameraderie and understand that what happened in the story
could happen again.

Stories, however, provide information about what to do, how to do it and why it is important to do it a certain way. They also clearly communicate the penalty for ignoring established norms in that they relate what happened when someone failed to follow the guidelines. People pay attention and listen more attentively to stories than they do to formal types of instruction, making it easier to remember what they heard and to put it into practice. The story has particular power to change work behavior if the storyteller is a recognized member of the culture with the credibility to comment on the culture and tell stories about it. It is this authority that Durrance (1997), an expert on organizational training, describes when she states, “You don’t have to be an expert to use stories to motivate people, but you must be credible” (p. 28).

In the U.S., federal law mandates safety training for workers in many industries. Lawmakers believe it is critical for workers in inherently dangerous industries to receive what researchers call socially relevant information. Turning that socially relevant information into personally relevant information is the key to effective training.

Often, however, safety trainers use regulations, facts or statistics—all of which is clearly socially relevant information—to make their point about a work hazard. Yet, as Cole (1997) points out, “many learners who receive . . . this formally codified and socially relevant knowledge tend to find both the content and the instruction to be burdensome, dull and boring” (p. 334). Statistics and regulations are impersonal and uninteresting. Behind every regulation and statistical chart is a good teaching story. These stories can give a face to the impersonal and make it interesting—the people in the stories are just like the listeners, and the listeners understand them. They share a sense of camaraderie and understand that what happened in the story could happen again. Listeners empathize, especially if they are caught up in the story, and they remember the lesson.

Good stories are easy to find. Almost anyone who has worked in a job that includes exposure to danger has either heard of or experienced a close call. In some cases, these experiences end in tragedy. Workers love to share these stories because they reveal the dangers and complexity of the work, as well as instruct the listener on what to do in a similar situation.

Types of Stories

During the course of this project, researchers found that most of the teaching stories they gathered could be divided into four broad groups, categorized by the type of story or the role of the main characters. The categories are the author’s attempt to find meaning in the many stories that were shared with her, by defining common themes. All of these stories are useful for creating effective training, as long as one listens carefully for the underlying messages.

Hero stories.

These stories take the form of talking about people who are or were larger than life. They may be about skilled workers who taught the storyteller how to perform the job safely and productively, or about someone who saved a colleague in a crisis. They almost always refer to traits that are admired by the culture (e.g., hard work, dependability, toughness, courage, creativity under duress). The hero embodies these qualities, and the story gives guidance to listeners as to what the culture values, as well as what it expects. The unspoken (but strongly communicated) message to the listener is that the hero is the model for them and that this is the behavior and the moral fiber that the culture demands.

Villain stories.

These stories are also about the values and norms of the culture, but view them from the shadow side, as Campbell (1973) describes it. Rather than embodying the valued traits of the culture, the villain has turned away from them and is, therefore, one to be scorned, feared or punished. The norms, however, remain the same. The villain is not courageous, hardworking, kind or dependable. These stories reveal as much about the culture’s values and expectations as do the hero stories. The message to the listener is that this type of behavior is unacceptable and should be avoided.

Adventure or disaster stories.

interviewing scene

To tell a story effectively, the interviewees must
be able to ignore the camera and talk to the
interviewer as if they were alone.

These stories are about events and may have many characters involved, who all react to the event. The stories told about the events surrounding the attacks on the World Trade Center on Sept. 11, 2001, are examples. Stories of this nature can be heartbreaking or entertaining— adventures are thrilling by nature—but they can also serve to highlight dangers that are intrinsic to the work environment. Workers who regularly face unforeseen danger, such as underground miners, must make sense of this danger. This can be enhanced by sharing stories of what can go wrong and how people react when it does.

The research team gained valuable insight on the significance of this aspect of storytelling when working on a video about a major mine disaster, the Sunshine Mine Fire. The stories shared by survivors highlighted their resilience in the face of losing 91 of their friends and coworkers, as well as how they constructed meaning from this horrible disaster. The ability to make sense of what has happened and to find meaning behind it may be one of the strongest coping mechanisms available to workers who must face dangers each day. Adventure/disaster stories are good teaching stories because they inform listeners about what could go wrong, and they give them guidance by describing what others have done when this happens.

Fool stories.

These may be the most valuable type of training stories. Fool stories are about what happens when someone ignores warnings, forgets training, becomes careless or complacent, or perhaps acts unsafely. When a listener first hears these stories, the storyteller typically begins, “I knew a guy once who. . . .” As the storyteller comes to trust the listener, however, the stories will begin, “Let me tell you what happened to me one day.” These are warning or near-hit stories more than anything—if the protagonist suffers a serious injury or is killed, they quickly become disaster stories.

In these stories, the protagonist survives the event and learns a valuable lesson. The storytellers often use humor or show a lot of emotion, and admit that they were just plain lucky “that time.” They usually acknowledge that they learned a powerful lesson from the experience and that others could save themselves similar pain and embarrassment if they could just learn from the story. For a trainer, these stories are priceless.

A word of caution for trainers and safety professionals: The tellers of fool stories will not share their experiences if they believe they will be punished for doing so. These stories have great power but they must be shared in a safe environment.

The Research Project

Funded by NIOSH, the pilot project had an initial budget of $10,000 and a staff of only one full-time researcher. The objective of the project was to determine whether SRL could develop effective training materials that would be accepted by the mining industry. Colleagues at SRL’s sister lab in Pittsburgh, PA, had a proven track record of creating training materials for coal miners, so the Spokane project was limited to the underground hard rock segment of the industry. As the team began producing training materials that became popular with trainers, however, the project was expanded to a full research project that included surface mining and aggregate operations. The budget and staffing grew as well to eventually include a team of five.

Early discussions with trainers at different mine sites revealed that most mining operations did not have access to state-of-the-art multimedia training facilities, but most had the ability to show videos. The advisory group and research team agreed that video was the best medium to use—information and stories could be captured visually to produce VHS or DVD products, which could be distributed fairly inexpensively to mines across the country.

The technical advisory group provided a prioritized wish list of training topics; at the top of the list was handling explosives—not for blasters, who were well-trained in their art, but for others who work around or handle explosives as they are moved to different areas of the mine.

The team learned a great deal in the development of this video. While fairly knowledgeable about mining practices, SRL did not have in-house capability to do the video taping, so a professional crew was hired. This was a key decision, because the quality of the footage and sound track determined what kind of products would be produced. The footage captured during the project was professionally shot and of excellent quality. When it was time to put a video together, poor audio or blurry footage were not constraints.

Working with a professional crew also allowed the team to focus on the content of the training videos—the miners and the stories they told—rather than on technical concerns. A cautionary note: A person cannot establish the necessary trust and connection with an interview subject if s/he is also the one behind the camera. To tell a story effectively, the interviewees must be able to ignore the camera and talk to the interviewer as if they were alone.

Miners in underground hard rock mines have a unique subculture within the mining industry (Miller, 1991). Many of the mines use what is known as a gypo or contract system—miners are paid an hourly wage, but also have the opportunity to earn substantial production bonuses. (Their often-stated goal is to triple their day’s pay.) No bonus is paid if the miner does not make the round for the day, so gypo miners are accustomed to working hard and fast. They do not tolerate interruptions, especially those that might cost them money. When the project first started, many in the industry doubted the top miners would be willing to cooperate because of the tedious and time-consuming process of making a video. Their paychecks would most certainly be affected if they became involved. There was also a very real concern that miners would view the team as outsiders and refuse to collaborate because of a lack of trust.

Both of these fears proved groundless. Mining is a culture that attaches high value to expertise and skill; every experienced miner learned how to do the job from a master miner, who in turn was taught by another master. The status that accompanies recognition as a master miner is prized. The team discovered that the miners were very proud of their work and that being selected as an expert—and therefore recognized as a master/mentor who would also be a movie star—guaranteed bragging rights. Generally, it was not possible to identify ahead of time who these people might be, because the film crew did not know who was considered to be the best at a given task.

To have access to the worksites, equipment and workers needed, the team worked with individual gatekeepers at the mine sites—usually the safety directors or trainers—who suggested miners who would be good for certain roles or who might be willing to participate. Miners were given the right to decide whether they wanted to participate and few refused to do so. In fact, after the first few videos were released and circulated through the industry, the team had more volunteers than needed. This was an unexpected but pleasant problem to have.

When working with nonprofessional actors such as miners or fishermen, it is important to be patient and to allow them to grow comfortable with the filming process. They are not generally accustomed to performing in front of an audience or on camera, and are afraid of making mistakes or looking stupid. In the first couple of videos created, an attempt was made to write scripts for the miners/actors, but this quickly proved to be wasted time. Not only were the scripts written in language they would not use, but the miners had difficulty memorizing scripts, which made them even more uncomfortable with the process. It was better to simply tell them the role they were playing and ask them to do what they normally would do, explaining as they went, as if they had a new hire to train. They managed this with enthusiasm. If they missed an important point, the crew simply shot more video until the point was captured.

Generally, a list of topics was developed before the shoot started, but often the miners themselves made suggestions about things they thought should be included, or made corrections if they were asked to do something they thought was inaccurate. They were well aware of the shortcomings of current training, and were ingenious in their suggestions on how to improve it. They also had a good perception of what new hires would need to know to stay safe and be productive.

The project generated nine videos in its 7-year history:

  • Handling Explosives in Modern Underground Mines (1998);
  • Preventing Rock Fall Injuries in Underground Mines (1999);
  • Miner Mike Saves the Day (or Ground Support . . . It’s Important) (2000);
  • Hazards in Motion (2001);
  • Hidden Scars (2001);
  • You Are My Sunshine (2002);
  • Zen and the Art of Rock Bolting (2002);
  • Aggregate Training for the Safety Impaired (2003);
  • The Sky Is Falling! (2004).

An additional video was completed in 2007 for the commercial salmon fishing industry. The Most Powerful Thing...Deck Safety Awareness for Purse Seiners includes stories from fishermen injured during fishing operations.

All of the videos were screened before release with groups of topical experts who acted as technical reviewers and ensured that the material was both accurate and met the industry needs. The finished products were made available free of charge to safety practitioners. A database was created to track where videos were sent and to obtain feedback.

To date, more than 13,000 copies have been requested and sent to mines in every U.S. state and more than 50 other countries. Videos have also been used by other industries such as firefighting, the military, foundry workers and university public health programs. Lessons learned during the project have been documented in several publications (Cullen & Fein, 2005; Camm & Cullen, 2002; Cullen, 2006) and are being used in other projects designed to create training and safety awareness materials.

Evaluation of Effectiveness

Evaluation of training programs is a recognized challenge, regardless of the industry under discussion. Lewis and Thornhill (1994) admit, “There seems to be widespread agreement with the proposition that evaluation is the least well conducted aspect of all training activities [in organizations]” (p. 25). Their study found that many organizations do not evaluate training at all and rely on the “act of faith effect,” which concludes that “training must be a good thing” and is, therefore, worth doing (p. 27).

Kirkpatrick (2001) developed a four-level evaluation schematic in which evaluation data progresses from the easiest and most informal to very resourceintensive:

  1. reactions to or perceptions of training;
  2. improvement in knowledge or skills as a result of training;
  3. changes in behavior or adoption of new skills and knowledge;
  4. overall impact on the organization (p. 122-132).

It was believed that miners’ reactions to the video would be fairly easy to obtain. Increases in knowledge of those who viewed the videos were possible to measure, but accurately assessing either changes in miners’ behavior or permanent impact on organizations or the industry as a whole would be difficult at best. Nonetheless, several evaluation studies of the videos were completed.

The first evaluation study, conducted by Fein and Isaacson, looked at the initial five videos produced. The team used a mixed-methods approach, with true/false tests, multiple choice, Likert scales to assess trainee perceptions, and more qualitative, open-ended questions. The team also interviewed a random sample of trainers who had requested the videos to learn how the videos were used and how they were perceived by trainees. Subjects were given the opportunity to provide feedback on the products, which were then analyzed for common themes. The results of the study are available in the team’s final report (Fein & Isaacson, 2001) and are also summarized in the final report on the project (Cullen & Fein, 2005).

There were several confounding factors to doing a traditional statistical study. U.S. law requires that every miner be trained before entering a mine, so designating a control group of untrained workers for the sole purpose of testing the effectiveness of the training materials was impractical if not unethical. The research team was not certified under the law to provide training itself, but relied on trainers to voluntarily use the videos and report how they were accepted. Additionally, the miners themselves were not obligated to take the pre/posttests, but were asked to do so voluntarily. While most miners asked to assist in the evaluation did so, no data were available on how many opted out.

A further complication for the evaluation team stemmed from the demographics of the workforce itself. At the time the study began, the U.S. mining population was aging, with an average age of around 50 with more than 30 years of mining experience. Clearly, these workers would not be trained by the videos so much as reminded by them. Assessing any increase in knowledge for this group would be difficult.

The SRL research team hoped that all miners— both new and experienced—would find the videos interesting enough that they would pay attention to them and to the lessons they contained. This turned out to be the case. Scores on the posttests were higher for both groups. For the four videos for which data were gathered, miners who identified themselves as having less than 1 year experience showed improvements between 9.4% and 14.6%, while experienced miners showed improvements between 0.4% and 11.6% (Cullen & Fein, 2005).

The results of the pre/posttest questionnaires given to trainees indicated that the videos were, in fact, effective training tools. The miners enjoyed watching them—they freely commented on the quality and on how much they liked the stories.

Fein and Isaacson (2001) discovered, however, that miners are not disposed to taking tests of this type. The videos were shown to classes of new trainees and to experienced miners during annual refresher classes. While the videos themselves were well accepted (judging from comments made by both trainers and miners), the tests were not. Veteran miners believed that the true/false and multiple-choice tests were disrespectful of their experience, and that anyone could guess the right answers to the questions. New miners often did not have enough knowledge of the mining language to understand the questions; literacy was also a barrier to some.

Several videos are more accurately described as safety awareness products rather than traditional training on how to do something. These were very difficult to assess for increases in knowledge. It was obvious to the evaluation team that this type of testing was not providing the best indication of effectiveness.

Accordingly, Fein and Isaacson (2001) developed a different type of assessment tool to evaluate the remaining videos in the study. This tool also uses a mixed-methods approach, but does not use true/false or multiple-choice questions. Instead, it is made up of a limited number of open-ended questions that relate directly to the topic of the video under evaluation. An example is, What should you do when first entering your work area at the start of the shift? The same questionnaire is given before and after watching the video, and the answers are compared for each trainee. If the video is effective as a safety awareness product, the evaluation team believed that there should be both more answers and more specific answers than on the pretest version.

This type of tool gives experienced miners credit for prior knowledge, and the video itself becomes a reminder rather than a primary training tool for them. For new miners, however, the video shows them how to do the things they will be asked to do underground, which are also explained by a credible master-mentor. Their responses after watching showed significant improvement in awareness and knowledge of what to do and why.

The posttest survey also included several questions for trainees that asked them to rate themselves on Likert-type scales regarding how much they learned, how much they liked or disliked the video, etc. A separate questionnaire was developed that asked trainers to observe the trainees and report reactions to the video, as well as to rate it for training value, including the effectiveness of the stories. The details and results of the first use of this instrument show that the new assessment tool is a better indicator of knowledge gained (or perhaps information remembered) than were the traditional true/false tests (Fein, 2003).

Similar evaluations using a mixed-methods approach were completed for the remainder of the videos. Each video was created to answer a specific question or to meet a specific need. In all cases, because of the qualitative nature of the materials developed, the assessments themselves were also qualitative rather than quantitative. The results of the evaluations show that they were effective in meeting that need. (These studies have been completed, and the results will be published soon by NIOSH.)

Conclusion

The goal of this small pilot-project, funded by NIOSH in 1998 and subsequently for another 6 years, was to create effective training for the mining industry. The nine videos that it produced were developed in operating mines, using real miners as role models and teachers. They used the stories told within the industry to convey specific safety awareness messages and did so in a way that was interesting and accepted by the occupational culture of the trainees.

Evaluation studies have shown that the videos are effective, but perhaps their wide distribution in the industry could also be viewed as another measure of acceptance. The videos are provided free of charge to trainers, but SRL/NIOSH has been asked to replace them constantly—apparently many miners borrow them to show their families. If two necessary keys to effective training are getting the attention of trainees and gaining cultural acceptance of the messages taught, the NIOSH videos appear to have succeeded. There is no reason to believe these lessons would not be useful for other industries.

References

Billett, S. (1994). Situated learning in the workplace: Having another look at apprenticeships. Industrial and Commercial Training, 26(11), 9-16.

Bruner, J. (1990). Acts of meaning. Cambridge, MA: Harvard University Press.

Camm, T.W. & Cullen, E.T. (2002). Releasing the energy of workers to create a safer workplace: The value of using mentors to enhance safety training. In R.H. Peters (Ed.), Strategies for improving miners’ training (NIOSH Publication No. 2002-156, Information Circular No. 9463, pp. 35-38). Pittsburgh, PA: U.S. Department of Health and Human Services (DHHS), Centers for Disease Control and Prevention (CDC), NIOSH, Pittsburgh Research Laboratory.

Campbell, J. (1973). The hero with a thousand faces. Princeton, NJ: Princeton University Press.

Cole, H.P. (1997). Stories to live by: A narrative approach to health-behavior research and injury prevention. In D.S. Gochman (Ed.), Handbook of health behavior research methods: Volume 4 (pp. 325-349). New York: Plenum.

Cullen, E.T. (2006, March). The power of storytelling in getting through to miners. In E.T. Cullen, T. Camm, M. Jenkins, et al. (Eds.), Getting to zero: The human side of mining (NIOSH Publication No. 2006-112, Information Circular 9484, pp. 14-23). Spokane, WA: DHHS, CDC, NIOSH, Spokane Research Laboratory.

Cullen, E.T. & Fein, A.H. (2005, Aug.). Tell me a story: Why stories are essential to effective safety training (NIOSH Publication No. 2005-152, Report of Investigations 9664). Cincinnati, OH: NIOSH.

Durrance, B. (1997). Stories at work. Training + Development, 51(2), 25-29.

Fein, A.H. (2003). Evaluation of zen and the art of rockbolting. Report prepared for DHHS, CDC, NIOSH, Spokane Research Laboratory.

Fein, A.H. & Isaacson, N.S. (2001). Video-based training program for underground miners evaluation report. Report prepared for DHHS, CDC, NIOSH, Spokane Research Laboratory.

Hansen, C.D. (1995). Occupational cultures: Whose frame are we using? Journal for Quality and Participation, 18(3), 60-64.

Kirkpatrick, D. (2001). The four-level evaluation process. In L.L. Ukens (Ed.), What smart trainers know: The secrets of success from the world’s foremost experts (pp. 122-132). San Francisco, CA: Jossey-Bass/Pfeiffer.

Knowles, M.S., Holton, E.F. & Swanson, R.A. (1998). The adult learner (5th ed.). Woburn, MA: Butterworth-Heinemann.

Kowalski, K.M. & Vaught, C. (2002, Sept.). Principles of adult learning: Application for mine trainers. In R.H. Peters (Ed.), Strategies for improving miners’ training (NIOSH Publication No. 2002-156, Information Circular No. 9463, pp. 3-8). Pittsburgh, PA: DHHS, CDC, NIOSH, Pittsburgh Research Laboratory.

Lave, J. & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, U.K.: Cambridge University Press.

Lewis, P. & Thornhill, A. (1994). The evaluation of training: An organizational approach. Journal of European Industrial Training, 18(8), 25-33.

Livo, N.J. & Rietz, S.A. (1986). Storytelling: Process and practice. Littleton, CO: Libraries Unlimited Inc.

McCarl, R. (1997). Contested space: The above and below ground landscape of Idaho’s Coeur d’Alene mining district. Salt Lake City, UT: University of Utah Graduate School of Architecture.

Miller, H.B. (1991). A comprehensive analysis of contract mining and wage incentives in underground metal mining. Unpublished masters’ thesis, Colorado School of Mines, Golden, CO.

MSHA. (2002). Federal metal and nonmetallic mine training, safety and health standards. Price, UT: Mine Safety Associates.

Moen, T. (2006). Reflections on the narrative research approach. International Journal of Qualitative Methods, 5(4), Article 5. Retrieved May 28, 2008, from http://www.ualberta.ca/~iiqm/backissues/ 5_4/html/moen.htm.

Neuhauser, P.C. (1988). Tribal warfare in organizations. New York: HarperBusiness.

Owenby, P.H. (1992). Making case studies come alive. Training, 29(1), 43-46.

Patton, M.Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage Publications.

Pegg, M. (1999). The art of mentoring. Industrial and Commercial Training, 31(4), 136-142.

Schein, E.H. (1996). Culture: The missing concept in organizational studies. Administrative Science Quarterly, 41(2), 229-240.

Simmons, A. (2001). The story factor: Inspiration, influence and persuasion through the art of storytelling. Cambridge, MA: Perseus Books Group.

Slater, M.D. (2002). Entertainment education and the persuasive impact of narratives. In M.C. Green, J.J. Strange & T.C. Brock (Eds.), Narrative impact: Social and cognitive foundations (p. 157). Mahwah, NJ: Lawrence Erlbaum Associates.

Van Maanen, J. & Barley, S.R. (1984). Occupational communities: Culture and control in organizations. In B.M. Staw and L.L. Cummings (Eds.), Research in Organizational Behavior: Volume 6 (pp. 287-366). Greenwich, CT: JAI Press.

Voynick, S.M. (1978). The making of a hard rock miner. Kearney, NE: Morris Publishing.

Wlodowski, R.J. (1985). Enhancing adult motivation to learn. San Francisco, CA: Jossey-Bass.

Wyman, M. (1979). Hard rock epic: Western miners and the industrial revolution, 1860-1910. Berkeley, CA: University of California Press.

All of the videos/DVDs developed by this program are available from NIOSH’s Spokane Research Lab. For information, visit www.cdc.gov/ niosh/mining/products.