Virtual world for helping teens practice assertiveness skills
Kenneth Nemire, Joshua Beil, Ronald W. Swan
Interface Technologies Corporation, Capitola, California
Human Development Associates, Watsonville, California

ABSTRACT
Smoking is on the rise among adolescents. This pilot project combined the well-documented benefits of Life Skills Training (LST) with the unique multisensory, 3D qualities of virtual environment (VE) technology to address some of the disadvantages of traditional prevention programs by engaging teens better, presenting information more persuasively, and making prevention programs continuously available in computer labs. In an eight-week pilot study, 45 seventh-grade students were randomly assigned to LST, VE, or non-intervention control groups. The VE system included goggles, synthesized speech, head and hand trackers, hand-held controller, and speech recognition. Questionnaires measured participants’ smoking knowledge and behavior, participants' reports on the usability of the VE system, and reports of simulator sickness symptoms. Structured interviews with randomly selected participants from each group revealed more detailed information. Data indicated the VE group retained more information and had more positive experiences learning about dangers of smoking and assertiveness skills than did the LST group. Usability data showed ease of use and learning of the VE system, with no significant symptoms of simulator sickness. These data indicated that this VE application is a promising tool for keeping teens healthy.

Keywords: virtual reality, prevention education, learning environment, assertiveness skills, tobacco, drugs

1. INTRODUCTION

The societal costs of smoking tobacco are tremendous. Smokers are three times more likely to die of cancer than non-smokers. Since the 1950's, lung cancer has increased more than 400% in women; more than 80% of cancer deaths are linked to cigarettes; and, smoking contributes to the contraction of a variety of other cancers, as well as to heart disease and high blood pressure. The nation's annual medical bill from tobacco-related illness is $50 billion1. When other costs are included, such as time lost from work, the U.S. Congress Office of Technology Assessment estimates that smoking costs more than $140 billion every year (more than $4,400 a second).

Unfortunately, smoking is on the rise among adolescents.2,3 Although further research of smoking prevention and cessation in adolescents is needed, little progress has been made;4 additional efforts to prevent teenage smoking are needed. While many prevention programs are effective, there are some drawbacks to their use. Prevention programs are accessible only at certain times and for brief periods, and require periodic booster sessions to prevent smoking incidence. Combining the beneficial aspects of smoking prevention programs with the unique multisensory, 3D qualities of virtual environment (VE) technology, may address some of the disadvantages of traditional prevention programs. First, VE technology can be particularly engaging and acceptable for teens since the computer-generated sights and sounds are similar to popular video games. Second, VE technology may present information and skill building exercises, such as refusal skills, more persuasively than is possible with text and videotapes. Third, the VE program could be continuously available for students in classrooms and computer labs, providing sources for booster sessions.

1.1. Virtual environment technology.
Immersive virtual environments (IVEs) have properties that make them uniquely suited for supporting cognitive-behavioral interventions and educational processes. A virtual environment (VE) is defined as an application that allows persons to navigate and interact in real time with a computer-generated world. An immersive VE (IVE) is distinguished by presenting the individual a multisensory, 3D computer-generated world that surrounds the person. In putting on an accompanying headset and starting the IVE program, the viewer is fully immersed--or surrounded--by the computer-generated "world"; by turning one's head the viewer sees a new view or angle of the scene. The effect simulates the way a person sees, hears and interacts in the physical world. VE technology is being developed for a range of education and training applications.5

1.2. Life Skills Training (LST).
LST is a cognitive-behavioral prevention program that has been shown to be effective up to six years after the original intervention.6,7,8 The main purpose of the LST approach is to facilitate the development of generic cognitive-behavioral skills such as decision-making, as well as skills and knowledge more specifically related to resisting social influences to use tobacco products and other drugs. There are three major LST components.9 The Substance-Specific Component contains material describing the short- and long-term consequences of substance abuse; information about prevalence rates among both adults and adolescents in order to correct normative expectations; information and class exercises demonstrating the immediate physiological effects of cigarette smoking; material concerning media pressures to smoke or drink; and techniques for resisting peer pressure to smoke or use other drugs. The Personal Skills Component contains material designed to foster development of critical thinking and responsible decision-making, provide students with techniques for coping with anxiety, and provide students with the basic principles of personal behavior change and self-improvement. The Social Skills Component contains material designed to improve general interpersonal skills such as effective communications, and verbal and non-verbal assertive skills. These skills are taught using a combination of instruction, modeling, rehearsal, feedback, reinforcement, and extended practice through homework assignments. Teaching these skills has been effective in preventing substance abuse, as well as enhancing general and specific coping abilities in non-abusing populations.

1.3. Integrating LST with VE technology.
Virtual reality may prove to be exceptionally attractive and accepted by adolescents and thus more effective than other instructional media10. Virtual worlds are engaging. People who are introduced to this technology want to use and re-use the programs often because virtual worlds provide both a sense of excitement and a very personal and private space due to the enclosing head-mounted display. As a result, teens are more likely to enjoy and use a tool that provides a multisensory experience that surrounds them and reacts to their movements than they are to sit passively and listen to an audiotape or watch a video. There are a number of other ways, based on issues known to influence the effectiveness of prevention programs,9,11 in which immersive VE technology may be useful in augmenting the effectiveness of LST:

While the magnitude of the effects of prevention programs range from 50 to 80% reduction in the incidence of substance use behavior, there are few data on the critical components that make programs effective.5 Other delivery technologies, like immersive VE technology, may enhance effectiveness.

The three-dimensional images and interactivity possible with immersive VE technology may enable the presentation of information and skill-building exercises (e.g., refusal skills), more persuasively than is possible with videotapes and other modes of presentation.

The multisensory nature of immersive VE technology may allow more compelling presentations of educational material such as showing the immediate negative consequences of smoking.

The immersive option of VE technology may eliminate social and environmental distractions and result in greater individual focus, concentration, integration of experience and transfer of learning.

The computer-based nature of VE technology makes it relatively easy to modify training scenarios to allow teens to practice general and specific skills in a variety of situations.

VE technology may be more compelling and acceptable to adolescents. The computer-generated sights and sounds that surround the person are similar to popular video and arcade games. In general, teens understand and accept the technology and are more likely to use it, thus providing greater opportunities for learning and practicing LST skills.

This pilot project integrated VE technology with educational and skill-building material from Life Skills Training (LST). The VE system described in this article was designed as an adjunct to the LST program, providing another media for presenting information and allowing students to practice cognitive-behavioral skills. The dynamic combination of cognitive-behavioral approaches and virtual sights and sounds promises to introduce and reinforce powerful tools for helping individuals learn the life skills necessary to avoid using tobacco products and avoid engaging in other detrimental behaviors. Material from all three LST components were provided in the virtual learning environment; however, only results from some of this material are presented here: Substance-Specific (short- and long-term consequences of substance abuse); and Social Skills (verbal and non-verbal assertive skills taught using a combination of instruction, rehearsal, feedback, and reinforcement).
This article provides data from a pilot study designed to evaluate whether prevention education material presented with VE technology resulted in enhanced learning when compared with using traditional methods found in prevention education groups, and whether the virtual learning environment would be more engaging than traditional programs, which would help motivate students to use the VE tool.

2. METHOD

2.1. Participants
Seventh grade students were recruited from a local middle school and screened to obtain the following characteristics: no alcohol or other drug ingestion for 72 hours prior; no psychiatric morbidity; no psychotropic medications; no history of seizure disorders or known intracranial pathology; no susceptibility to motion sickness; normal or corrected-to-normal vision and hearing; English readers to ensure understanding of the questionnaire. Study participants were treated in accordance with guidelines from the American Psychological Association. Initially, seventy-two seventh-grade students were randomly assigned to one of three conditions: LST group session, VE program, and non-intervention control. Attrition due to scheduling and transportation issues over the eight-week period resulted in 15 students per group. Of these students, 44 were Latino. Gender was about equally represented in the three groups.

2.2. Apparatus and Materials
The LST program included a workbook,12 group discussion, and role-playing activities related to practicing assertiveness skills.
The VE system included goggles (Virtual I/O), synthesized speech, electromagnetic spatial trackers on head and hand, hand-held controller, and a simulated speech recognition system. The VE program provided information on the dangers of smoking and opportunities for practicing assertiveness skills with virtual characters; additional information was provided with a workbook12 and discussion.
Questionnaires were used to measure participants’ smoking knowledge and behavior, perception of the smoking prevention programs, and simulator sickness.

Smoking Knowledge And Behavior Questionnaire. This questionnaire consisted of 60 questions that covered the following categories: (1) use of cigarettes and other drugs, (2) intention to use cigarettes and other drugs, (3) perceptions of use among peers, (4) perceptions of use among adults, (5) attitudes towards smoking, (6) probability of refusal, (7) probability of using specific types of refusal skills, (8) autonomy in decision making, and (9) probability of using techniques to reduce anxiety.5 Only some of this data will be presented in this article. Prior to data analyses, all items were re-stated as positive statements to facilitate analysis and interpretation.

Usability Questionnaire. Likert scales were developed to investigate issues related to: (1) overall effectiveness of the test and instructions, (2) aesthetics; e.g., appeal of what participants saw, heard and felt, (3) usability of virtual displays; e.g., comfort, ease of use and perceptibility of visual and auditory displays, (4) usability of virtual controls, e.g., comfort and ease of using the speech and manual input devices, (5) immersion; i.e., feeling or sense of being in another world. Prior to data analyses, all items were re-stated as positive statements to facilitate analysis and interpretation.

Simulator Sickness Questionnaire (SSQ). The SSQ has been validated as a tool to evaluate the severity of motion sickness symptoms associated with using vehicle simulators,13 and was administered to members of the VE group. The SSQ consisted of 16 items describing possible symptoms of simulator sickness: general discomfort, fatigue, headache, eyestrain, difficulty focusing, increased salivation, sweating, nausea, difficulty concentrating, fullness of head, blurred vision, dizzy eyes open, dizzy eyes closed, vertigo, stomach awareness, and burping. The sixteen symptoms can be rated on a four-point scale, with "0" indicating no symptoms and "3" indicating serious symptoms.

Structured interview. The structured interview questionnaire consisted of 28 items about different aspects of the prevention education program. The purpose of the structured interview was to obtain more detailed information about attitudes towards smoking and about the smoking prevention program. Questions about the component providing information about physiological effects of smoking included "What did you think/feel about the part of the program dealing with smoking myths and realities?" and "How was the program helpful or not helpful in teaching you about the dangers of smoking?". Questions about the component providing opportunities for practicing assertiveness and refusal skills included "What did you think/feel about the part of the program dealing with assertiveness?", "What did you think about the role playing exercises?", and "How many different ways do you know of refusing an offer of cigarettes?" Comments about the presentation medium were elicited with questions like "What did you think about the (LST or VE) program?" and "Compare this smoking prevention program with other programs you have participated in."
The data from the interview were subjected to a content analytic procedure.14,15 Content analysis is a research method that uses a set of procedures to make valid inferences from text; the text in this case were answers to structured interview questions. The first step in the content analytic procedure was to construct categories that reflected the primary areas of interest; the many words and phrases of the text were classified into much fewer content categories. Words, phrases or other units of text classified in the same category are presumed to have similar meanings.

For the initial analyses, interview text was categorized as having positive, neutral, or negative connotations for seven components of the prevention education programs: self-esteem, decision-making, dangers of smoking and using other drugs, coping with anxiety, assertiveness and refusal skills, presentation medium, and program facilitator. In this article, only results from dangers of smoking and using other drugs, assertiveness and refusal skills, and presentation medium categories are reported. When coding text, two researchers (who were not interviewers) classified each word or phrase as belonging to only one of the nine categories (three components x three values) discussed in this article; phrases were classified according to the category in which it best fit. The number of words or phrases in each category were counted. Counting assumes that higher relative counts reflect higher concern with the category. The initial question of interest was which, if any, group learned more about dangers of using drugs and about using refusal skills, and which group rated their program most positively. Learning was classified as negative or positive according to particular learning criteria; for example, recall of more than five dangers of smoking was counted as one positive instance, as was recall of more than five different assertive ways of refusing offers of a cigarette. Coders were blind as to the experimental group in which the interview text belonged (e.g., did not know that particular text was generated by a student in the VE or LST group).

2.3. Procedure

Advertise/Recruit/Screen/Assignment. Seventh grade students were recruited from a local middle school. All respondees were screened so only those meeting study criteria were included. Participants were randomly assigned to one of three experimental or control groups.

Informed consent/Baseline (Day 1). Each group met at a different time so procedures specific to their group could be provided without biasing the other groups. The principal investigator provided a complete description of the study's procedures and requirements, answered questions, and obtained informed consent from parents and students.

Skills training/Assessment (Days 2,3,4,5,6,7, 8, 9). Members of the LST and VE groups met for one 50-minute session per week for eight weeks. Each session focused on a different chapter from the LST Student Guide, with single sessions devoted to self-esteem and decision-making, and double sessions devoted to dangers of smoking, anxiety coping skills and assertiveness skills. Only some of this data will be presented in this article.

Members of the LST group met as a group. The group consisted of sharing personal goals, discussing workbook material, and role playing. The group was conducted by an experienced, independent cognitive-behavioral therapist unaware of the hypotheses of this experiment.

Before starting the first session, members of the VE group were given a one-half hour tutorial on how to use the VE apparatus. During this familiarization session, students were not presented with information related to smoking prevention. Following this orientation, members of the VE group met individually for eight sessions with a research assistant. The first 10 minutes were spent discussing personal goals and workbook material. The next 30 minutes were spent in the virtual world. The remaining time was used to put on and off the equipment, complete tests and questionnaires. Before entering the virtual world, the student completed the SSQ and the research assistant measured phoria. SSQ and phoria were measured immediately upon exiting the virtual world and about ten minutes later. The SSQ, but not phoria data, will be discussed in this paper. The usability questionnaire was given at the first and last session.

Members of the Wait group did not have any sessions during this time period.

Post-exposure assessment (Day 10). Members of all three groups returned one week from Day 9 so they could complete the first post-exposure smoking knowledge and behavior questionnaire, and participate in a focus group on smoking prevention programs. One-on-one interviews were conducted with five randomly selected students from each group prior to conducting the focus group; the two interviewers were not aware of the content analytic procedures to be used. The LST, VE and Wait groups met at different times.

Interacting with the virtual learning environment. Students put on a head-mounted display (HMD) to enter a virtual park (Figure 1). The HMD presented visual and auditory stimuli from the virtual park. The HMD also had a microphone so students could talk to virtual characters, a sensor on a gloved hand so the computer could draw a virtual hand that moved like the physical hand moved, and a hand-held controller to allow interactions with the computer simulation. The controller enabled the student to move through the virtual park, initiate the speech recognition system, and grab virtual objects.

The virtual park consisted of a number of objects, including trees, benches, and virtual characters who tried to get the students to smoke cigarettes. The encounter began with the character talking to the student (male synthesized voice). A menu of four different response options for the student was then shown in the upper left screen. The student could respond to any one of the options (using the speech recognition system). The response choices for the student ranged from accepting a cigarette or making the suggestion to smoke marijuana (less desirable responses) to refusals with offers of prevention information or alternative activities (more desirable responses). Upon choosing a response, a virtual tutor (female synthesized voice) provided verbal feedback. Students also received more points for more assertive or informative responses. Following is sample dialogue with feedback from the virtual tutor:

Smoker: "Let me turn you onto something" (offering cigarette)
Response options (with tutor's feedback):
A. Thanks, but I don't smoke ("Good job of being assertive")
B. I've got other plans, wanna come? ("Good alternative suggestion")
C. I don't know if I should ("You are too unsure of yourself")
D. Sure. Give me one. ("Bad move")

Once the student chose a response, the student's score would change accordingly, the virtual tutor would provide feedback, then the virtual smoker would respond to the student's choice to continue the dialogue. When the character responded to the students' choice, another menu appeared providing the student with four more phrases to say. This dialogue continued for several rounds until it ended, allowing the student to explore other parts of the virtual world or interact with other virtual smokers.

When the students earned a certain number of points, they had the option of interacting with different versions of a virtual lung (Figure 2). In the virtual lungs, students could choose to see the effects of smoking cigarettes on the lungs as a function of smoking frequency and duration (e.g., a pack a month for a year, two packs a day for ten years). Students also had the option of interacting with objects in the virtual lungs to listen to different pieces of information (e.g., "Less than one out of five kids your age smoke cigarettes", "Each cigarette you smoke takes six minutes off your life", "Smoking cigarettes makes it harder for you to run and play sports"). Each time students listened to different pieces of information, they earned more points. After listening to the information, students had the option to answer multiple-choice quizzes, provided as text, covering information that had been provided. Students earned game points for all interactions in the virtual lungs.

2.4. Design

The major independent variable was the type of smoking prevention program (VE, LST group, no program). The primary outcome measures were self report of smoking knowledge and behavior by responding to a questionnaire5 and participating in a structured interview. In addition, members of the VE group were administered questionnaires for assessing the usability of the VE program and the presence of simulator sickness symptoms as a result of using the VE program.

Smoking Knowledge And Behavior Questionnaire. A two-way ANOVA was used to evaluate presence of group differences on this questionnaire. Independent variables were group (LST, VE, and Wait) and questionnaire item (60 items). The dependent variable was the difference between pre-exposure and post-exposure scores. The Joint Bonferroni test served as the post hoc analysis to indicate the differences in cell means from the group means.

Usability Questionnaire. Students' ratings on the questionnaire items were analyzed to determine whether they were significantly different from neutral, a rating of "5" on the 9-point scale. A one-sample t-test, with questionnaire item as the independent variable, was performed for each of the questionnaire categories. If significant differences were found (i.e., p < 0.05) for item category (e.g., "immersion"), one-sample t-tests were performed to determine whether specific items in each category were statistically different from "5". Criteria for these post-hoc comparisons were set at 0.05/n, where "n" was the number of items in that category.

Simulator Sickness Questionnaire (SSQ). Repeated measures ANOVAs were used to evaluate whether simulator sickness symptoms resulted from exposure to the virtual learning environment. Independent variables were SSQ item (16), repetition of administration per each session (3), and number of sessions (5). (SSQ data from the two relaxation sessions were not analyzed and only five of the remaining sessions were analyzed for all participants to accommodate absences of two participants). If significant differences were found (i.e., p < 0.05), an adjusted Bonferroni test was used to determine whether specific cells were statistically different. Criteria for these post-hoc comparisons were set at 0.05/n, where "n" was the number of items in that category.

Content analyses. A two-way ANOVA was used to evaluate presence of group differences on this structured questionnaire. Independent variables were group (LST and VE) and category (dangers of smoking, assertiveness, and medium of presentation). The number of negative instances in each of the three categories ("dangers", "assertiveness" and "medium") were subtracted from the number of positive instances. This difference score was used as the dependent variable. The Joint Bonferroni test served as the post hoc analysis to indicate the differences in cell means from the group means.

3. RESULTS

3.1. Smoking knowledge and behavior questionnaire

3.1.1. Use and intent to use drugs. Pre-exposure data showed no significant differences between groups on the Smoking knowledge and behavior questionnaire when there were three groups of 24 students per group. However, by the end of the study, there were only 15 students per group, and an analysis of the pre-exposure difference scores showed differences on some items. Consequently, we used difference scores (post-exposure minus pre-exposure scores) as the dependent variable for subsequent analyses.

The post-exposure difference scores indicated no differences between the three groups in frequency of drug use, F(2,294) = 0.41, p=.663; no differences in type of drug used, F(6,294) = 0.22, p=.971; and no interaction between group and type of drug used, F(12,294) = 0.79, p=.659. The mean pre- and post-exposure ratings for all three groups ranged from 1.09 to 1.28, where "1" represented "never used drugs" and "2" represented use of drugs a few times but not in the past year. The mean ratings indicated that students entered the study as non-users and left as non-users.

The post-exposure difference scores also indicated no differences between the three groups in intent to use drugs, F(2,252) = 1.69, p=.186, no differences in intention for using different kinds of drug, F(5,252) = 0.5, p=.778, and no interaction between group and intent to use different types of drugs, F(10,252) = 1.57, p=.114. The mean pre- and post-exposure ratings for all three groups ranged from 1.11 to 1.44, where "1" represented "definitely no intent to use drugs" and "2" represented "probably no intent to use drugs". The mean ratings indicated that students entered the study with no intention to use drugs and left with no intentions to use drugs.

3.1.2. Assertiveness and refusal skills. The post-exposure difference scores indicated no differences between the three groups in their willingness to say "no" when offered a range of drugs, F(2,267) = .67, p=.515; no differences across groups in their willingness to say "no" to a particular type of drug, F(5,267) = .33, p=.896, and no interaction between group and drug, F(10,267) = 1.46, p=.157. The mean pre- and post-exposure ratings for all three groups ranged from 1.08 to 1.59, where "1" represented "definitely would say 'no'" and "2" represented "probably would say 'no'". The mean ratings indicated that students entered the study with a willingness to refuse offers of drugs.

The post-exposure difference scores also indicated no differences between the three groups in the type of refusal methods they would use, F(2,314) = .07, p=.934; no differences across groups in the preferred refusal method, F(6,314) = 1.51, p=.176, and no interaction between group and method, F(12,314) = .61, p=.831.

3.2. Structured interviews

Participants in the VE group provided more positive statements about the program module (dangers of smoking, learning refusal skills, presentation medium) than did participants in the LST group. The mean difference between positive and negative comments was 8.3 for the VE group and 3.3 for the LST group, F(1,24) = 6.79, p = .016. No main effects for category nor the interaction of group and category were found.

3.3. Usability questionnaire

For each section of the usability questionnaire, a one-sample t-test was performed to compare the sub-section’s grand mean ratings to neutral or "5".

Overall Effectiveness of the Test and Instructions. The mean rating for these items was 7.72, t(207) = 20.54, p = .000. Post-hoc analyses showed most of these items to be significantly positive as well, including "It was fun to be in the virtual world" (mean of 7.81), "Being in the virtual world taught me about the dangers of smoking" (mean of 8.06), and "I would like to be in the virtual world again" (mean of 7.5).

Aesthetics. The mean rating for these items was 6.4, t(191) = 7.78, p = .000. However, post-hoc analyses showed only two of these items to be significantly positive. These were "I liked the colors in the virtual world" (mean of 7.25) and "I liked how the virtual lung looked" (mean of 7.75). The rest of the items, while all with mean ratings greater than "5", did not meet criteria for statistical significance. These included items such as "I liked how the virtual park looked" (mean of 6.56), "I liked the way the characters in the virtual world looked" (mean of 6.44), and "I liked how my hand looked in the virtual world" (mean of 5.19).

Virtual Displays. The mean rating for these items was 6.86, t(271) = 12.89, p = .000. Post-hoc analyses showed most of these items to be significantly positive as well, including "It was easy to use the virtual reality equipment" (mean of 8.12), "It was comfortable wearing the goggles" (mean of 7.56), and "I understood the synthesized voice" (mean of 7.06), and "It was easy to read the words on the screen with the goggles" (mean of 7.19). However, other items were not rated as highly. These included items on the realism of the interaction such as "The way the characters spoke to me seemed realistic" (mean of 5.81) and "When I grabbed virtual objects it felt like I was grabbing real objects" (mean of 4.94). The latter rating is not surprising; while there were kinesthetic sensations from interacting with the hand-held controller, there were no tactile or force feedback devices in the VE system.

Virtual Controls. The mean rating for these items was 7.14, t(223) = 13.16, p = .000. Post-hoc analyses showed most of these items to be significantly positive as well, including "It was easy to move around in the virtual world" (mean of 8.06), "If I was refusing an offer to smoke in real life, I would say things just like the way I spoke to the virtual characters" (mean of 7.47), "My conversations with the characters seemed realistic" (mean of 7.25), and "It was easy to use the menus to have conversations with the virtual characters" (mean of 7.19). However, other items were not rated as highly. These included items on the realism of the interaction such as "When I talked with the characters it felt like I was talking with a real person" (mean of 6.81).

Immersion. The mean rating for these items was 6.17, t(143) = 4.7, p = .000. However, post-hoc analyses showed no item to be significantly positive.

3.4. Simulator sickness questionnaire

To determine whether there was an effect of session on the item ratings, a repeated measures ANOVA was conducted with session as one independent variable (5 sessions), SSQ item as a second independent variable (16 items), and ratings as the dependent variable. There was no main effect of session, F(4,561) = 2.37, p=.064), nor for the interaction of session by questionnaire item, F(60,840) = 0.87, p=.74). Therefore, participants' mean ratings across session were used in subsequent data analyses.

Two different subsequent analyses were conducted. One analysis was conducted to evaluate whether the first post-exposure symptom ratings were significantly different from zero, which would indicate whether exposure to the goggles and virtual world caused simulator sickness symptoms. The second was conducted to evaluate whether the second post-exposure symptom ratings scores were significantly different from zero, which would indicate whether symptoms resulting from exposure had disappeared. The results are shown in Table 1. To reduce Type I error, the criteria for significance was determined to be .05/16 = .003. All mean ratings were less than "1", indicating minor simulator sickness symptoms. The results indicate the symptom ratings were not statistically different from "0", although items 3, headache, and 4, eyestrain on the first post-exposure measure were close.

Table 1. The mean ratings (and p values) of simulator sickness symptoms for the first and second post-exposure measures indicated that exposure to the virtual learning environment did not cause significant symptoms

Evaluations during this pilot study indicated effective presentation of prevention information as well as ease of use and acceptability of the virtual learning environment. Other groups have indicated the potential benefits of virtual environment technology for treating phobias such as fear of flying,16,17 fear of heights,18,19 and fear of spiders,20 and there is interest in using the technology in other clinical and mental health arenas.21,22,23. This pilot project showed the utility of VE technology for promoting mental health through acquisition of prevention education and skills.

It is critical to emphasize that the VE program was designed as an adjunct to a prevention education program like LST, not as a competitor. The purpose of this study was to determine whether some material in the LST (or other prevention education) program could be better presented with a VE medium. The results indicated that this was so, and indicated new directions for developing prevention education programs.

4.1. Performance.

While we did not expect to find differences in use of cigarettes and other drugs in an eight-week study using relatively few students, we did expect to find differences in learning. Despite the relatively small sample size and short time period for the prevention education, there were significant benefits of using the VE program. Students participating in the VE program learned more about the dangers of smoking and about refusal skills than did students in the LST program, and rated their overall program more positively.

Analyzing data from the smoking, knowledge and behavior questionnaire indicated no difference in learning between groups whereas data from the structured interviews showed greater learning in the VE group. The difference in results probably are due to the different ways the questions were asked. To score high on the questionnaire, students only had to recognize the correct response, whereas to score high during the structured interview, students had to recall the information. Recall would result in lower scores for those who really hadn't mastered the information.

It should be noted that these results are confounded by an experimental design in which the LST group participated in a group learning environment and the VE group participated in a one-on-one individual learning environment. These differences in learning environment also may have contributed to differences in results. It is hypothesized that one of the primary advantages to the VE program derived from the greater interactivity afforded the students and the immediate feedback provided the student by the virtual tutor in the virtual park and by the testing component in the virtual lung. In the group LST program, all students had the opportunity to role play and to answer questions. However, the amount of time they actually were active was very small; much of the time, students would watch others in role plays or listen to others provide answers to questions. In contrast, students using the VE program actively participated in all the role plays and in answering all the questions; they could not just sit and watch. Such active interactions are critical for the most effective learning.

Similarly, students in the group LST program could attend or not to feedback from peers and the facilitator; many inherent distractions in a group setting could attract the students' attention away from task-relevant activities. However, the exclusivity and immediacy of the VE program eliminated distractions and required attention to the process. Subsequent studies will disambiguate the results.

4.2. Ease and effectiveness of use.

The usability evaluations showed that participants liked using the virtual learning environment, were able to see and hear stimuli sufficient to perform the tasks, and were able to easily use the manual and speech controls to interact in the virtual world. The SSQ data also indicated no adverse affects from using the VE apparatus.

The design team was concerned with providing functional fidelity, in which task-relevant cues were provided in the environment, rather than providing physical fidelity, in which all aspects of the physical stimuli were matched by virtual cues. Actual physical environments provide rich multidimensional patterns of sensory inputs to an observer. It is not necessary, however, to realistically represent all aspects of the physical environment in the VE because not all information that is available in either the physical or VE is actually functional; i.e., essential for performing the task.24,25 In fact, it is difficult to provide physical realism to simulated environments, and it may not be necessary.26,27 For instance, training with flight simulators appears to be most effective when the simulator is designed to provide functionally relevant rather than objectively correspondent stimulus cues;28 what is critical is that the right task cues are presented in the virtual environment.

Designing the VE program for this pilot study concentrated on effectiveness of presenting information (e.g., legibility of text and comprehension of the synthesized voice) and effectiveness of the manual and speech controls. These areas were positively rated. In contrast, there was little expectation for the aesthetics of the computer graphics to be rated particularly well because there was not as much effort spent on these areas while developing the pilot project. Similarly, there was no expectation of positive ratings for questions related to the realism of the interaction. The usability evaluations showed ease of use and effectiveness in expected functional areas. Despite their relatively low ratings of the aesthetics (though still significantly positive overall), students indicated their enjoyment in participating in the virtual world. The usability results indicate that the design team was successful in providing functionally relevant cues.

4.3. Future work

These data indicated that this VE application is a promising tool for keeping teens healthy. Subsequent studies will evaluate utility of the virtual learning environment as an integrated component of group prevention programs, like LST and DARE, for a larger sample size and for a longer period of time.

ACKNOWLEDGMENTS

This research was supported by the National Cancer Institute under contract number 1-R43-CA74603-01. We appreciate their support. We thank Gil Botvin, Jennifer Epstein, Judith Lomba and Heather Turey for program guidance; Linda Perez and Pajaro Valley Prevention and Student Assistance for recruiting study participants; Derek Rucker for data collection; Eddo Stern and Edward St. Lawrence for computer programming; Helene Wood, Wellness and Rehabilitation Center and Watsonville Community Hospital for providing facilities. Any opinions, findings, conclusions or recommendations expressed in this publication are those of the author and do not necessarily reflect the views of the National Cancer Institute.

REFERENCES

1. Carey J, Bongiorno L, France M. "The fire this time". Business Week: August 12, 1996.

2. The Youth Risk Behavior Surveillance System survey, Centers for Disease Control, Atlanta, GA, 1988.

3. Johnston LD, O'Malley PM, Bachman JG. National survey results on drug use. Monitoring the Future, Vol. I, Secondary School Students, NIDA, Rockville, MD, 1992.

4. Reducing the health consequences of smoking: 25 years of progress, Department of Health and Human Services (USDHSS), Public Health Service, Rockville, MD, 1989.

5. Youngblut, C, Educational uses of virtual reality technology (IDA Doc. D-2128), Inst. for Defense Analyses, Alexandria, VA, 1998.

6. Botvin GJ, Botvin EM, Ruchlin H. "School-based approaches to drug abuse prevention: Evidence for effectiveness and suggestions for determining cost-effectiveness," Bukoski W., ed. Cost-Benefit/Cost Effectiveness Research on Drug Abuse Prevention: Implications for Programming and Policy, NIDA Monograph, U.S. GPO, Washington, DC, 1997.

7. Botvin GJ, Epstein JA. "Preventing cigarette smoking among children and adolescents," Seidman DF, Covey L, eds. Helping smokers quit: Handbook for the helping professions, Lawrence Erlbaum Associates, Mahwah, N.J., 1997.

8. Botvin GJ, Tortu S. "Preventing adolescent substance abuse through life skills training," Price RH, Cowen EL, Lorion RP, Ramos-McKay J, eds. Fourteen Ounces of Prevention, APA, Wash DC, 1988.

9. Botvin GJ, Wills TA. "Personal and social skills training: Cognitive-behavioral approaches to substance abuse prevention," Bell C., Battjes R, eds. Prevention research: Detering drug abuse among children and adolescents, NIDA Research Monograph 63, DHHS No. (ADM) 90-1334USGPO, pp. 8-49, Wash. D.C., 1985.

10. Evans RI, Raines BE. "Control and prevention of smoking in adolescents: A psychosocial perspective," Coates TJ, Petersen AC, Perry C, eds. Promoting Adolescent Health: A dialog on research and practice, Academic, NY, 1982.

11. Evans RI, Raines BE, Getz JG. Applying the social inoculation model to a smokeless tobacco use prevention program with little leaguers, USDHHS Monograph 2. NIH Pub. No. 93-3461, USGPO, pp. 260-275, Washington, D.C., 1992.

12. Botvin GJ, Life Skills Training: Promoting health and personal development. Princeton Health Press, Princeton, NJ, 1996.

13. Kennedy RS, Lane N. "Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness," The Int. J. Aviation Psychology, 3(3), pp. 203-220, 1993.

14. Holsti, O.R., Content analysis for the social sciences and humanities, Addison-Wesley, Reading, MA, 1969.

15. Weber, R.P., Basic content analysis, Sage, London, 1990.

16. Hodges LF, Rothbaum BO, Watson B, Kessler GD, Opdyke D. "A virtual airplane for fear of flying therapy," Proc. VRAIS '96, pp. 86-93, IEEE, Los Alamitos, CA, 1996.

17. North M, North S, Coble J. "Virtual reality therapy for fear of flying," Am J Psychiatry, 154, pp. 130, 1995.

18. Lamson RJ, Meisner M. "Effects of virtual reality immersion in the treatment of anxiety, panic, & phobia of heights," Proc. 2nd Int. Conf. Virtual Reality and Persons with Disabilities, Cal State U., Northridge, 1994.

19. Rothbaum BO, Hodges LF, Kooper R, Opdyke D, Williford J, North MM. "Virtual reality graded exposure in the treatment of acrophobia," Behav. Ther., 26(3), pp. 547-54, 1995.

20. Carlin A, Hoffman H, Weghorst S. "Virtual reality and tactile augmentation in the treatment of spider phobia: A case report," Behav. Research & Therapy 35(2), pp. 153-58, 1997.

21. Glantz K, Durlach N, Barnett R, Aviles W. "Virtual reality (VR) for psychotherapy: From the physical to the social environment". Psychotherapy, 33(3), pp. 464-473, 1996.

22. Riva, G., Bolzoni, M., Carella, F., Galimberti, C., Griffin, M. J., Lewis, C. H., Luongo, R., Mardegan, P., Melis, L., Molinari-Tosatti, L., Poerschmann, C., Rovetta, A., Rushton, S., Selis, C. & Wann, J., "Virtual reality environments for psycho-neuro-physiological assessment and rehabilitation," Morgan, K. S., Hoffman, H. M., Stredney, D. & Weghorst, S. J., eds., Medicine Meets Virtual Reality: Global Healthcare Grid, Vol. 39, pp. 34-45, IOS Press, Washington, DC, 1996.

23. CyberPsychology & Behavior (Special issue, Virtual reality and mental health symposium), vol. 1 (2), Mary Ann Liebert, Larchmont, NY, 1998.

24. Brunswik E. Perception and the Representative Design of Psychological Experiments. University of California Press, Berkeley, CA, 1956.

25. Warren R, Riccio GE. "Visual cue dominance hierarchies," Trans. SAE, 6, pp. 937-951, 1985.

26. Prophet WW, Boyd HA. Device-Task Fidelity and Transfer of Training: Aircraft Cockpit Procedures Training (HumRRO-TR-70-10), HumRRO, Alexandria, VA, 1970.

27. Semple CA, Hennessy RT, Sanders MS, Bross BK, Beith BR, McCauley ME. Aircrew Training Device Fidelity Features, (AFHRL-TR-80-36), AF Human Resources Lab, Brooks AFB, TX, 1981.

28. Jones ER, Hennessy RT, Deutsch S. Human Factors Aspects of Simulation, National Academy Press, Wash, D.C., 1985.