ASSESSING ENGAGEMENT AND SATISFACTION IN TWO RELATIONAL AGENT INTERVENTIONS FOR PHYSICAL ACTIVITY AND SUN PROTECTION

Relational agent (RA) health interventions are slowly becoming part of the health behavior change field. The current assessed participants’ experience in a 12month long internet and RA intervention for physical activity (PA) and sun protection, using primarily a qualitative approach. Two groups were recruited. Group 1 consisted of participants who completed the intervention. Group 2 consisted of participants who accessed the intervention only two or three times. Thirty-four participants were recruited for group1 and eight participants for group 2. A standardized semistructured open-ended interview was used for data collection. Two interview scripts were developed and consisted of twenty-five questions (group1) and ten questions (group 2) that assessed different components of participants’ experiences with the program, including motivation, engagement, satisfaction/dissatisfaction, interaction with a RA, and behavior change. Manifest content analysis and latent content analysis were used to assess participants’ responses. An analysis of variance was used to assess levels of satisfaction between men and women. The study results showed, overall, participants in group 1 were motivated, satisfied with the intervention, liked the relational agent and reported behavior change in PA and sun protection. Conversely, most participants in group 2 were dissatisfied with the intervention, disliked the RA and reported no behavior change. No gender effect was noted for satisfaction. The implications of these results, including the importance of RA intervention and qualitative methods in health behavior change, are discussed.

Relational agent health interventions are slowly becoming part of the health behavior change field (Brickmore et al., 2010;Campbell, Grimshaw & Green, 2009).
To date, a number of these interventions have been used to promote different healthrelated behaviors, including PA (Brickmore, Gruber & Picard, 2005), medication adherence (Brickmore et al., 2010) and also in HIV prevention research (Kok, Vriens, de Zwart & Hospers, 2006). Given the evidence of their effectiveness, relational agent interventions can potentially make substantial contributions to decrease different risky behaviors . Brickmore et al. (2010) describe the relational agent-based intervention as interactive throughout the course of the intervention. The agents provide participants with support and encouragement regarding the target behavior. There is extensive research indicating that providing support and encouragement are central when helping a person to engage in a new behavior and also maintaining it overtime (Prochaska & Velicer, 1997;Abramson & Michie, 2008).
Relational agent interventions, overall, have shown to be more effective when compared to non-relational agent treatments (Campbell et al., 2009). There is empirical evidence that the "caring" and "supportive" features of the relational agent contribute to behavior change .
Given the potential of these interventions for health behavior change research, in-depth assessments of participants' experiences with these interventions are needed to increase our understanding of participants' interactions with the relational agents and subsequent outcomes. This current study utilized a qualitative approach to explore participants' experiences in a relational agent intervention for PA and sun protection, titled Online Tailored Interventions & Relational Agents for Sun Protection and PA.
Qualitative methods have been used effectively to gain insight into participants' experiences in health-related interventions, including behaviors such as smoking cessation and physical activity (Allender, Gill & Foster, 2006;Doucet, 2009).
The use of qualitative research could potentially provide important information regarding the use of relational agents for behavior change. Given the implications for qualitative research in the advancement of knowledge in relational agent interventions and prevention, researchers at the Cancer Prevention Center (CPRC), at the University of Rhode Island, are conducting in-depth assessments of participants' experiences with these interventions.

Current Study Aims:
Primary Aim: This study assessed participants' experiences, including level of satisfaction and level of engagement in a computer-delivered, relational agent intervention for PA adoption and sun protection.
Secondary Aim: This study explored whether participants' level of engagement and level of satisfaction differed by race (Black and White) and gender (men and women).
individuals expected to die from the disease (Siegel, Ma, Zou & Jemal, 2014). The estimates of melanoma cancer rates for 2013 were 76,690 and show a decline of 590 cases for 2014. However, the expected death rates from this disease continue to remain alarming. According to Valdivieso, Kujawa, Jones and Baker (2012), survivors of this disease often experience severe psychological consequences, including the development of depression due to deformity from surgical procedures and financial burden.
Data on skin cancer continue to show lower prevalence rates among minority groups, such as African Americans, in comparison to Whites (Battie, Gohara, Verschoore & Roberts, 2013). However, mortality in these subgroups are significantly higher than Whites. There is empirical support to show that skin cancer rates have increased among minority groups, including African Americans (Battie et al., 2013). There is no explanation in the literature to elucidate this change. The paucity of research on skin cancer with minority samples, especially African American populations, likely supports this gap in the literature. Battie et al. (2013) Moreover, they mentioned this increase might be due to a lack of knowledge on UV exposure and skin cancer risks among minority subgroups. A number of studies show that 65% of African Americans reported never using sunscreen even though they reside in sunny climates (see Battie et al., 2013). Additionally, numerous studies show approximately 60% of African Americans believed they are not at risk for skin cancer (see Battie et al., 2013).
The adverse effects of physical inactivity and chronic UV exposure represent major health concerns and thus a public health priority is to identify factors to promote these behaviors and lower skin cancer risks. The assessment of barriers, such as cognitions and behaviors specific to physical inactivity and lack of sun protection behaviors is important for the development of effective strategies.

Transtheoretical Model (TTM).
The TTM is a comprehensive framework comprising of multiple dimensions of behavior and behavior change (Prochaska &Velicer, 1997;Velicer et al., 2000). This model has been applied to many and different health behaviors (see Hall & Rossi 2008), including physical activity (Blissmer & McAuley, 2002;Marshall & Biddle, 2001) and sun protection (Prochaska et al., 2004;Prochaska et al., 2005). The core constructs of the TTM are decisional balance, stages of change, self-efficacy and processes of change. The decisional balance construct describes an individual's weighing of pros and cons (i.e., perceived benefits and perceived risks) for engaging in a behavior (Velicer, DiClemente, Prochaska, & Brandenburg, 1985) and been demonstrated to be a powerful construct in predicting future behavior (Hall & Rossi, 2008;Velicer et al., 1985). There are five stages of change, and each stage represents a temporal and developmental order of readiness to change. Specifically, each stage identifies a person's intention regarding a behavior: Precontemplation (no intention to change), Contemplation (considering changing), Preparation (intending to change), Action (engaging in the change behavior) and Maintenance (regular change for 6 months or longer) (Velicer et al., 1985;Prochaska &Velicer, 1997). Stage progression can be either linear and or cyclical (Marshall & Biddle, 2001). In the cyclical formulation, individuals in later stages (i.e., preparation and or maintenance) may regress to earlier stages and thus show a temporary change (Marshall & Biddle, 2001;Velicer et al., 1985). The progress of change has shown to be mediated by an individual's self-efficacy for change and decisional balance (perceived pros and cons for becoming physically active) as they move through the stages (Velicer et al., 1985;Blissmer & McAuley, 2002).
The self-efficacy construct is based on the Bandura's self-efficacy model (Bandura, 1977) and models of behavior maintenance (Velicer, Diclemente, Rossi & Prochaska, 1990). This construct measures "the intensity of urges to engage in a specific behavior when in difficult situations" (Plummer et al., 2001, p. 544), and the confidence to avoid engaging in a particular behavior despite being in difficult situations (Velicer et al., 1990).
The Processes of Change construct of the TTM consists of two correlated experiential and behavioral domains (DiClemente & Prochaska, 1982). These domains describe covert and overt activities and strategies through which people modify their problem behavior (DiClemente & Prochaska, 1982;Prochaska et al., 1985). Different problem behaviors are associated with different processes. The experiential processes represent more covert cognitive and affective experiences or activities, and the behavioral processes represent overt strategies, such as developing or finding healthy alternatives, finding social support, and changing the environment to promote behavior change (DiClemente & Prochaska, 1982;Prochaska et al., 1985).
According to Velicer et al. (1985), the four TTM constructs (Decisional Balance, Stage of Change, Self-efficacy and Processes of Change) work in concert throughout the process of behavior change.

Project RAISE
Study Design. Project RAISE is a randomized, computer-tailored intervention (one for physical activity and one sun protection) and involves a 3 group by 3 experimental design with repeated measures on the second factor. The three experimental conditions were: 1) a control group, 2) a multiple risk internet group, and 3) multiple risk internet with relational agent condition. The recruitment of participants involved a random digit-dial approach. Participants are randomly assigned to one of the three conditions. Participants were assessed on three occasions: baseline, 12 months follow up (end of treatment), and 24 months (12 months after end of treatment).
Recruitment. Participants accessed the intervention through a CD containing the software or were emailed a link to download the software unto their computer.
Measures. Participants completed surveys assessing key constructs of the TTM. The survey questions assessed behaviors relating to PA or sun exposure, stage of change, decisional balance, self-efficacy and processes of change. Responses to these questions were used to develop tailored feedback for participants. For the relational agent interventions, participants were matched to one of four relational agents based on their demographic information for race and gender. The pictures of the relational agents are in Appendix A.
Sample. A sample of 1364 participants was recruited at baseline. The majority of participants were White (n=1162) and women (n=861). The mode age for participants was 49 years old. The samples for Black and Hispanic participants, respectively, were 90 and 56.
Computer-Tailored Intervention. Computerized-tailored intervention (CTI) has shown substantial promise in health behavior interventions, including PA (Pekmezi et al., 2010). CTI is typically used in population-based intervention has shown to produce higher rates of participation than clinical-based approach (Pekmezi et al., 2010). Expert system interventions are also used in population-based research and are usually computer-based. This form of intervention has been used across different health behaviors. Expert system interventions imitate human reasoning by providing feedback and decision-making rules determined by experts (Velicer & Prochaska, 1999). The feedback is based on the theoretical framework that identifies important constructs for behavioral change. Expert systems provide both normative (compared to population norms) and ipsative feedback (compared to an individual's previous scores) for the most salient variables (Pekmezi, et al., 2010).
Activities. Participants were provided with reports that included ipsative feedback based on their health risks and attitudes toward PA and sun protection. A tracking chart helped participants monitor their PA and sun protection behavior weekly and over the course of the 12-month intervention. Moreover, a workbook provided participants with activities designed to help them reduce physical inactivity and unprotected sun exposure and thus progress to the next stage of change. Lastly, participants received an email reminder if they did not access the program for seven days, which a continued every 7-day period until they accessed the intervention, to help to remain on track with the program.

Relational Agents
Relational agents are computerized characters "that use speech, gaze, hand gesture, intonation and other nonverbal modalities to emulate the experience of human face-to-face conversation with their users" (Brickmore & Picard, 2005, p. 293). These traits help users to relate to the relational agent and build relationships overtime as if they were human (Brickmore et al., 2010). The relational agents' idiosyncrasies include "attributes such as small talk, story-telling and humor, empathy, encouragement, praise, hand gestures and many more" (Campbell et al., 2009, p.1).  mentioned relational agents not only have a memory of previous interactions with users, they are able to recall specific information including trivial information in effort to build relationships. According to Campbell et al. (2009), this approach imitates the evolution of human interaction, showing that individuals get to know each other as they converse more and build on previous conversations. Moreover, Campbell et al. (2009) Bakken et al. (2000), feeling cared for is crucial for client engagement and satisfaction with treatment. It is established knowledge that engagement in treatment is an important factor for positive outcomes (Broome, Flynn, Knight & Simpson, 2007;Joe, Broome, Kowan-Szal & Simpson, 2002). Research on treatment engagement identifies several predictors that can predict whether an individual will follow a treatment regimen or maintain treatment (Broome et al., 2007;Joe et al., 2002).

Engagement and Satisfaction in Health Studies with Relational Agents
There is empirical support indicating that relational agent interventions are able to promote long-term engagement (Brickmore, Caruso, Clough-Gorr & Heeren, 2005;Brickmore, Gruber et al., 2005). In an intervention for PA, Brickmore, Gruber et al. (2005) found the majority of participants reported liking the relational agent experiment and wanting to continue working with the agent in comparison to participants in a control group. Moreover, in a relational agent intervention for PA with older adults, Brickmore, Caruso et al. (2005) found that participants reported a high level of acceptance, likeability and also demonstrated significant improvement in "daily steps walked" compared to a control condition. Those participants also reported feeling that the relational agent cared about them. In this study, participants rated their interaction with the relational agent, Laura, from a scale ranging 1(a stranger) to 7 (a close friend), with a mean rating of 6.8. Brickmore et al. (2010) found that in a sample of patients with schizophrenia, a relational agent intervention for medication adherence was both "accepted and effective" with this group. According to Campbell et al. (2009), in-depth examination of participants' experiences with these interventions is needed to gain deeper insights into the nature of the relational agents with participants. Such a study will help to provide insights into participants' perception on the particular features of the agent that make them feel cared for and may provide additional information that will improve relational agent interventions.

Engagement Research
Most engagement studies have been conducted in the areas of medical care and psychotherapy. In these areas, engagement is assessed by focusing on the clients' or patients' experiences during treatment (Broome et al., 2007;Barber et al., 2001).
Client engagement has been conceptualized as treatment adherence, involvement in treatment and therapeutic alliance (Broome et al. 2007;Simpson, 2004). Factors cited to impact engagement include patients' (or clients) expectations, attitudes toward the target behavior (importance and readiness), perceived barriers and self-efficacy to engage and maintain the behavior (Rose, Bowman, Radziewicz, Lewis & O'Toole, 2009). Therapeutic alliance has been established as a strong predictor of positive psychotherapy outcome (Horvath & Luborsky, 1993;Shirk & Saiz, 1992;Bordin, 1979) and or outcome (Horvath & Symonds, 1991). Garcia and Weisz (2002) found that "therapeutic relationship problems" was a strong factor in treatment dropout or termination. Bakken et al. (2000) found that having a social support network in favor of seeking help was associated "with keeping the first appointment "(p. 193) and also staying longer in treatment. Moreover, Garcia and Weisz (2002) found that "therapeutic relationship problems" (i.e., lack of therapeutic alliance) was the best predictive factor between completers and dropouts in a mental health service setting.
Therapeutic relationship or therapeutic alliance is described as "patient-centered treatment;" wherein the patient feels he or she is able to communicate with her or his provider or clinician and feels listened to and cared for (Stewart et al., 2000). Good communication skills are central to developing and maintaining a "patient-centered treatment" (Stewart et al., 2000). These communication skills include making sure that the patient's emotional and relational needs are valued and addressed in a similarly important manner relative to the informational objectives of the treatment.
Cultural factors, such as race and gender are also cited as correlates underlying participation and attrition in PA interventions (Banks-Wallace & Conn, 2008;Pekmezi & Jennings, 2009 Jennings, 2009) or not shown to provide higher PA outcomes than the standard intervention (e.g., Newton & Perri, 2004). To this researcher's knowledge, no relational agent studies to date have assessed cultural factors and treatment engagement over time.  found that, in a cross-sectional study with a White relational agent for PA with an older, African American sample, found that participants viewed the relational agent as likeable, credible and showed a desire to continue working with her. Additional research is needed to establish whether race-matched relational intervention is related to prolonged engagement, and possibly satisfaction with treatment.

Satisfaction Research
In the social science literature, satisfaction is based on "the experience of the consumer" (Cronin, Brady & Hult, 2000, p.9). This involves the experience of the consumer during the time of the service as well as after service was provided.
Satisfaction is achieved if the consumer reports a positive experience while using the service or after the service is provided. There is a growing body of evidence showing that satisfaction is highly correlated with perceived value/quality of the service or product (Collins & Nicolson, 2002;Anderson et al., 2001;Giese & Cote, 2000).
According to Dearing, Barrick, Dermen and Walitzer (2005), satisfaction is the outcome factor with the patient (or client) having engaged in treatment over a period of time. Through the engagement process, the client assesses whether the service is meeting his or her needs. Thus, a client who is satisfied with treatment is likely to follow the treatment protocol and remain in treatment. There is empirical evidence in the area of health research demonstrating level of satisfaction as an important correlate for both behavior change and maintenance (Chow, Quine, & Li, 2010). Given the implication for treatment satisfaction has great implications for treatment engagement, and outcome, an understanding of the factors underlying satisfaction with treatment is crucial for the advancement of the behavior change research field.

Participation in Health Trials
To date, there is substantial research on factors underlying recruitment or participation in health-related research (Resnicow et al., 2005;Gasglow, Litchtenstein & Marcus, 2003). Computerized tailored interventions-delivered have been shown to be effective in behavior change, including PA (Kroeze, Werkman & Brug, 2006).
Accordingly, this strategy supports retention in health research. According to Noar, Benac and Harris (2007), these interventions, comprised of personalized messages based on the information provided by the individual, are likely more engaging than generic behavior change information. A qualitative review of studies addressing determinants for participation in PA (and sports) conducted by Allender et al. (2006) provides in-depth knowledge regarding differential motivation factors (e.g. healthy benefits) as well as barriers (e.g., lack of social network) for PA engagement among several age groups, including adults and middle-aged adults. Determinants on research participation show minorities groups, including African Americans are difficult to recruit and retain in health research (Wendler et al., 2005;Thompson, Neighbors, Munday & Jackson, 1996). Low social economic status (SES) has also been shown to negatively impact recruitment in health-related research for almost all subgroups (Gross, Filardo, Mayne & Krumholz, 2005).

Preliminary Data for Participation in Project RAISE
A content analysis on usability or participation data was conducted for eightyfour participants who accessed program only a few times (1-4 times) by the 12-month intervention period. The response categories regarding the hindrance of participation most frequently endorsed were as follow: Of the 84 participants, 31% (n=26) reported "lack of time," coupled with "lack of interest." Another frequent category of response was "forgetfulness" regarding accessing the program (n=16). Of that sample, three participants mentioned not remembering due to getting a new computer and or had relocated several times. Moreover, twelve participants (14%) mentioned that the software "could not load" as their reason for not using the program. A number of participants further explained the loading issues were due to program "incompatibility." Ten participants reported that "computer crashed" prevented them from participating in the intervention. Six participants simply stated "computer problem." Six other participants reported "lack of interest" (n=6) as the reason for not participating in the study. Other response categories were: relocated and purchased new computer (n=4), did not install program or software (n=3), "too lazy" (n=3), no computer access (n=2), back injury (n=1), Log-in issues (n=1), partial blindness (n=1), software stopped working (n=1). Two participants could not recall their reasons for not continuing with the program.

A Qualitative Assessment of Participation and Satisfaction
To date, qualitative research has gained increased credibility and acceptability in many fields of science that have traditionally relied upon a quantitative paradigm (Chow et al., 2010;Pope & Mays, 2006). Collins & Nicolson (2002) attribute this change to inadequate understanding of certain behaviors with quantitative methodologies. However, despite the benefits of qualitative research, health-related research continues to remain overwhelmingly quantitative (Chow et al., 2010;Pope & Mays, 2006). To date, there have been a number of theory-based health-related intervention studies that have failed to promote behavior change (see Pekmezi & Jennings, 2009;Baruth et al., 2010). These failures have been attributed primarily to the lack of relevant factors for behavior acquisition and maintenance for the target populations (Pekmezi & Jennings, 2009). A meta-analysis conducted by Thomas et al. (2001) on increasing the use of qualitative research in traditional quantitative fields shows that qualitative research has been useful in helping researchers gain an understanding on the factors underlining, for instance, medication adherence as well as satisfaction with medical care. According to Thomas et al. (2002), most of these studies used an interviewing approach for data generation or data collection. The authors also reported numerous studies using both qualitative research and quantitative approaches in an effort to gain a better understanding of behavior and behavior change. Thus the field of psychology, specifically health psychology, could potentially benefit from using a qualitative approach or a mixed method approach.

Qualitative Interviewing
The purpose of interviewing in qualitative research is to gain the perspective of the individual being interviewed (Kvale & Brinkman, 2009;Kvale, 1996). Kvale (1996) describes interviewing as an important tool that helps the interviewer or researcher assess and or access information that he or she cannot otherwise observe.
These types of information, include feelings, thoughts, and emotions, which according to Kvale (1996), may not be adequately assessed with quantitative approaches. Patton (1980) and other researchers (Kvale, 1996;Seidman, 1991) identify three main interview approaches used in qualitative research. These are: 1) informal conversational interview, 2) general interview guide approach and 3) standardized open-ended interview.
Qualitative interviewing through the approach of Informal conversation interview largely consists of understanding natural occurrences through the individual's perceptions (Driedger, Gallois, Sanders & Santesso, 2006;Seidman, 1991). This form of interviewing is used primarily in observational field research and can utilize a phenomenological framework for understanding the person's experience regarding a particular topic or phenomenon (Driedger et al., 2006). Thus, the Informal conversation interview is an in fact a conversation and thus lacks a predetermined number of questions and a particular format that allows the interview to be open and adaptable to the interviewee's interests or priorities (Driedger et al., 2006). According to Wengraf (2002), questions are generated based on themes that emerge during the interview. For instance, questions may emerge as the interviewer observes the individual in a particular setting or engaging in a behavior (Wengraf, 2002). Therefore researchers are able to address and tailor questions based on the interaction and also in the context of the particular observed situations or behaviors (Wengraf, 2002;Patton, 1980). Of note, there exists much disagreement among phenomenological researchers on data recording during the interview (Wengraf, 2002;Patton, 1980). Wengraf (2002) and Patton (1980) state that data recording such as note taking during the interview is likely to hinder the interview process. In fact Patton (1980) advises that researchers write down the information after leaving the interview and or observation/situation. Conversely, Seidman (1991) mentioned some note taking during the interview is not likely to impact the interview process. The latter approach has been criticized with some researchers questioning the accuracy or credibility of the recall of data afterwards. Major and Savin-Bade (2010) suggest recording data during the interview but only through the use of an audio taping device to maintain the flow of the interaction.
Major and Savin-Baden (2010) caution researchers against using a phenomenological approach, stating that while this method is intended to collect a breadth of data to gain a deeper understanding of the individual's experience, given that it is largely guided by the objectivity of the interviewee, the data collected might in fact lack the depth necessary to understand the phenomenon of interest. According to Wengraf (2002), this issue is particularly prevalent in situations where individuals lack the experience needed to help indirectly guide the interview and remain faithful to the framework. Furthermore, the large amount of data collected across individuals might be difficult for data analysis and comparisons (Wengraf, 2002;Seidman, 1991).
The general interview guide differs from the informal conversation interview approach given that it is rather focused than conversational (Patton, 1980(Patton, & 2002Kvale, 1996). An interview guide is prepared to make sure that essentially the same information is obtained from a number of people by covering the same material and makes interviewing different people more systematic and comprehensive (Patton, 1980. Although the guide keeps the interaction focused it also allows flexibility and opportunities for individual's perspectives to emerge. The standardized open-ended interview differs greatly from the two previous interview approaches since it inherently adheres to a particular standardization (Seidman, 1991).
In the standardized open-ended interview, predetermined or standardized questions are asked to all respondents in the same manner (Seidman, 1991;Patton, 1980). This approach minimizes potential effects of having differential wording and or presentation of a certain topic or issues on participants' responses. Patton (1980) states "by controlling and standardizing the open-ended interview the evaluator obtains data that are systematic and thorough for each respondent but that reduce "flexibility and spontaneity" (p. 1). Consequently, this approach facilitates data analysis, including comparison of patterns across individuals. Wengraf (2002) states that a major flaw of open-ended interview is that the interviewer or researcher is unable to pursue other topics that may emerge during the interview. He added that the standard lines of questioning might hinder the researcher from truly assessing participants' views. He added that a differential line of questions might be more appropriate for a particular respondent (s) than the pre-determined or the standard approach.

Standardized Open-ended Interview in Health Psychology
Qualitative research studies in health psychology typically use a used a standardized open-ended interview approach (Creswell, 2012). In psychology, a few studies have effectively used this method for the purpose of program evaluation, for instance Early Head Start (McAllister, Green, Terry, Herman & Mulvey, 2003). In such studies, the evaluator or researcher explores participants' experiences and satisfaction with care regarding their perspectives, concerns about operations and processes of the delivery of care (Chow et al., 2010;Collins & Nicolson, 2002). Data collected are then used to develop ways to promote both consumers' engagement and satisfaction. This approach is likely to be beneficial in health-related interventions studies by helping to identify barriers and or facilitators for behavior change in a minority population where such studies have been overwhelmingly unsuccessful. A standardized open-ended interview could be specifically beneficial in the assessment of interventions targeting physical activity and sun protection. This approach could particularly be helpful in participation and retention of African Americans in health related research. As mentioned, numerous interventions on promoting PA targeting this group have been unsuccessful (see Pekmezi & Jennings, 2009). This failure has been attributed primarily to a lack of knowledge on relevant factors (i.e., barriers and facilitators), which in turn may lead to dissatisfaction with intervention and attrition.
Utilizing a qualitative approach to help assess intervention components could be valuable in the efficacy and effectiveness of interventions, especially with minority subgroups. There is no substantial research using a qualitative approach to evaluating components of health promotion interventions. These studies typically applied a deductive approach for data collection (e.g., Collins & Nicholson, 2002;Chow et al., 2010). A deductive approach regarding data collection for research purposes is based on a set premises established by a particular theoretical framework (Wengraf, 2002).
Through this theoretical framework, the researcher assesses factors believed to be barriers and facilitators of behavior acquisition and maintenance (Kreuter et al., 2004).
Consequently, the individuals are guided in their responses and may not report their true sentiments or feelings. Thus, an inductive approach is likely to be more effective in identifying pertinent factors related to participation/engagement and satisfaction in health-related research. When using the inductive approach, an acquisition of information is not structured or guided based on a singular theoretical framework.
Rather, numerous factors are theorized to be associated to a construct (i.e., engagement, satisfaction) are explored (Hyde, 2000). Curry, Nembnard and Bradley (2009) emphasize the importance of an inductive approach and standardized openended interviews in health-related research to facilitate disclosure about ineffective methods regarding a behavior to gain insight into behavior change. Resnicow et al. (2005) suggest that future research should interviews with purposive samples (participants who complete the intervention and acquired and maintain the behavior and those who drop out of the intervention) from interventions targeting PA among African Americans to gain sufficient understanding regarding salient factors for the promotion of this behavior. They added the interviewer or researcher should encourage respondents to provide feedback specifically on ways that such interventions could have been beneficial in promoting change.

Facilitating in Depth Information in Interviewing
Seidman (1991) mentioned that qualitative interviews often suffer from several pitfalls, which consequently impacts the quality of data gathered from the interview.

He mentioned that this is particularly relevant to standardized open-ended interviews
where the interviewer has a precise amount of information that needs to be covered for the interview (typically a 1hour) and often takes place only once. According to Seidman (1991) the primary pitfalls relate to incongruence often found between questions and the actual content of the interview.
Consequently some researchers who intend to assess respondents' personal feelings about a particular topic may in fact collect data representing these individuals' opinions. The distinctions for these contents are defined below.
According to Patton (1980), qualitative interviewing questions for the purpose of assessing an individual's experience or behavior are developed to illicit information about what the person has done as a result, for example, of participating in a PA program. Specifically, these questions address behaviors, and activities the person has adopted consequently from participating in the program. Additionally, these questions explore the individuals' goals and desires and ways the program either met those needs goals and or failed to do so. An example of this question is: "If I followed you through a typical day, what would I have seen you doing" (Patton, 1980, p.207).
On the other hand, Opinion Value Questions assess an individual's opinions regarding a particular topic program (Patton, 1980;Kvale, 1996). For example, a researcher may interview experts in a particular area to determine if the process of disseminating a program is appropriate for a particular group. These questions are used primarily for focus groups seeking information about program development as well as dissemination of information (Patton, 1980). An example of an Opinion Value Question is: 'what is your opinion about the program? Moreover, "Feelings Questions" assess emotional reactions that people have due to their experiences (Patton, 1980;Kvale, 1996). These questions can also evaluate individuals' thoughts about a particular matter, but not necessarily their experiences. Patton (1980) mentioned the latter component is often confused with Opinion value questions due to confusion about feeling content and opinion content. Patton (1980)  the words "feel" or "feeling" in the questions as a cue to the interviewee to help illicit the appropriate response (Patton, 1980). Accordingly, the same question that assesses a person's feelings about the program would be worded as follow: "How do you feel about [this program]"?
Lastly, Knowledge Questions are used to collect information rather than opinions or feelings (Patton, 1980;Kvale, 1996). According to Kvale (1996), the interviewees or participants are assumed to have certain knowledge about the topic of interest. For example, a researcher may conduct an interview with faculty members of a particular program at a university about the application and approval process for admission. According to Kvale (1996)), while we may question whether the information provided is actually factual and not the individuals' beliefs or perceptions, however, in this case the faculty members are aware that the information being collected must be based on facts. Moreover, the success of an interview also depends on the clarity of questions. He added that clarity anchors the interview purpose and conveys to the interviewee the manner in which questions should be answered (Kvale, 1996). According to Seidman (1991), the absence of clarity and confusion occur when the researcher aims to assess more than one topic in a given question. The following is an example of such a question: 'Tell me about your experience in the program and why you dropped out of the program after one month?' Accordingly, the respondent may choose to respond to the second part of the question first, leaving on topic unanswered. Seidman (1991) recommends asking singular questions to promote clarity.
According to Seidman (1991), confusion also occurs when the researcher uses terms that are unfamiliar to the respondent. For example, he states that a researcher assessing an individual's experiences in a particular program should know the term(s) the individual used to refer to the program to prevent confusion. For example, in a computerized-tailored program for PA, participants interact with a computerized character referred to as a "PA agent" who provides them with suggestions on different ways to engage in physical activity as well as creative ways to include fruits and vegetables in their diet. Thus a researcher assessing the participants' experiences and feelings about the computerized character 'telling them how to engage in PA' should use the term "PA agent" rather than the term "avatar," which is also used to refer to computerized characters.
Another component to improve clarity in qualitative interviewing is to avoid "why" and "what" questions (Patton, 1980(Patton, & 2002Seidman, 1991). Patton (1980) mentions that "Why questions, especially, " go beyond what has happened; what one has experienced; how one feels about …and what one knows to the making of analytical deductive inferences," (p.228). Because there could be numerous factors in a person's decision to why he or she participated in a PA program, providing a particular answer might be difficult. Patton (1980) states that avoiding asking "why" and "what" questions lessens the likelihood of "I don't know," or "I don't really know" responses. Clarity also helps with building rapport between the researcher and the interviewee (Patton, 1980(Patton, & 2002. Patton (1980) and Seidman (1991) added that the interviewer establishes rapport by conveying to the individual the value of his or her experience, feeling, emotions and opinions in fostering a deeper understanding of the particular topic. As a result, the individual is more likely to feel at ease and he or she is more likely to be more forthcoming about his or her experiences, feelings and emotions.

Potential Challenges with Qualitative Interviewing
As mentioned, the quality of data gathered from the interview depends largely on the interviewer. However, Patton (1980) states that the characteristics of the interviewer may also impact the quality of the data that are collected. Patton (2002) found that numerous researchers reported conducting interviews wherein the interviewee made it near impossible to gather valuable information regarding the topic of interest. He explains that some researchers reported having interviewees who provided only short answers to questions that were intended to illicit detailed responses. Kvale 1996 mentioned that researchers can find themselves questioning the credibility of responses or found the responses to be ambiguous and or inconsistent. Lincoln and Guba (1985) and Denzin and Lincon (2000) recommend supplementing the questions with the use of elaboration probes to alert the interviewee to provide more information to a particular subject. Elaboration probes can be communicated to the interviewee in several ways. The researcher can slightly nod his or her head, provide verbal quiet "uh-huh," or use a direct verbal form. These statements remind the interviewee of the importance of remaining consistent with the interview topic, conveying that the interviewer is also interested in other things the interviewee has to say.
To date, the area of health psychology continues to remain overwhelmingly quantitative. Chow et al. (2010)'s "study on HIV client's satisfaction" show that human behavior can be too complex to be studied and understood through a singular paradigm. Thus, qualitative research should be used particularly in areas wherein knowledge is lacking or to advance our understanding about human behavior and mechanisms of behavior change. The systematic integration of these two paradigms in intervention research would eventually provide a more comprehensive picture of human experiences, behaviors and thus better inform strategies for implementing behavior change.

METHODOLOGY
This current study's primary aim sought to explore participants' levels of satisfaction, when using a relational agent to promote PA and sun protection behaviors within the context of an intervention. The secondary aim investigated whether any levels of satisfaction within the intervention varied on account of race and gender.
Findings were intended to improve relational agent interventions in health research.
Research Questions: Assessing participants' experiences and levels of engagement produce the following research questions: 1) What was the experience of individuals who participate in a relational agent intervention for physical activity and sun protection? 2) What were the reasons for participating in the program? 3) What expectations did individuals have before participating in the program? 4) What were the levels of satisfaction among individuals? 5) Did the levels of satisfaction vary for demographic subgroups (e.g. male and female; Black and White)? 6) What promoted consistent and increased level of participation among individuals? 7) What prevented individuals from participating more? 8) What was the level of trust individuals had in the information provided? 9) How did the individuals' participation in the program influence their future behavior? Lastly, participants were asked to report their levels of satisfaction with the intervention based on a scale of 1 to 10, where 10 is the "most satisfied" and 1 is "least satisfied." Mixed Methods Research: This study used a mixed method design to explore its research questions. Mixed method design consists of both qualitative and quantitative approaches (Johnson & Onwuegbuzie, 2004;Tashakkori & Teddlie, 1998). A mixed method approach best serves this study's qualitative leanings, as it allows for consideration of variability and complexity of subjective experience.
Moreover, the systematic integration of the two research paradigms qualitative and quantitative is likely to provide additional understandings into an individual's experiences, producing results that are more comprehensive. The use of quantitative data, for example, on participants' levels of satisfaction can be used to further assess the qualitative data found within a differing component of the intervention. Though this study was executed through a qualitative lens when coding, analyzing and interpreting data, and designing the interview guide, these two research paradigms have real potential to inform each other.
Two distinct approaches were used in the assessment of participants' experience: 1) Participants' subjective evaluation of the intervention, and 2) a comparison of participants' expectations and subsequent evaluations of the program.
Philosophical Orientation: Respective philosophical orientations exist for both qualitative, quantitative and mixed research alike (Johnson & Onwuegbuzie, 2004).
This study warrants a pragmatic orientation, especially when considering potential hindrances on engagement. Engagement in PA, for example, may likely be hindered by variant, 'everyday issues,' which may in turn effect levels of engagement in the program (Doucet, 2009). Moreover, because this research aims to assess information related to experiences and levels of satisfaction among participants, a single-domain model, such as a positivist approach, was insufficient in capturing potentially different experiences and levels of satisfaction. According to Biesta (2010), gaining in depth knowledge of individuals' experiences involves "the combination of action [i.e., conducting the interview] and reflection" (Biesta, 2010, p. 112). This approach had implications for understanding individuals' actions or behaviors involving environmental and social contexts. In regards to PA engagement, individuals may report not engaging in PA, due to environmental factors, such as availability of sidewalks in their neighborhoods or the social and time constraints of family responsibility. Sun protection behaviors are likely to be hindered by resistance, like those who experience daily exposure to the sun, from working outside, for instance.
While sun exposure produces widely known and accepted risks for skin cancer, there are other elements, which may produce resistance, such as racial demographics and the cultural beliefs within them. African American participants, for example, are less likely to engage in sun protection behaviors, nurturing a cultural belief that skin melanin levels make this immune from cancer risks (Biesta, 2010).
Trustworthiness is a central component of qualitative research and is used to ensure rigor and enhance credibility of findings, for example (Seidman, 1991).
According to Wengraf (2002), a researcher establishes trustworthiness by implementing several verification strategies. These strategies include prolonged engagement, member checking, reflexive journaling, dependability audit and triangulation. Prolonged engagement indicates the researcher has been immersed in the data over an extended period, one in which interviews, transcriptions, code or node development are conducted. This researcher conducted all of the interviews over four and a half months. The transcription of interviews, as well as code development, were completed, respectively, over a four and half months and one month. Member checking indicates the researcher assesses the reliability of information by asking the interviewee for clarification or encouragment, reminding the interviewee that his or her opinion is valuable. During the interview, this researcher restated and summarized information, asking participants to determine accuracy. Participants were asked for clarification, if the interpretation was incorrect. Additionaly, participants were encouraged to report their "true" and "honest" opinions regarding their experiences with the intervention. They were also informed that their opinions were valuable.
Reflexive journaling allows the researcher to document the process from original ventures to the "arrival of final conclusions" (Bazeley & Jackson, 2013, p.30).
This researcher recorded ideas and strategies in the development of codes and response categories and subcategories. Dependability audit consists of informal consultation of experts in relevant fields, espeically those in the decision-making process regarding code development and methodological approaches. This researcher consulted with faculty throughout the research process, giving special attention to those that examined all decisions made from this study's inception to analysis and ultimate interpretation of results. Lastly, methodological triangulation involves the incorporation of both quanitative and qualitative data for the assessment of particular component or phenomenon. In this study, quantitative analysis used to further assess qualitative data on participants' level of satisfaction.

Interview Structure: A standardized semi-structured open-ended interview
was used for data collection. Relevant literature informed interview questions and were compiled in collaboration with other researchers involved in this study's respective questions of 'how' versus 'why.' These questions have been shown to illicit deeper and or more detailed information and minimizes potential effects of having differential wording and or presentation of a certain topic or issues on participants' responses (Patton, 1980(Patton, & 2002Seidman, 1991). Patton (1980) states that "by controlling and standardizing the open-ended interview the evaluator obtains data that are systematic and thorough for each respondent but that reduce" p. 1). Consequently, this approach facilitates data analysis, including comparison of patterns across individuals (Patton, 1980).
Participants: A total sample of 1364 participants was recruited at baseline. The majority of participants were White (n=1162) and women (n=861). The mode age for participants was 49 years old. Preliminary findings showed the following sample size for the three groups: 1) a control group, n=451, 2) a multiple risk internet group, n=465, and 3) multiple risk internet with relational agent, n=448. Two different samples were recruited from project RAISE to participate in the study. The samples were selected based on the four relational agents 1) Black male relational agent or health agent, 2) Black female relational agent or health agent, 3) White male relational agent and 4) White male relational agent or health agent). Two grouping variables were used to recruit individuals suitable for the study: group 1 consisted of participants who completed the 12-month intervention and or the 12-24month survey, and group 2 consisted participants who accessed the intervention only two or three times. For the first group, the goal was to administer 10 interviews for each target subgroup: 1) Black men, 2) Black women, 3) White men, and 4) White women. The second group was a purposive sample of ten participants (5 men and 5 women) who accessed the program two to three times.
At the time for the current study, a total sample of 216 participants completed the 12-month long intervention and or the 12 to 24-month survey for PA and sun For group1, a sample of 100 individuals was contacted over a month period, via telephone. Of that sample, 8 participants (Whites=7, Black=1) completed the interview. Three individuals refused the invitation to participate in the study. The remaining eighty-nine participants in group 1 were contacted three more times, after the initial phone call over a one-month period. Most participants were not reachable.
Each unsuccessful attempt at establishing telephone contact time was paired with a voicemail that explained the nature of the call. A return contact number was also provided. Those individuals reached by telephone made an appointment to complete the interview another time. Subsequent telephone calls for these individuals were not answered. Individuals informed of the study via a telephone message did not return the calls to verify whether they would participate in the study.
For group 2, a sample of thirty individuals were contacted over the course of two months. There were no interviews completed within group 2. Similarly to group 1, after an initial phone call, individuals were contacted three or more times over the course of a month. Nearly all participants remained unavailable. A telephone message was left after each attempt to establish contact, explaining the reason for the call, and a return contact number was provided. Those individuals reached by telephone made an appointment to complete the interview another time, also. Subsequent telephone calls were also not answered. Individuals informed of the study via a telephone message did not return the calls to verify whether they would participate in the study.
To increase the sample size, an incentive of a $15 gift card was used. Letters of invitation, which included information on the incentive, were mailed to all participants who did not complete interviews in group 1 (n=89). For group 2, a total of fifty participants, including the thirty-five unavailable participants, were contacted.
One letter was sent to group 1 (see Appendix B), while another was sent to group 2 (see Appendix C). Thirty-five letters were mailed at a time. Prospective participants were contacted a week after the letters were mailed. Seven were returned due to an "incorrect address" and relocated individuals (group 1, n=5 and group 2, n=2).
Ultimately, five participants from group 1 refused to participate in the study, while one participant was deceased. For group 2, five participants refused to participate. Over a three-month period, thirty-four interviews were conducted for group 1, and eight interviews for group 2. A retention diagram illustrates the recruitment process for both groups.  Table 2 provides detailed descriptive information.

Procedures
The Internal Review Board at the University of Rhode Island approved all procedures utilized in this research.
Confidentiality: All participants signed a consent form as a part of Project RAISE (see Appendix D). In the consent form, the principal investigator, Wayne F.
Velicer, Ph.D., indicated to participants that they would be asked to participate in several telephone surveys during the next two years. This study was included as part of these telephones surveys. The original consent form was revised and included this researcher's primary advisor. Each individual was asked if he or she would like a copy of the revised consent. Participants were also informed of their confidentiality and rights in this study as well. This researcher read the following statement to participants after they were contacted: All information is strictly confidential, for research purposes only. Your name and other personal information will not be shared with anyone other than the members of the research team. You may refuse to answer any or all questions. Refusals will not affect you relationship with the University of Rhode Island. All records for this project will be handled according to Federal Guidelines and Rhode Island Law on confidentiality of healthcare information.

Data Collection and Measures
Demographic Data: Archival data from the Cancer Prevention Research Center's Survey Center provided valuable information pertaining to gender, age, race, and stages of change. Group assignment and levels of participation (participants who completed the intervention and or the 12-24 month survey) were provided by the center.

Stages of change:
This measure assesses an individual's readiness to engage in PA and sun protection behaviors. Stages include the Precontemplation Stage, which included participants who were not consistently engaging in PA or protecting themselves from the sun, along with not intending to begin within the next 12 months.
The Contemplation Stage consisted of participants not consistently engaging in PA or protective behaviors yet were seriously considering doing so within the next 12 months. The Preparation Stage included individuals not currently engaging PA or protection, but planned to start within the next 30 days. Lastly, participants in the Maintenance Stage consisted of individuals who have engaged in PA or sun protection behaviors regularly over the past six months.
Interview Scripts: The two interview scripts for group 1 and group 2, respectively, consisted of twenty-five questions and ten questions that assessed different components of the participants' experiences with the program as well their opinions for improving similar and future interventions (see Appendix E for group 1 and Appendix F for group 2). The two interview scripts used a semi-structured approach to address the research questions. The interview scripts were developed and revised in collaboration with other researchers involved in this study: Ginette G. Social Desirability: The potential for participants' answers to be influenced by a socially desirable response set produces concern, whenever individuals are asked to report their feelings and attitudes. More specifically, social desirability bias is most likely to occur whenever individuals are asked to respond to socially sensitive or socially controversial issues, such as attitudes toward race, gender, sexuality or religion (van de Mortel, 2008;King & Bruner, 2000). Additionally, individuals are more likely to provide socially desirable answers, when asked to share subjective feelings and attitudes in-person, due to perceived pressure to conform or the fear that one may become disliked. This research assessed participants' experiences in an intervention for PA and sun protection, which is neither a socially sensitive nor controversial, thus reducing the risk for responses and participation with a potential for producing socially desirable outcomes. Moreover, all interviews were conducted over the telephone, not in-person, further ensuring a reduced likelihood of socially desirable responses.
Storing of Data: Interviews were recorded using two separate recording devices. Each file was downloaded and labeled with a participant's ID number, acquired through the parent study, Project RAISE. Files were stored on a computer and in a password-protected folder. Files were additionally placed on a portable hard drive that was stored in a locked cabinet. Interviews were transcribed verbatim, over a period of two months. Each transcribed interview was reviewed, along with the original audio file to ensure an accurate transcription. Transcribed interviews contained participants' ID number, rather than their names and were stored separately from demographic information. Only researchers and personnel involved in this project had access to these files.

Analytical Procedures
Qualitative: The compiled qualitative data was analyzed, using the qualitative data analysis software NVivo 10. This software enables content analysis through the assessment of relationships in the text and allows the researcher to annotate or mark sections of text(s) and specific items for analysis. Manifest content analysis and latent content analysis were used to assess participants' responses. Manifest content analysis was used primarily to assess elements (and frequency of elements) present and countable in the physical data, such as particular wording (Miles & Huberman, 1994).
Latent content analysis was used for interpretive analysis of meaning, underlying the physical data (Miles & Huberman, 1994). This approach enabled an exploration of the context in which certain word choice was exhibited, helping to further analyze meaning. For example, a participant describing his initial interaction with the relational agent or the relational agentagent as "different" required the coding of larger sections of his response. This provided a larger contextual framework against which to study words, such as "different." Content analysis allowed this researcher to organize the data by use of categories. A node, or a category, is a reference containing all information pertaining to a specific concept category, such as a location, person, or other area of interest or a theme 5 (Bazeley & Jackson, 2013). A node can consist of different child nodes also 5 It is important to note the difference between a category and a theme given that the terms continue to be used interchangeably, and thus incorrectly, in the literature. A category is a used for the organization and management of data in NVivo. In the current study, different sections of each interview were coded under single words, complete sentences, paragraphs and larger sections of the interviews. Larger sections within interviews were coded to provide context or clarification. A node model was used to organize data with overarching categories, as well as different subcategories for comparison (see Figure  For group 2, the components were assembled as 1) Reasons for Participation,

2) Satisfaction or Dissatisfaction with the Intervention, 3) Relational Agents, 4)
Reaction to Tailored Feedback, 5) Reasons for not completing the Intervention, 6) Any Impact from the Intervention, and 7) Suggestions for Improvement. Additionally, reference in which contains similar data that help identify or describe the characteristics of a category. On the hand, a theme describes a "meaningful essence that runs through the data" (Morse, 2008, p.27). Therefore, a theme can be conceptualized as a consistent factor(s) that helps explain a particular phenomenon. For example, Mama et al., (2011) found that a number of Latina and African American women were apprehensive about engagement in regular PA or healthy eating because they were concerned of "losing" their curves or posteriors which they viewed as essential parts of their ethnic and feminine identity. participants in both groups were asked to provide "additional information" that was not covered by the interview questions. Some of the overarching categories consisted of two to seven subcategories, which assessed varying aspects of a given intervention component. For group 1, categories 2, 3, and 4 consisted of 2 to 7 questions. Each question represented a subcategory of the overarching category. Categories 1, 5, 6, 7, 8, 9 and 10 were assessed using a singular question.
A number of participants were not asked all compiled interview questions, due to their responses to a previous question(s). For example, the Relational Agent category consisted of four questions. One question asked for a participant's feelings as to whether the relational agent cared about the participants PA or sun protection behaviors, while another asked what it was like interacting with a relational agent, while another asked. In an interview where a participant's response to the second example indicated that he disliked the relational agent and suggested the removal of that component in future and similar studies saw no need to ask additional questions.
The former example of a question asked became irrelevant and therefore was not asked. Therefore, every category (or subcategory) of responses was not comprised of thirty-four responses for Group 1 and eight responses for group 2.
Quantitative: An analysis of variance analysis was conducted to determine whether levels of satisfaction differed between men and women. Data for this analysis was based on a numerical value for level of satisfaction using a scale of 1 to 10, with 1 representing least satisfied and 10 representing most satisfied. The small sample of Black participants prevented an investigation into levels of satisfaction by race. The quantitative software statistical SPSS 20 was used for this analysis.

Organization of results
Responses are organized into different areas for each category. The number of participants who endorsed a particular category of responses are also provided. The results for group 1 are provided first, and the results for group 2 are provided second. The results for level of satisfaction (or dissatisfaction) with the intervention between men and women are provided third and the result for Stages of Change for PA and sun protection are provided last.

Reasons for Participation
Participants were asked about their Reasons for Participation ( WM). Responses in the "other" category were: "Simply because I was asked" (interview 586372; BM), "I didn't participate for any specific health reasons" (interview 587027; WM), and "I'm in the medical field" (interview 654459; WM).

Motivation and Engagement with the Intervention
Motivation. Several questions were used to assess motivation and engagement as well as specific components that promoted these behaviors: 1) What motivated you to keep using the program? 2) What was your favorite part of

Satisfaction or Dissatisfaction with the Intervention
Similarly to the motivation and engagement category, satisfaction (as well as  Two participants stated they "did not have any expectations" of the program. Two other participants mentioned they were dissatisfied with the program and reported the following reasons: "I think it just took a lot longer than I thought it was going to" Participants' levels of satisfaction, from a scale of 1 to 10, ranged from 7 to 10 (n=32). The following are five exemplars of impressions with the intervention as well as levels of satisfaction:

Relational Agents
Interacting with a Relational Agent. Participants' interaction with the relational agent was assessed based on seven questions. For question 1, participants were asked: What was it like for you to interact with a relational agent? Thirty-two participants reported a positive interaction with the relational agent. A number of participants described their interaction with the agent as: "user-friendly" (interview 587027), "entertaining" (interview 615633), "interactive" (interview 652440), "easy and effortless" (interview 652207), and "informative" (interview 626033) and the relational agent as a "virtual pal" (interview 627122). One participant mentioned the relational agent "…made me feel better about myself"(interview 651438; WM). Moreover, another participant described his interaction with the agent like "… watching a cartoon" (interview 653644; WM). However, his responses below demonstrate that a positive interaction with the relational agent, thereby suggesting the word "cartoon" was his only frame of reference for an animated and or a computerized character: Three participants mentioned that they disliked the relational agent or their interactions with the relational agent. Specifically, one participant mentioned that he preferred the "...interaction with a human being" (interview 655948). Another participant recalled several technological issues with the program that led to an aversion to the program and the relational agent (interview 626637).

Mannerisms.
Participants were asked whether they felt the mannerisms of the relational agent were similar to those of a human being (Did you feel her mannerisms, for example facial gestures or hand gestures, were similar to those of a human being?).
Thirty participants recalled his or her mannerisms were similar to those of a human being. The following response exemplars described three participants' assessments of the agent's mannerisms: Three other participants described the agent's mannerisms as "too unrealistic" (n=1), and "robotic" (n=2). Another participant mentioned that he disliked the agent because of his aversion toward animation: "I personally am not a super animated person" (interview 657274). Lastly, one participant recalled that the mannerisms of the agent were "somewhat in between robotic and similar to a human being" (interview 652440; WM).
Humor. Participants were asked if they found the jokes that the relational agent shared with them throughout the course of the intervention to be humorous (During the program, the relational agent shared many jokes with you. I was wondering if you found them to be humorous). Eleven participants recalled that the jokes were humorous. Eight participants recalled that "some of the jokes" were humorous.
Thirteen participants could not recall whether the jokes were humorous. Two participants recalled the jokes were humorless (n=1) or "off-handed" (n=1).

Interaction with a Relational Agent over Time.
Participants were asked about their feelings toward the relational agent throughout the course of the intervention (Did your feelings change over time or remain the same?). Twenty-six participants recalled their positive feelings toward the agent remained the same throughout the intervention. Six other participants mentioned their feelings improved over time as they became more familiar with the agent and or with the routine of the program. One of those participants recalled that the agent felt "like a friend" overtime (interview 586627; WM). Two exemplars of the responses are: Twelve participants provided various suggestions for improvement in effort to make the relational agent and or the interaction more interesting or appealing. The suggestions included: improving the "monotone" voice of the agent to a normal pitch (n=4), providing the options to choose a relational agent of the opposite gender, race and age groups (n=3). Other suggestions were: adding natural backgrounds or "outdoor scenes" (n=3) along with the relational agent, establishing a wider database of responses to keep information or feedback from becoming repetitive (n=2); and displaying information and graphs on the same page with the relational agent as well as having the agent explain the graphs to participants (n=1). One participant suggested that researchers replace the relational agent with a real person (n=1).

Other Intervention Components
Participants were asked whether there were other things they liked about the program (Were there other things you liked about the program?). Fourteen participants reported they liked the health information on the benefits of sun protection and PA. Of those participants, five recalled being interested in the information on the importance of sun protection and skin cancer prevention. Other participants mentioned the program and or the relational agent served as a constant "motivator" to become or to remain healthy (n=14). Moreover, four other participants recalled that they liked the "ease" of the program. Three of those participants stated the following: "it really was easy to download" (interview 652687), "it really was easy to use" (interview 653244) and "I could go at own pace" (interview 615633). Other participants reported that they liked the "email reminders" (n=2), the "graphs" (n=1), the "workbook" (n=1) and the "reports" (n=1). Lastly, two other participants mentioned they could not recall.

Behavior Change
Participants were asked about the ways in which the program helped them change their PA and sun protection behaviors (How has the program been helpful in changing your PA and sun protection behaviors?). Eleven participants reported that the program helped them to engage and maintain regular PA. Two response exemplars are: Moreover, four other participants stated the program served as a reinforcement for continuing with own routines for PA. Two participants mentioned that they became more aware about the importance of sun protection for cancer prevention as a result of participating in the intervention. Lastly, two participants reported that their participation in the program did not lead to change in neither PA nor sun protection behaviors.

Component(s) Most Attributed to Change
Participants were asked about the component(s) of the intervention they attributed most to their behavior change (n=26), becoming more aware of the benefits of sun protection and reducing cancer risk (n=2) or as a reinforcement for their own PA routines (n=4) (Was it the character, which was the relational agent, the reports, the workbook or the tracking charts?) 7 . Twenty-two participants attributed their 7 It is important to note that this question was worded originally as the following: Which part of the program was most helpful in getting you to change. Was it the character, which is the relational agent, the reports, the workbook or the tracking charts? The wording indicates to participants that could only choose one component of the intervention. The words "and or" behavior change to the relational agent. Ten participants mentioned that the combination of all aforementioned components were equally helpful in changing their PA and sun protection behaviors: The tracking charts (n=4), the reports and the relational agent (n=3), the reports and the tracking chart (n=1), the reports (n=1), and, the relational agent and the workbook (n=1).

Reaction to Tailored Feedback
Participants were asked if the tailored feedback they received throughout the intervention, which included suggestions for increasing and maintaining regular PA and sun protection behaviors were in fact helpful ( One participant mentioned he was aware of the suggestions for PA and sun were included after five interviews after two of the participants mentioned only one component help them to change their exercise and sun protection behaviors. As result, it is possible that the other three participants may have attributed their behavior change to more than one components of the intervention. protection prior to the intervention. Another participant was unable to recall whether he ever received any feedback.

Doubting of Feedback.
All participants were asked if they ever doubted any of the feedback or statements that were given to them (Were there times when you doubted the truth of the statements that were given to you? For instance, you may have heard/read that support from others is extremely helpful to meet your healthy lifestyle goals. This may not be true for you because you find that working alone helps you achieve your goals more quickly. If so, can you recall the statements that you doubted and explain why they were doubtful for you?). All of the thirty-four participants reported they never doubted any of the statements or feedback that were given to them. Two participants explained their reasons for finding the information credible:

Current Access
Participants were asked whether they would prefer current or continued access to the program. (Do you wish you still had access to the program?). Eighteen participants reported they would like to have current access to the program. Of that sample, six participants mentioned that the program would continue to serve as a motivator or as a "good prod" (n=2) for maintaining regular PA and sun protection behaviors or serve as a reference for "reviews" or suggestions for PA and sun protection (n=4). Two exemplars of responses were: Four other participants mentioned they were unsure whether they would like current access to the program. Two of the participants provided the following responses:

Suggestions for Improvement
Participants were asked to report their suggestions for improving the overall

Additional Information
Participants were asked if they would like to provide any additional information that was covered by any of the interview questions (Is there is anything you would like to add that perhaps I did ask about?). Thirty-three participants did not provide any additional information. One participant asked whether the results of the study could be shared with participants.

Reasons for Participation
Participants were asked about their reasons for participating in the program (Can you tell me some reasons why decided to participate in the program?). The eight participants reported the following reasons: helping with research (n=2), cancer prevention (n=2), motivation for PA and weight loss (n=2), a curiosity about the program (n=1), and one participant mentioned simply "because you asked me to" (n=1).

Satisfaction or Dissatisfaction with the Intervention
Participants were asked whether they were satisfied or unsatisfied with the intervention (Would you say that the program met your expectations? One participant mentioned she did not have any expectations of the program. Participants were asked about their satisfaction or impression with the program Participants were asked whether they would recommend the program to a family member or a friend (If a family member or friend were in a similar situation, would you recommend they participate in this program? Tell me some reasons why?). Four participants reported "no" to recommending the program to a friend or a family member. Two participants explained their reasons: Three participants mentioned they would recommend the program to a family member or a friend (interviews 639055, 640167 & 655904). They mentioned, overall, that the program could help others to engage in PA and also in sun protection behaviors.

Relational Agents
Interacting with a Relational Agent. Participants were asked about their interaction or reaction to the relational agent (how was it like to interact with a relational agent?). Six participants mentioned they disliked the relational agent. Their responses were: Interview 563478: I did not like that at all and I quit doing it (WF).

Interview 626599:
The exact problem was that it was a robot, not a real person (WF).

Interview 638910:
It was just so cold and impersonal (WF). Of the two other participants, one reported that she not remember her interaction with the relational agent (interview 652209). Another participant mentioned that he disliked the relational agent at first, "but the second time I thought it was really cool that I was not being bossed around" (interview 655904).

Mannerisms.
Participants were asked to assess the mannerisms of the relational agent (Did you feel that his or her mannerisms were similar to those of a human being?). Six participants answered this question. Three participants found the mannerisms of the agent to be dissimilar to those of a human being. Two participants described the relational agent as a "robot" (interview 626599) and "awkward" (interview 639055). Another participant suggested that researchers use "real people" instead of the relational agent.
Humor. Because participants in group 2 accessed the program 2 to 3 times, they were not asked to assess the humor component of the relational agent. This component is shared with participants after a few more sessions into the program.

Any Impact from the Intervention
Participants were asked whether the program had any impact on their PA and sun protection behaviors (Has the program had any impact on your exercise and sun protection behaviors?). Four participants reported an impact on their PA and sun protection behaviors even with their brief participation in the program:

Reasons for Not Completing the Intervention
Participants were asked about the reasons they did not complete the intervention (Can you please tell me what prevented you from using the program more than you did? Was there anything in particular about the program (e.g., the agent? The reports? The workbooks? The tracking charts?). Four participants reported their dislike for the relational agent was their reason for not completing the intervention.
One of the four participants mentioned that his questions to the agent were not "answered thoughtfully" (interview 563478). Three other participants repeated the same responses in the satisfaction and dissatisfaction subcategory: "computer issues" (interview 651470), "work schedule" (interview 652209), and software incompatibility with computer (655904) prevented them from completing the intervention. One participant reported that health issues prevented him from completing the program (interview 640167).

Suggestions for Improvement
All eight participants were asked about any suggestions for improving the program, including the relational agent (Do you have suggestions on ways to improve the program?). Seven participants answered this question. Three participants mentioned the program did not need to be improved (interviews 652209, 655904 & 640167). Three participants provided the following suggestions: Another participants stated that any improvement to the program would not improve his interest in the program: Interview 651470: Not really. I just don't like talking to computers, so I don't think I would enjoy it anyway (WF).

Satisfaction and Gender for Group 1
Level of satisfaction between men and women was investigated using a single analysis of variance (ANOVA) for group 1. The sample consisted of eighteen male participants w (M=7.78, SD=1.58) and sixteen female participants (M=7.92, SD=1.81).
The results showed no statistically significant effect for gender in levels of satisfaction with the intervention (F(1, 33)=.054, eta-squared=.002, p-value=.512). The sample size for group 2 was too small for quantitative analysis.

Stages of Change
Results for Stages of Change for PA for group 1showed most participants were in the Preparation stage (n=23). Conversely, for sun protection, most participants were in the Precontemplation stage (n=23). For group 2, most participants were in the Preparation stage (n=6) for PA. For sun protection, the stages were: Precontemplation (n=3), Contemplation (n=2) and Preparation (n=3). Tables 4 to 7 provide detailed information for Stages of Change for PA and sun protection for each group.

DISCUSSION
This study sought to assess the experience of participants in a relational agent intervention for physical PA (PA) and sun protection, using primarily a qualitative approach. The interview scripts consisted of interview questions that examined participants' experiences with different components of the intervention. The categories of responses found in this study were consistent with factors examined in the development of interview questions. In contrast, the codes or nodes used in this study were not determined a priority, but rather emerged from the data.

Group 1 Categories
Reasons for participation showed, overall, participants were interested in behavior change that would lead to health improvement, the topics of PA and sun results. If a participant did not have a favorite part of the intervention, he or she was likely to respond "no" to either way this question was worded.
The email reminder helped most participants (n=23) to remain on-track with the program, with most of the sample accessing the program on the same day. These findings indicate that periodic reminders motivated individuals to remain engaged in the program. This is particularly important for longitudinal studies that inherently require long-term participation and are more prone to inconsistency in engagement and eventually attrition. There is empirical evidence that periodic prompts motivate individuals to help them change their health behaviors (Fry & Neff, 2009). One participant recalled this component also served as a "reminder" to PA and use sunscreen regularly (interview 652687). It is possible this component also served as a motivator for participants to engage in and maintain regular PA and sun protection behaviors. In this study, participants were not asked whether email reminders helped them maintain regular PA and sun protection behaviors. Given the potential of this component to promote behavior change, researchers may find it helpful to ask participants directly whether email reminders helped them remain on-track or get back on-track with these behaviors.
Regarding Satisfaction or Dissatisfaction within the program, almost all participants reported being satisfied. Satisfaction, in general, was noted as an outcome of met expectations and positive impression. Most participants described the program as "informative," "motivational," "accessible," "positive," "interactive," and "supportive." Levels of satisfaction ranged between 7 and 10. Engagement in healthy behaviors, especially PA, has been shown to be a difficult behavior to acquire and maintain. Most people beginning to PA often fail to maintain such behavior over time, due to lack of support and motivation. The current findings show the program provided participants with support, as well as motivation to engage in PA and sun protection behaviors.
The relational agent was well received by participants. Participants described the agent, overall, as "interactive," "entertaining," "informative," and even a "virtual pal." A number of participants mentioned their feelings toward the agent had improved over time, indicating that some individuals may need time to become familiar or acclimated with this component. Thus, researchers should consider assessing participants' experiences with the relational agent, after a length of time has passed within the intervention.
The relational agent, in general, was comparable to a human health counselor.
Participants stated the agent was informative about the importance of PA and sun protection and served as a motivator to help them engage and maintain these behaviors. Two participants mentioned preferring the relational agent over of a human counselor, due to his or her supportive nature. Most participants felt the agent cared about their PA and sun protection behaviors. However, a number of participants, including participants reporting a positive impression of the agent, also found the agent was, at times, limited in answering some of their questions thoughtfully. This indicates that a broader database of tailored messages might be helpful with this task.
Participants were unable to recall the specific questions, due to length of time between the intervention and the interview. To help assess this information, researchers should consider asking participants to assess the quality of responses from the agent at some point during the intervention. Furthermore, they wish to identify frequent questions not answered thoughtfully. This information can provide a better understanding of additional factors for tailoring.
This category also revealed a number of individuals adamant that such a comparison could not be made. They did not explain their reasoning. These individuals reported a positive interaction with the agent and were satisfied with the program. This reaction did not appear to impact behavior change. It is likely these participants also felt the agent did not accurately respond to some of their questions.
That these individuals viewed the agent as informative and effective in the promotion of PA and sun protection, but still viewed it as a computerized character, can serve as an alternative hypothesis.
Regarding the mannerisms of relational agents, most participants mentioned their mannerisms were similar to those of a human being (n=30). These findings suggest the agent's speech, gaze, hand gesture, intonation and other nonverbal modalities imitate the experience of human face-to-face conversation. Humor is important for developing a sense of connection between the relational agent and users.
Most participants found the jokes shared by agents to be humorous. The mannerisms and humor of the agent played important roles in building relationships with users, by displaying human-like reactions and keeping the process fresh. It is possible this component also made the agent more relatable and the program more interesting, as well as exciting. In future studies, researchers should consider asking participants whether the humor component made the agent more relatable and familiar.
Suggestions for Improvement indicated that in general the agent was well developed. Four participants reported his or her voice as monotone, suggesting that variations in tone should be used to replicate that of a human being. Other suggestions were provided for the improvement of the overall program, including options to choose a relational agent of the opposite gender and to have agents representative of different races and different age groups (n=3). The addition to natural backgrounds or "outdoor scenes," along with the relational agent (n=2), displaying information and graphs on the same page as the relational agent, along with having the agent explain the graphs to participants (n=1), were also suggested for improvement.
The improvement of the agent's voice tone to include a normal or varied pitch could produce a voice more similar with that of a human. Choosing the gender, race, and age of one's agent could make a participant more relatable to users. For example, older participants may feel more motivated and or inspired by an older relational agent, as compared to being matched to a younger agent. The placement of graphs, along with having the agent explain the information, may help improve understanding of information or make the learning process more interactive. Lastly, the use of natural backgrounds could make the program more visually appealing.
The Other Intervention Components category did not provide additional information about the participants' experiences with the program. Participants simply responded similarly along the lines of "I was interested in PA," or "I like the sun protection information," to previous categories, such as the Reasons for Participation, Motivation and Engagement category and Satisfaction category and the Impression of the Program subcategory.
Regarding behavior change, almost all participants reported change in PA and or sun protection (n=26). A number of participants reporting change in sun protection recalled being unaware of the cancer risks attributed to prolonged and unprotected sun exposure prior to their involvement in the program. Four participants reported behavior changes in the PA. Responses also mentioned finding some of the sun protection recommendations, such as wearing sunscreen before sun exposure or t-shirt in the sun, to be unrealistic or irrelevant to their lifestyle. A number of participants seemed to doubt the risk of UV-related skin cancer, due to a lifetime of unprotected sun exposure and being skin cancer-free. For example, a White participant who described his complexion as an "olive" tone mentioned that he was not at risk of developing skin cancer because he does not possess "fair skin." One of the three African American participants in this study mentioned being confused about the sun protection intervention because he is not of a "light or fair" complexion. These findings are concerning, given the prevalence of skin cancer, especially melanoma and non-melanoma, associated with prolonged and unprotected sun exposure among all groups. While African Americans have lower rates of skin cancer in comparison to their White counterparts, they are more likely to die from the disease (Fegal et al., 2013). Unfortunately, this researcher did not inform participants of the reality of increasing their cancer risks with continued unprotected and prolonged sun exposure, in order to adhere to the standardization of the interview guide. In retrospect, this researcher feels that participants should have been provided relevant health information at the end of the interview. Future studies with a similar approach should consider incorporating education that address frequent misconceptions about skin tone and UV-related cancer risk as part of the interview guide.
The relational agent was shown to be the primary component of behavior change. These findings indicate the effectiveness of the relational agent, as an important catalyst for behavior change and a subsequently valuable tool for health research. Moreover, a number of participants attributed their behavior change in PA and or sun protection to a combination of the relational agent, the reports, the workbook and the tracking chart. The current study also demonstrated other components of the program as having effective tools for the promotion of PA and sun protection. Altogether, these findings provide further evidence of the effectiveness of the RAISE program in promoting change in this group.
Reaction to Tailored Feedback showed that participants found the feedback and or recommendations for engagement in PA and sun protection to be effective.
However, three participants mentioned the recommendation for PA was irrelevant to their lifestyles, which could be attributed to the wording of this question. As a part of this question, participants were given the example of a tailored feedback for PA they may have received during the program: For instance, you may have heard/read that support from others is extremely helpful to meet your healthy lifestyle goals. As a result, you may have begun working out with a friend to help achieve your goals. Did you find the suggestions helpful? It is possible their responses were based solely on the singular message for PA. This question is somewhat leading and may have impacted their responses. In hindsight, this researcher thinks this question should have served only as a potential probe to participants who had difficulty answering or remembering messages or feedback for PA. Additionally, this researcher believes the assessment of tailored feedback should been have been conducted using two separate questions; one question that assessed the tailored feedback or messages for PA and one question that assessed the tailored feedback or messages for sun protection. For the Doubting of Feedback or Statement subcategory, all participants, including two participants who did not report behavior change, found the feedback or recommendations to be credible or valid.
Regarding Current Access to the program category, most participants would like continued access to the program. A number of participants mentioned the program would continue to serve as a motivator or as a guide to review suggestions for PA and or sun protection (n=18). By contrast, twelve participants mentioned that current access to the program would not provide additional benefits. One participant stated: "I got my whole routine so I really wouldn't need it" (interview 651438). For those participants seeking current access to the program, it may be helpful if future studies include a website on information or suggestions for PA for six months after the intervention. However, this option might be costly and thus impractical for most health programs. Another option is to allow or encourage participants to save or print information, including feedback and recommendations, throughout the intervention.
For the Suggestion for Change, the four suggestions that were provided could make such programs more accessible and more appealing to users: These were: 1) disseminating the program via cellular phone, 2) allowing users to stop anywhere in a session and to continue from that section at a later time, 3) shortening the length of surveys and, 4) suggestions for PA for individuals with physical limitations. The accessibility of the program via cell phone would allow participants to access program anywhere and at their convenience and potentially help increase and or maintain engagement. The ability for participants to stop at a section of a session and continue from there a later time would provide participants with more flexibility to complete the program 8 . Moreover, the use of shorter surveys might prevent participants from feeling overburdened. However, the shortening of surveys could impact the psychometrics of constructs for these behaviors, hinder data collection and, thus, tailoring. Suggestions for PA for people with physical limitations is not feasible, as participants are required to be physically healthy to participate in this study. It is important to note that this participant suffered a medical complication that prevented her from engaging in PA during the intervention.

Group 2
The Reasons for Participation category showed, similarly to group 1, participants were also interested in behavior change (interest in health improvement), in the topics of PA and sun protection (interest or curiosity about the program) and helping with research (help with research). The Satisfaction or Dissatisfaction with Intervention category showed that fifty percent of participants were satisfied with the program. Participants who reported a positive impression of the program also reported high levels of satisfaction with the program (ranging of values of 7 and 10). In addition, participants mentioned they would recommend the program to either friends or family members. Conversely, half of the participants who were dissatisfied with the program, due to difficulties with the software after accessing the program 2 to 3 times, reported un-met expectations and a negative impression of the program. Their levels of satisfaction ranged from 0 to 4. Although the satisfaction scale was not a ratio scale, and thus did not contain a zero value, one participant insisted on that value. The participants who reportedly experienced difficulties with the software mentioned receiving assistance from research personnel at the Cancer Prevention Research Center, but continued experiencing the same issues shortly thereafter. Not surprisingly, those participants reported "no" to recommending the program to either a friend or a family member.
Most participants in this group did not like the relational agent. A number of participants (n=6) described the mannerisms of the agent as "robotic" and "awkward." By contrast, two participants reportedly liked the agent and enjoyed their brief interactions with the health agent. They described his or her mannerisms as similar to those of a human being. These participants also felt the agent cared about their PA and sun protection behaviors.
The Any Impact from the Intervention category shows four participants reported increase in PA and or sun protection behaviors, due to the health information they received during their brief participation in the program.
One participant, who was ill during the duration of the intervention, reported regular engagement in regular PA and a weight loss of 108lbs at the time of the interview (interview 640167). Another participant mentioned that she became more aware of the importance of PA and sun protection, due to involvement in the program.
Lastly, the Reasons for Not Completing the Intervention category show that participants' responses were similar to those in the Satisfaction or Dissatisfaction with the Intervention and Interaction with Relational Agent categories. Specifically, four participants who reportedly disliked the agent reported it as the reason they stopped using the program. Because these participants used the program only 2 to 3 times, it is possible they needed more time to become more familiar with this component. A number of participants in group 1 stated that their feelings towards the agent improved over time. The two participants who experienced software problems also reported them as the reasons for not completing the program. These factors were the same for the Suggestions for Change category.

Satisfaction and Gender
Similar levels of satisfaction with the program were found across both men and women. This is essentially positive news, as it suggests that similar factors may underline levels of satisfaction in PA and sun promotion program across gender.
However, the lack of statistically significant differences in gender might be due to the small sample in this study, as the total sample size was 34. It is unclear whether a larger sample size would have yielded different levels of satisfaction with the program between the two groups. A larger sample is needed to determine whether a gender difference exists.

Stages of Change
Most participants were in the preparation Stage of Change PA. Conversely, most participants in group were in the Precontemplation stage of change for sun protection. It is unclear what underlines the difference in the stages of change between these two behaviors. One possible reason is that participants might not have knowledge of the relationship between sun exposure and skin cancer risk. However, the current study used baseline information for Stages of Change. Therefore, it is unclear if similar results remained after participants completed the intervention.
Participants in group 2 were in the Preparation stage for PA, at baseline. This information suggests they were planning to start engaging in PA within the next 30 days. Their dissatisfaction with the intervention likely underlines their attrition from the intervention and lack of behavior change.

CONCLUSION
Relational agents continue to be utilized within health research. The current study demonstrates agents to be interactive, supportive, motivating, and able to promote behavior change in PA and sun protection. A number of participants described the agent as a "virtual friend" or a "buddy." Most participants identified their relational agents as the intervention component most associated their behavior change. The current study additionally demonstrates other components of the RAISE program, such as reports, workbook and tracking charts were vital in a number of participants for changing their behavior. Taken together, these findings demonstrate an intervention that was interactive, supportive, motivating, and effective in promoting PA and sun protection behaviors for completers in group 1.
The current study further emphasizes the importance of maintaining a qualitative approach, while assessing participants' experiences with the relational agent interventions. Specifically, this approach has shown to be effective in assessing participants' feelings, emotions, and opinions concerning the intervention process and provided opportunity to field suggestions for improving this model. The insights qualitative methods produce cannot be stated enough and have real potential to benefit the development and dissemination of health programs. Solely relying on quantitative approaches is inadequate, when attempting to complete this task. Although qualitative methods have gained unprecedented popularity in the social sciences, health psychology, especially, remains predominantly quantitative. Among its scientific and health-based aims, this study also hopes to communicate the benefits of employing qualitative methods within its field. The current study results suggest that qualitative methods, or the triangulation of qualitative and quantitative methods, could help develop and establish important factors that relate to motivation, engagement, satisfaction and behavior change in other health-related behaviors.

Limitations
Sample Size. This study encountered a number of limitations. The sample size of Black participants was too small to conduct an in-depth assessment of this group's experience with the intervention. Thus, it is unclear whether Black participants would have reported similarly positive experiences with the Black relational agents in comparison to their White counterparts in group 1. Moreover, the small sample size for this group prevented the assessment of gender in Black participants' experiences with the agents.
Timelines. The length of time between the actual intervention and the current study may have influenced participants' ability to recall experiences of paramount components within their participation. While the intervention was distinctive, in that it included a relational agent, in that individuals will be more likely remember to an intervention compared to a standard or non-relational agent intervention, the completion of interviews soon after could have provided more details on participants' experiences.
Sampling Method. This study used purposive sampling, consisting of individuals who completed the intervention and or the surveys, for group 1. Thus, the study results may not be representative of most participant's experiences within the intervention. It is possible that participants who were dissatisfied with intervention disregarded the invitation to participate in this study.
Qualitative analysis. There are inherent limitations of the qualitative analysis used in this study. Either content analysis or manifest content analysis is subjective and can be influenced by the researcher's biases. The subjectivity of content analysis can lead to incorrect interpretation of data, as it is influenced by a researcher's skills and idiosyncrasies.
Notwithstanding these limitations, the current study makes substantial contributions to the understanding of participants' experiences with relational agent interventions. Moreover, the current study provides further evidence on the effectiveness of this type of intervention in promoting PA and sun protection behaviors. Additionally, this study underscores the importance of qualitative methodology in the advancement of health research, including behavior change. The process of behavior change is complex and thus qualitative approaches can be beneficial to the establishment of knowledge regarding participants' experiences with these interventions, as well as behavior change.

Future Directions
Future studies should include a larger sample of Black participants. The inclusion of Black participants may help assess a range of potential racial and cultural experiences, which may arise within this type of intervention, including their interaction with the relational agents. Moreover, potential differences between men and women could also provide useful findings.

APPENDIX A.
Relational Agents In Project RAISE APPENDIX B.

Dear
We are inviting you to participate in a research study that assesses your experience with PA and sun protection in Project RAISE. Your participation can provide valuable information to make our program better. First, the Project Raise staff would like to thank you for taking part in this important research program. Your participation would involve a single telephone interview that will be conducted by doctoral student Marie Sillice. Ms. Sillice will call you within the next week for this interview. The interview completion time is at least 20 minutes. You will receive a $15 Wal-Mart gift card via post mail after completing the entire interview.
If you have any questions or concerns about the study, please feel free to contact Marie Sillice at the URI Cancer Prevention Research Center at 401-874-9064. By taking part in this study, you are contributing to important research that may help others in your community. Thank you in advance for helping us. We are contacting you to invite to participate in a research study that assesses your experience with PA and sun protection in Project RAISE. First, the Project Raise staff would like to thank you for taking part in this important research program. Even though, you accessed the program two to three times, your participation can provide valuable information to make our program better. Your participation will take place via a single telephone interview that will be conducted by doctoral student Marie Sillice. Ms. Sillice will call you within the next week for this interview. The interview completion time is at least 10 minutes. You will receive a $15 Wal-Mart gift card via post mail after completing the entire interview. Consent Form for Research You have been invited to take part in a research project described below funded by the National Institutes of Health (NIH Study). The researcher will explain the project to you in detail. You should feel free to ask questions. If you have any more questions later, Dr. Velicer, (phone: in Rhode Island: (800) 555-2854; outside Rhode Island: (800) 777-3537), and Dr. Patricia Morokoff, (phone: in Rhode Island: (401) 874-2685), and they will discuss them with you. You must be at least 18 years old to be in this research project.

Description of the project:
The purpose of the NIH Study is to learn more about ways to help people change unhealthy behavior and to evaluate different ways to help people change unhealthy sun protection behaviors and sedentary lifestyles.

What will be done:
If you decide to take part in this study, here is what will happen: You will be asked to participate in several phone surveys during the next two years. You will be randomly assigned (like tossing a coin) to one of three research study groups. The number of phone surveys will depend on the group you are assigned to; at a minimum you will be surveyed again at 12 and 24 months after the initial survey. The maximum number of surveys you will be asked to complete is six. Each survey will take approximately 45 minutes. You may or may not be offered participation in a web-based program that is designed to help you change your health behavior. If you are in this group you will be required to access the internet and the specified web site in order to get the information that is especially tailored to your needs. If you are assigned to this group, you will be asked to complete a minimum of three interactions with the web site, each lasting approximately 45 minutes. You will be able to complete just part of a session and return later to complete the session. You can also access the website more than the minimum requirement. Entrance to the web site will be password protected. After two years, the study will be terminated and you will no longer receive survey questions, any materials and your password will expire.

Risks or discomfort:
This is a minimal risk study. There is a small risk of initiating an PA program in individuals who are previously sedentary. You might also experience discomfort or inconvenience associated with the surveys and with the eventual effort involved in participating in one of the web-based programs.

Expected benefits of study:
Participation in this research may help you make better decisions about your health. Even if there is no direct benefit to you for taking part in this study, your honest answers will provide valuable information in designing future health education programs, which may benefit others.

Confidentiality
Your participation in this study is strictly confidential. All data will be coded with a number and will be stored on password-protected computers, separated from your name. Only authorized researchers will have access to any identifying information. There will be no reports remaining that identify you as an individual project participant. Information linking to your name will not be released to anyone outside the research group.

Decision to quit at any time
The decision to take part in this study is up to you. You do not have to participate. If you decide to take part in this study, you may quit at any time. Whatever you decide will not penalize you in any way. If you wish to quit, simply inform Dr. Velicer (phone: in Rhode Island: (800) 555-2854; outside Rhode Island: (800) 777-3537) and or Dr. Patricia Morokoff: (401)-874-2685) of your decision.

Rights and Complaints
If you are not satisfied with the way this study is performed, you may discuss your complaints with Dr. Velicer (phone: in Rhode Island: (800)  You have read this Consent Form and have no further questions concerning your participation in this project at this time. You understand that you may ask any additional questions at any time and that your participation in this project is voluntary. By participating in the project, you agree that your answers can be used without your signed consent. This is the program that helps individuals deal with PA and sun protection. I will not be contacting again regarding your satisfaction in this program. However, you will continue with your participation in the program and someone else from the research team will be contacting you as usual. 12 What kinds of questions are included in this survey?
You will be asked questions about your views and experiences about the program. There are no right and wrong answers to these questions.

13
How will you protect my information?
All personal information provided in this call is strictly confidential. We adhere to strict federal and state guidelines to ensure that individuals' rights, confidentiality and privacy remain protected. I understand that you are a busy person. We are trying to get as many different viewpoints as possible including people like you with busy schedules. We would like to accurately represent a wide range of views.

Q1
Can you tell me some reasons why you decided to participate in this program?
For instance, some people participated because they wanted help to start exercising.

Q2
Would you say the program met your expectations?
If yes, can you give me some specific examples?
If no, can you tell some of the things you didn't like?

Q3
What was it like for you to interact with a relational agentagent?

Q4
Did you feel her mannerisms (e.g. facial gestures or hand gestures) were similar to those of a human being?

Q5
During the program, the relational agent shared many jokes with you? I was wondering if you found them to be humourous.

Q6
Do you feel that she really cared about your exercise and sun protection behaviors?

Q7
Throughout your interaction with the agent, did your feelings change over time?
If yes, tell me about something that happened or something she said that made you feel this way (Follow up: Can you recall when you first felt this way?)

Q8
How would you compare her (or him) to human health counselor?

Q9
Is there anything about the relational agentagent that you would change?

OR
Can you suggest anything changes about the agent that would make her more appealing?

Q10
Were there other things you liked about the program?
Follow up: Was there anything in particular that you especially liked?

Q11
What motivated you to keep using the program?
Follow-up: Was there something you looked forward to?

Q12
What was your favorite part of the program?

Q13
Tell me some things about the program you didn't like as much?

Q14
As a part of the program, you received some feedback about ways to increase engagement regular physical activity. For instance, you may have heard/read that support from others is extremely helpful to meet your healthy lifestyle goals. As a result, you may have begun working out with a friend to help achieve your goals.
Did you find the suggestions helpful?

Q15
Were there times when you doubted the truth of the statements that were given to you? For instance, you may have heard/read that support from others is extremely helpful to meet your healthy lifestyle goals. This may not be true for you because you find that working alone helps you achieve your goals more quickly. Can you recall the statements that you doubted and explain why they were doubtful for you?

Q16
How has the program been helpful in changing your behavior?
Follow up Qx: What part of the program was most helpful in getting you to change?
(e.g. the agent, the reports, the workbooks, and or the tracking charts?) Q17 As a part of this program, you also received email reminders to access the program if you did not log on for a while. Did this help with keeping you on tract?
Follow up: Did you log on to the program right after receiving the email?
If no, how long after receiving the email did you log on?

Q18
Can you give some examples of ways in which the program has not been as helpful in changing your behavior?

Q19
Tell me ways in which things have stayed the same or gotten worse Q20 I am wondering if there is anything that you would like us to do differently. Are there some things you would like to see more or less of?

Q21
Do you wished you still had access to the program?

Q22
If a family member or friend were in a similar situation, would you recommend they participate in this program? Tell me some reasons why?
Q23 Describe your overall satisfaction or impression with the program?

Q24
On a scale of 1 to 10 where 10 is the most satisfied and 1 is the least satisfied, how would you rate your level of satisfaction with the program?

Q25
Is there anything you would like to add that perhaps I did not ask about?
We have completed the interview. I want to thank you so much for giving us your time today. We truly respect and appreciate the time that you have put forth to provide the feedback that will allow researchers to develop programs that will help people to live healthier. Once again, I thank you for your time.

APPENDIX F
Interview Script for Group 2 Relational Agent for Sun Protection and PA Hello, my name is Marie Sillice and I am calling from the Cancer Prevention Research Center at the University of Rhode Island to speak with you about your participation in Project RAISE. This is the program that you participated in within in the past year that deals with PA and sun protection. The reason I am calling is to talk with you about your experience with the program for a research study. Even though, you accessed the program a few times, your participation can provide valuable information for this research study. I would like to take a few minutes of your time to ask you some questions about your experience with the PA and sun protection program. You will receive a $15 Wal-Mart gift card via post mail after completing the entire interview. The call should take at least 10 minutes to get through all the questions. Does this sound okay?

Confidentiality Statement:
All information is strictly confidential, for research purposes only. Your name and other personal information will not be shared with anyone other than the members of the research team. You may refuse to answer any or all questions. Refusals will not affect you relationship with the University of Rhode Island. All records for this project will be handled according to Federal Guidelines and Rhode Island Law on confidentiality of healthcare information.
If no: Would there be a better time I could call and ask you a few questions? Record time to call back.
If yes: We have received valuable information about improving the program from people like you who have participated twice or three times in the program. So with your help, we would like to continue making improvements. If no: Clarify any concerns they may have regarding their participation.
If yes: I would like to record our conversation so I won't miss anything we talk about. All the information we discuss will be confidential and it will not affect your participation in this program or future participation in any other programs. All of your personal information such as your name and contact information will not be attached to your responses. Does this sound okay?
If no: Inquire as to reasoning for not wanting to continue. Clarify purposes of the call using the Question and Answer guide. If another time is better, record time and call. Thank them for continuing to participate in the program.
If yes: Do you have any questions or concerns before we begin today?
If yes: Answer anticipated questions using the Question and Answer Guide.
If no: Proceed with the questions listed below. We are in fact interested in the reasons you only used the program twice (or three times). Your experiences and your feedback can help with ways we can improve the program.
2 Do I get paid for participating?
Yes, you will receive a $15 gift card for completing the entire interview. Your participation would be helping future participants like yourself if you gave your input today.
3 Why are you calling me?
I am calling you to get a better idea of your experiences with the program and your level of satisfaction. We think you can provide valuable information to make our program better.

4
How long is this going to take?
It usually takes about 10 minutes or less.
5 Who are you again?
My name is Marie Sillice and I am a research assistant for Project RAISE. 6 What program is this again?
This program is called Project RAISE. The purpose of our program is to learn more about ways to promote health and reduce cancer risks 7 I don't remember giving out my information to be in this program.
This is the program that helps individuals deal with PA and sun protection. That's very helpful and that's exactly what type of information we would like to hear.
11 Will you be calling me again?
I will not be contacting you again regarding your participation in the program.

12
What kinds of questions are included in this survey?
You will be asked a few short questions about your views regarding the program. There are no right and wrong answers to these questions.

13
How will you protect my information?
All personal information provided in this call is strictly confidential. We adhere to strict federal and state guidelines to ensure that individuals' rights, confidentiality and privacy remain protected.
14 Who can I contact to verify this study? Who is in charge? The information you share during this call is completely confidential. Your responses are stored without any identifying information such as your name, address, etc. In this way, it is then grouped with other data and looked at this way.

16
Who are the members of the research team?
Primarily, they are research faculty at the Cancer Prevention Research Center at the University of Rhode Island who have their PhD's in health psychology.

17
Do I have to go anywhere for this interview?
No, this interview is going to be held over the phone.
I'm too busy. I don't have the time.
I understand that you are a busy person. We are trying to get as many different viewpoints as possible including people like you with busy schedules. We would like to accurately represent a wide range of views.

Interview Questions
Questions Q1 Can you tell me some reasons why you decided to participate in this programs?
For instance, some people participated because they wanted help to start exercising and protecting their skin from damages due to sun exposure Q2 What was it like for you to interact with a relational agent?
(Prompt:… in comparison to working with a personal trainer)

Q3
Did you feel her mannerisms (e.g., facial gestures or hand gestures) were similar to those of an actual person?

Q4
During the program, the relational agent shared some jokes with you? I was wondering if you found them humorous.

Q5
Do you feel that the agent really cared about your overall health?

Q6
Can you please tell me what prevented you from using the program more than you did?
Was there anything in particular about the program (e.g., The agent? The reports? The workbooks? The tracking charts?)

Q7
Do you have suggestions on ways to improve the program?
(e.g., the agent, the reports, the workbooks, and or the tracking charts?)

Q8
On a scale of 1 to 10 where 10 is the most satisfied and 1 is the least satisfied, how would you rate your level of satisfaction with the program?

Q9
Is there anything you would like to add that perhaps I did not ask about?

Q10
Would you recommend this program to a family member or a friend?
We have completed the interview. I want to thank you so much for giving us your time today. We truly respect and appreciate the time that you have put forth to provide the feedback that will allow researchers to develop programs that will help people to live healthier. Once again, I thank you for your time.

Other (n=4)
Interview 586372: Simply because I was asked.
Interview 587027: I think I got an e-mail [phone call] about it and it seemed like an interesting study.
Interview 653983: I don't remember because it was a long time ago.
Interview 654459: I'm in the medical field.