Benefits Associated With Science, Technology, Engineering, and Mathematics Learning

Innovation and discovery are important for driving the advancement of human society. As a result, there has been a large growth in science, technology, engineering, and mathematics (STEM) employment opportunities. Science and technology have become a strong prerequisite for successful integration into higher academics and the labor force of the United States. The increase in demand for STEM positions necessitates motivating future workers from schools in kindergarten through 12th grade. A metaphor of a pipeline has been used to describe the retention of students through the completion of an academic program in STEM fields. Several science organizations have created the Next Generation Science Standards (NGSS) to improve Science learning in K-12. However, it is of interest to see how the NGSS are being taught in the field and what impact this may have on students' performance. This study investigated academic achievement differences among fourth and eighth-grade students between schools that are affiliated with a University in New England compared to matched schools that are not. Schools were matched by using ratios of students that receive free and reduced lunch. Testing results from the New England Common Assessment Program (NECAP) for physical science, earth/space science, life science, and inquiry were specifically investigated. A comparison of t-tests examining differences between the schools yielded one significant difference for Inquiry scores on an academic achievement test t(35) = 0.01, p = 0.98, Hedge’s g = 0.003. Specifically, the university-affiliated students performed higher (M=47.2, SD=10.4) in 2015 compared to the students not affiliated (M=42.3, SD=10.2). The other investigated content areas were not significantly different between the university-affiliated schools and the non-affiliated schools. Interestingly, the adoption of NGSS by the university program may explain the differences observed, as the NGSS supports student inquiry skills. Teachers from school sites used in the quantitative analyzed were interviewed for potential differences in areas of support, style of teaching, STEM integration, knowledge of NGSS standards, and attitudes towards creativity. Common practices by other schools not within the network may not be supporting student inquiry skills to the same level. A qualitative review of teachers' familiarity with NGSS through transcribed interviews supported this finding. Future research may followup on how these differences affect students' retention further down the pipeline at the high school and college level. It may also be of interest to investigate whether science knowledge or inquiry skills are better predictors of retention for students.


LIST OF TABLES
and mathematics (STEM) employment opportunities, which has surpassed other fields (Gonzalez & Kuenzi, 2012). Science and technology have become a stronger prerequisite for successful integration into higher academics and the labor force of the United States.
These opportunities require advanced knowledge and abilities for highly specialized positions as well as mid-level skills positions. The increase in demand for STEM positions necessitates motivating future workers from schools in kindergarten through 12 th grade and to improve the skill sets required to be competent in a STEM position.
Despite this large demand for filling STEM positions and the current unemployment rate, many STEM jobs remain unfilled (Pathways to Prosperity Project, 2011).
The United States has traditionally enjoyed a position of leadership in terms of innovation; however, there are signs that this position may be lost to rising competitors.
Students in the United States have been performing below other countries in terms of mathematics literacy (Gonzales et al., 2008). According to the U.S. 2011 National Assessment of Educational Progress (NAEP), 73% of students are not proficient in mathematics upon completing the eighth grade (National Center for Education Statistics, 2011). Additionally, results from the Programme for International Student Assessment (PISA, 2015) show that 15-year-old students from the U.S. rank 46 th out of 70 in science literacy and 31 st out of 70 in mathematics literacy. Considering these rankings, the current rate of unfilled STEM-related jobs is likely to remain unfilled without appropriate changes.

REVIEW OF LITERATURE
There is a current interest among educators and scientists to make changes in order to improve the competitiveness of the U.S. and its future economic prosperity (STEM Education Coalition, 2014). This trend is similar to a previous educational movement in the U.S. when education reform became a matter of national security during the post-Sputnik era and resulted in revising secondary-school science curricula (National Science Foundation, 1980). The current proponents of STEM education seek to revise current curricula and to integrate these domains rather than present them as separate entities.
Currently, science, technology, and mathematics have established representation in the curriculum of middle schools. One of the main limitations to the implementation of STEM is a lack of exposure to engineering during early education (Swift, Watkins, Swenson, Laseter, & Mitchell, 2003). Research on the understanding of engineering and technology concepts among young students is limited (Davis, Ginns, & McRobbie, 2002;Lachapelle & Cunningham, 2007). Additionally, particular groups of students have been observed to have low percentages of representation in STEM, especially women and minorities in terms of college enrollment (Ross et al., 2012;Swail, Redd, & Perna, 2003).
Women do not receive the same level of encouragement as their male counterparts and racially/ethnically underrepresented groups in science (i.e., Latinos, African Americans, and American Indians/Alaska Natives) often feel isolated and less likely to pursue career advancement (Burke & Mattis, 2007). Underrepresented racial/ethnic minorities and women show increased rates of dropout from STEM programs at the college level (Ross et al., 2012). According to Griffith (2010), one in five women and racially/ethnically minority students leave this area of study before completion. Therefore, it is especially important to motivate students from these under-represented groups at an early age by providing meaningful experiences in STEM areas.
The metaphor of a pipeline has been used to describe the retention of students through the completion of an academic program in STEM fields (Dischino, DeLaura, Donnelly, Massa, & Hanes, 2011;Epstein & Miller, 2011). The pipeline has been divided into sections to focus on particular strategies for ensuring a successful transition for students in STEM programs beginning with primary school, extending to secondary school, post-compulsory education, and ending at the maintenance of a career (Osborne, Williams, Tytler, & Cripps, 2008;Pekar, 2015). During the early phase of the pipeline when students are in primary school, students should be engaged with meaningful experiences to engage early interest, increase awareness of careers in STEM, and build self-efficacy (Osborne et al., 2008). These early experiences prepare students when they move through the pipeline and transition to post-compulsory education and beyond.
National science associations, including the National Science Teachers Association and the American Association for the Advancement of Science, have created standards to raise levels of both student and teacher performance and have contributed towards the Next Generation Science Standards (NGSS; NGSS Lead States 2013). These standards place more of an emphasis on deep understanding and developing scientific process skills for students K-12 and represent an integrated approach of many scientific disciplines to generate greater interest in science and engineering, improve learning of core content, and understanding of core concepts that underlie many areas of science (Stage, Asturias, Cheuk, Daro & Hampton, 2013). The NGSS standards have incorporated practices of scientists and engineers and have defined what is necessary for students to show competency.
Much of science education tends to focus on the acquisition of factual knowledge, through key terms and definitions. Methods for teaching these skills have been traditionally routed around text-based procedures although curricular reform tends to emphasize inductive thinking (Aulls & Shore, 2008). Information from children's science textbooks is an important resource for early science courses by providing coverage of many topics (Donovan & Smolkin, 2001;Ford, 2004) and textbooks provide exposure to vocabulary and academic language (Gee, 2004;Snow, 2010). Exposure to vocabulary, academic language, and models of science, are beneficial for learners, but incomplete by themselves. Experience with phenomena, making empirical inquiries, and examining data are also necessary experiences to help learners practice science-related process skills.
The traditional and dominant approach, which is also referred to as direct teaching, text-based, or transmissive learning, places a great deal of emphasis on basic skills, using the textbook as the primary source of learning, have learning based on repetition and memorization of a broad range of topics, placing the teacher in a role of authority to disseminate information to students, and obtain assessment of student progress through frequent standardized testing (Cawley, Foley, & Miller, 2003;Duschl, Schwengruber, & Shouse, 2007;Elliot, 1986;Schmidt, Raizen, Britton, Bianchi, & Wolfe, 1997). The textbook curriculum is the dominant model of science classes for kindergarten through sixth grade (Banilower et al., 2013). There are several limitations to the textbook-driven curriculum: There are minimal opportunities to learn and to practice process-related science skills, and the scope of content is fixed (Harris et al., 2015;Parmar & Cawley, 1993).
Dependency on textbooks is problematic because this approach fails to provide multiple representations (i.e., several variants of the material) and minimally addresses how students learn to think and behave like scientists. Although written and oral discourse are important in fields with an advanced vocabulary, an over-emphasis on new vocabulary can reduce the amount of time available for students to practice and understanding science content. Mastropieri and Scruggs (1992) examined two school textbooks and found 750 new vocabulary words in one and 1,831 in another. Another concern is that some science textbooks can have incorrect content or be poorly organized, which can disrupt understanding of students, even those with good grades (Champagme & Bunce, 1989). A textbook-driven approach contrasts with research on science teaching effectiveness (Darch & Carnine, 1986;Lovitt & Horton, 1994). Shymansky, Kyle, and Alport (1982) studied hands-on approaches to teaching science compared to textbookcentric courses and showed that hands-on classrooms were able to outperform on every measured criterion. Hands-on approaches to instruction tend to lead to better performances than textbook programs (Bredderman, 1982;Flick, 1993).
Inquiry-based curriculum (i.e., hands-on, constructivist, and exploration) tend to be more experiential in nature and allow students to learn in a manner based on their interests. Curriculum reform aimed at improving inquiry-based learning can be fraught with challenges, such as inadequate resources, time, or training (Songer, Lee, & Kam, 2002) and administrative and teacher support can improve these reforms (Fogleman, McNeill, & Krajcik, 2011). Research demonstrates that children taught with a guided hands-on approach outperform peers in textbook-oriented science courses (Alfieri, Brooks, Aldrich, & Tenenbaum, 2010;Bredderman, 1982;Furtak, Seidel, Iverson, & Briggs, 2012;Palincsar, Magnusson, Cutter, & Vincent, 2002;Varelas et al., 2014) and an increased student interest in STEM (Wright, White & Bates, 2015). Materials for science classes are important and a review by the National Science Foundation has found a small positive effect of inquiry-based curriculum materials for student learning (Shymansky, Kyle, & Alport, 1983;Shymansky, Hedges, & Woodworth, 1990). This trend has continued with more recent studies of inquiry-based science curriculum materials as well (Geier et al., 2008;Marx et al., 2004;Songer, Kelcey, & Gotwals, 2009;Wilson et al., 2010). Altogether, support for curriculum materials as well as inquirybased instruction makes a difference for science learning in children.
University Partnerships with school districts are often beneficial by providing support for teachers in order to improve students' experience with science and district level support for using best practices and research. Partnerships often use published kit curriculums to support STEM education in kindergarten through eighth grade, are aligned with Common Core State Standards and Next Generation Science Standards, and provide ongoing professional development for science teachers. These programs often have benefits for students' academic achievement in STEM. Research on the benefits of university partnerships relating to feelings of support, approaches towards teaching, familiarity with science standards and best practices, and scientific creativity, however, have yet to be further explored.
The present study contributes to the literature base on educational approaches to teaching science and associated outcomes with particular approaches. The findings may also identify outcomes that are associated with a stronger motivation to continue pursuing STEM fields in the high-school years and beyond.
Given the increasing relevance of technology towards both workplace and academic requirements and the impact of STEM on learning, it is prudent to examine the effectiveness of different models of science learning in terms of a child's scientific achievement, STEM interest, and scientific creativity.

Standards
Given the importance of science proficiency for students in terms of future careers, education goals, and informed citizenship, it is integral to monitor the process of scientific learning in schools. The current national science standards are the Next Generation Science Standards (NGSS), which examine how students understand how science is conducted in real-world settings and exposes them to disciplinary core ideas as well as cross-cutting ideas that permeate all the sciences (National Research Council, 2012;NGSS Lead States, 2013).

Attitudes Towards Science
In addition to student performance on standardized tests of achievement, teachers' attitudes towards STEM content is also of interest. It has been demonstrated that teachers' feelings of support from the school environment and members of the administration improve their attitudes towards reform and utilization of inquiry-based teaching practices (Supovitz, Mayer & Kahle, 200l;Supovitz & Turner, 2000).
Utilization of inquiry-based teaching practices, incorporation of NGSS standards, and integration of STEM content is expected to increase interest towards STEM among young students as well as increase academic proficiency, (National Research Council; 2001; President's Committee of Advisors on Science and Technology, 2010). These factors were investigated as well as teachers' attitudes towards students' scientific creativity.

Scientific Creativity
The process of science involves problem-solving, hypothesis generation, experimentation, and innovation; all of these processes require creativity (Lin, Hu, Addey, & Shen, 2003). Creativity is defined as a process of producing something novel and useful (Sternberg & Lubart, 1995). Torrance (1988) considered fluency, flexibility, and originality as key features to creativity, suggesting that they can be used to rate products of creative endeavors. Fluency refers to the number of original ideas produced.
Flexibility refers to the diversity of ideas or number of different kinds of ideas.
Originality refers to the number of unique ideas within a large set of ideas. There is a tendency for individuals to think of creativity as only existing within the arts. This pervasive association is known as the "art bias" (Glăveanu, 2011(Glăveanu, , 2014Runco, 1999Runco, , 2007 and can impact teachers' understanding of creativity beyond artistic domains (Zeljko, 2015). Creativity, as it relates to science, has been referred to as the process of recognizing gaps in a problem, creating and testing hypotheses to solve the problem, and sharing/presenting findings (Dass, 2004). This specialized form of creativity, that is, scientific creativity, goes beyond novel production of useful products by including sensitivity to the existence of problems. Being sensitive to problems is an important part of the problem-solving process (Hu & Adey, 2002).
Most research on scientific creativity in educational settings has investigated secondary-education students whereas less has been conducted among elementary-school students (Aktamis & Ergin;Hu & Adey, 2002). It has been identified that general creativity tends to decrease once children reach fourth grade (Plucker, 1999); one of the purported reasons for this trend has been investigated by Beghetto (2007).
Beghetto (2007) identified that classroom discussions provide students an opportunity to develop creative thinking skills, but that cultivating these skills would require attention and support from teachers' responses. Teachers may dismiss novel comments in order to refocus the direction of a conversation. Whereas relevance to a current topic is an important concern, too much emphasis on relevance can be problematic as students can become reluctant to share novel ideas. Beghetto proposed that a balance between rewarding originality and relevance should be sought by teachers.
The results of this study indicated that the prospective teachers had a greater preference for relevant responses and less preference for unique responses and that this pattern was stronger among those who planned to work in math or at the secondary level as unique responses were viewed as potential distractions. The teachers with a high preference for unique responses viewed them as a good starting point for students to continue exploring (Beghetto, 2007).
This research project improves awareness of the effectiveness of different kinds of science learning in terms of performance on national standards of science assessment.
Although scientific academic-achievement standards routinely are tested, usually beginning at the fourth grade, it is of interest to investigate whether different kinds of teaching instruction (e.g., integrative vs textbook-based) would lead to greater academic achievement. Additionally, differences among teachers in terms of the support they receive may impact their teaching style, integration of materials, and interest in creativity.
The present study examined potential differences in science among middle-school children as a result of the style of teaching they receive. The study focused on science classes that use an integrated approach to teaching sciences compared to those that use a traditional text-based approach. The intent was to extend the research on student gains associated with different approaches to teaching science and different attitudes towards teaching STEM content.
This study addressed the following research questions:

Sources of Data/Participants
The current study used existing publicly available data on standardized academic achievement at the fourth and eighth-grade level. Existing data on the New England Common Assessment Program (NECAP) for Science for fourth and eighth graders were used to compare differences in science achievement and inquiry. These scores were compared between 40 schools that were affiliated with a university program and 40 matched schools that were not affiliated with a university program. Of these 40 schools, 29 were elementary schools and 11 were middle schools. Schools were matched based on similar ratios of economically disadvantaged students to all other students at that school site. This information was provided for each school within the NECAP school level report.
For the interview part of the study, participants consisted of 10 teachers, five from schools that were affiliated with a STEM program at a university and five from nonaffiliated schools. The number of participants was determined by referring to an article on best practices for sample size within qualitative studies by Baker, Edwards, and Doidge (2012). Selection of participants was based on whether the teachers had been in their school setting for one year and had taught science, technology, engineering, or mathematics content.

Measures
The NECAP was used to assess scientific academic achievement. Specifically, the questions concerning support and teaching style were developed based on feedback and literature on constructivist teaching (Alfieri, et al., 2010). Questions concerning STEM integration were based on prior research by Lederman and Niess (1997) whereas questions regarding creativity were developed based on research on teachers attitudes towards creativity in the classroom by Cropley (1992). Questions regarding the NGSS were developed as a result of the findings from the quantitative portion of this study. This methodology of the semi-structured interview was chosen for the openness and flexibility it allows during the interview process (Kvale, 1996).

Procedure
Data on scientific achievement was obtained from a Department of Education's website in New England for the results of the NECAP in order to examine scores from schools affiliated with a university program and schools outside of this network (http://reporting.measuredprogress.org/NECAPPublicRI/select.aspx).
A descriptive qualitative design was used in this study to analyze the information obtained from the interviews with teachers. The goal of qualitative descriptive research was to investigate new areas through an interview and to develop a comprehensive summary of the responses provided by the interviewees (Sandelowski, 2000). Once the information had been transcribed and summarized, a content analysis was used to examine similarities and differences among the data (Graneheim & Lundman, 2004). To goal of this study was to focus on the content directly provided during the interviews, also known as manifest content, which represents a surface-level interpretation (Graneheim & Lundman, 2004).
The first step to analyzing manifest content requires the researcher to become familiar with the transcriptions and to review interview responses to each question several times in order to generate a general impression of the data. From these impressions, responses that best exemplify the common patterns are identified and collected. These responses were coded and researchers compared and discussed all codes until 100% agreement had been reached with each code. Similarities and differences across interviews and codes were also analyzed, which is useful for this study's investigation of teacher's perceptions within a supportive network and those in a typical academic setting. This information was used to develop themes and specific quotations were chosen to illustrate the themes and the experiences of the teacher participants (White & Marsh, 2006).
The second step was to analyze differences in categories endorsed by the different groups of teachers. Specifically, the number of categories endorsed by each group was examined based on selected questions. Differences were examined for exploratory purposes.
Upon approval by the University of Rhode Island Institutional Review Board, interviews with 10 teachers were scheduled. Specifically, snowball and purposive sampling techniques were utilized for participant requirement based on stated eligibility criteria. Participants were recruited through an email solicitation (see Appendix C) as well as a follow-up phone call (see Appendix D). Recruitment occurred until five teachers affiliated with a university program and five nonaffiliated teachers who met the eligibility criteria agreed to participate. The first ten participants recruited for this study were five of each category and recruitment procedures were terminated afterward.
Arrangements for a location convenient to the interviewee were identified prior to interviews. Participants who taught STEM content and had been at their school for at least one year were asked to sign and return the consent form to the researcher.
Participants were informed of the benefits and risks, which were minimal, associated with participation as well as how their confidentiality was maintained through the completion of the study (see Appendix E). After obtaining consent, the audiotaped semi-structured interview commenced lasting no more than 45 minutes. Upon completion, participants were thanked for their involvement.
Once each interview was completed, it was transcribed verbatim. During this process, any identifiable information was removed and if necessary, pseudonyms were used to ensure the anonymity of the participant and school site. Each audiotape and transcription was reviewed multiple times to ensure the accuracy of information contained within. Following transcription, the information was coded for common themes (Braun & Clarke, 2006).

Data Analysis
Data were screened for univariate outliers, kurtosis, and skewness and analyzed using the SPSS package software. School-wide comparisons of scientific academic achievement were examined using paired samples t-tests between schools affiliated with university program and those outside of this network. The dependent measures were determined based on the presence of inter-correlations. In other words, if the measures of physical science, earth and space science, were sufficiently correlated, a composite measure of science literacy would be created. Scientific inquiry would form the second dependent measure for this study.
In order to address the data collected from the interview, a qualitative data analysis of common themes was conducted. For an in-depth description of the qualitative research process used for the second part of this study see Appendix F.

Ethical Considerations
Before beginning each interview, informed consent was obtained from each participant and all data were kept confidential and no responses were linked to any identifiable information. All data were password protected and stored on the researcher's computer and kept within a secure location. Only those directly involved in the study had access to the data.

Demographic and Quantitative Data
Forty university-affiliated schools were selected for an analysis of NECAP performance. The NECAP reports also provided descriptive information on the students that attended such as the number of students that received free and reduced lunch and labeled them as socio-economic status (SES)-disadvantaged students. The number of SES disadvantaged students and non-disadvantaged students from the 2014-2015 report was divided to create a ratio of disadvantage for each of the selected university-affiliated schools. The average ratio for all 40 of the university-affiliated schools was 0.65; meaning that there were 0.65 disadvantaged students for every one nondisadvantaged student in these schools. Forty schools outside of the university network were selected based on similar grade level and disadvantage ratio for matching purposes and the average ratio across these schools was 0.68. Finally, the ratios for each paired school cluster were compared to each other to ensure that they were similar enough (i.e., within a 4-1 ratio), this process yielded a grand average ratio of .526 ** .807 ** .720 ** Note. N = 48, **. Correlation is significant at the 0.01 level (2-tailed).  .334 ** .414 ** .379 ** Note. N = 80, **. Correlation is significant at the 0.01 level (2-tailed).
Several analyses were conducted on the six dependent measures to investigate possible curriculum based differences in science and inquiry achievement scores obtained from the NECAP. Preliminary analysis revealed that the six dependent measures had little skewness or kurtosis, reasonably satisfying the normality assumption (see Table 4).
Correlations among the six variables ranged from -.24 to .82, reducing the concern for multicollinearity (see Table 5). Note. **. Correlation is significant at the 0.01 level (2-tailed).
A series of paired t-tests were used to assess the differences in science comprehension and inquiry scores between matched university-affiliated schools and schools outside the network based on similar ratios of SES disadvantaged students. This statistical method was used based on a similar approach used by Lin et al., (2003)  year that differences were observed with university-affiliated schools having higher NECAP inquiry scores than the non-affiliated schools. Also, the NECAP Science knowledge scores for this year approached a significant difference with a small effect size (see Table 6 and 7 for sample descriptives for the dependent measures).  Note. **Significant at the 0.01 level (2-tailed).

Demographic and Contextual Data for Interviews
Teaching participants provided personal information regarding different aspects of their teaching career. Questions covered various areas including (a) their age, (b) their gender, (c) their race/ethnicity, (d) years at their current position, (e) grade level taught, (f) size of classroom, (g) highest education completed, (h) degree/major, (i) credentials, (j) grade levels taught previously, (k) and school district.
The average age of the ten participants was 47.8, ranging from a low of 30 to a high of 63. The majority of participants were female; only two of the ten participants were male. All teachers identified as being either White or Caucasian. In terms of years at their current position, the interviewed teachers had an average of 14.7 years, ranging from 2 to 28 years. The average number of years taught for the university-affiliated teachers was 9.2 and the non-affiliated teachers taught for an average of 20.2 years. Many teachers reported ranges of classroom size, 7 to 28 students, but the average of the maximum number of students reported by each teacher was 27.22. Participants reported a high variability in terms of degrees earned (see Table 8) what their college major was (see Table 9), and credentials earned (see Table 10). Finally, four of the participants taught grades 1-5, five of them taught grades 6-8, and one was not currently in a classroom, but rather worked with teachers throughout her district (see Table 11).

Research Questions
The results of the interviews are organized based on the five research questions.
For each research question, descriptive tables are provided to summarize the categories generated from each of the participant's responses. Narration has been used to present the findings; specifically, quotes are used to support the conclusions and interpretations made by the research (White & Marsh, 2006).

Research Question 1: What supportive factors have you encountered in your school district?
The first interview question asked participants to describe the factors that have contributed to their continued success with teaching. Participants' responses were organized into four categories, as illustrated in the following section (see Table 12). Provides the support 2 (20%) Note. Total is not equal to 10 (100%) given that some participants endorsed more than one response category. Miscellaneous. Three participants spoke specifically about the types of support they receive from their district. For example, one participant discussed the ease to which the district fulfills their request for curriculum related resources: As far as technology goes, we've actually been on the forefront of Rhode Island. I've been teaching for 26 years and I'd have to say 25 of those 26 years. As a matter of fact, in 2009 we were the first district in the state to go one to one. That was K-12. So Educational technology has always been apart of our culture and I've had students using google apps for education since 2009. This is part of our technology are just tools for the teachers and the students.
Provides the support. Two participants reported that they are the primary support for others in their district. For example, one participant stated the following: In addition to the first interview question about district level support, participants were also probed with nine further questions. The first follow-up question was "How would you describe the support that you receive from your site administration?" Table 13 represents the four categories that emerged.

Table 13
How would you describe the support that you receive from your site administration?
Categories n(%) Supportive, provides coverage for training, professional development, resources 4 (40%) Nominally supportive, but not really involved 4 (40%) Receives more support from district administration 3 (30%) Supportive/Accessible, provides support through direct involvement Note. Total is not equal to 10 (100%) given that some participants endorsed more than one response category.
Supportive, provides coverage for training, professional development, and resources. These individuals reported that they received a high level of support from their site administration in the form of coverage for professional development or training.
For example, one participant described the support received as follows: They have sent me to any trainings that I have found applicable to my profession.
They have provided me with three or four full days of professional development to deliver out content and training to my district teachers... I think that's it. I have an opportunity four times a year with each grade level for the district to have a couple hours to work on our curriculum with them and help them build their knowledge…Anything I want to do they let me go to (laugh). The principals would support by helping get subs.
Nominally supportive, but not really involved. Four participants described the support they received from their site administration as not very involved and only providing a nominal amount of support. One of the participants stated, "Site administration is not very involved and only provides a nominal amount of support." Receives more support from district administration. These individuals reported that they receive more support from their district administration than they do from their site administration. For example, one individual stated: I'm gonna have to call my site administrator, my assistant superintendent because we do not have a principal currently. So that support is big because she is the one that initiated the STEMscopes curriculum and she is 100% behind it. She is the one that made sure we had material, offers us support, gave us professional development on it. So she has really given us a lot of support for that.
Supportive/Accessible, provides support through direct involvement. Two individuals reported that they receive a high level of support from their directly involved and accessible site administration. For example, one participant described an outstanding amount of support from school principals who assisted in the classroom and coordinated a multi-grade family science night: They would also come into the classroom and observe science lessons and give us feedback, so I often invited the principals to come in and see what was happening in science. When I invited them in, they always would come to see what was happening in the classroom and then we would also do family night, during the day, so the principals often would help me get those together, so they would come to the classroom and work with the kids to prepare for the family nights and we would share science notebooks and we would do a culminating project that would be shared with families as well as we would partner with other grade levels and the principal would support us getting together with other grade levels to share our science.
The second probe asked participants about the support that they received from their district administration. Specifically, participants' responses fell into four categories (see Table 14).

Table 14
How would you describe the support that you receive from your district administration?
Categories n (%) Committed to professional development, supports new training/curriculum 7 (70%) Provides financial support 6 (60%) Mixed/Unsure/Low level of support 2 (20%) District and site administration are well aligned 1 (10%) Note. Total is not equal to 10 (100%) given that some participants endorsed more than one response category.

Committed to professional development, supports new training/curriculum.
Most of the participants reported that they received a high level of support from their district administration in the form of commitment to professional development, new training, and curriculum. Specifically, one individual described the commitment from the district as follows: Being part of [university-affiliated program], they committed to sending us to all that professional development, so they were very supportive in that way. We were released from the classroom while the professional development was done during the school day. Being released from the classroom as a new teacher, three and sometimes more, times per year.
Provides financial support. More than half of participants specifically mentioned the support they received from their district administration in the form of financial support. One individual shared "They would pay for the subs, so I feel they were very supportive." Another participant stated, "Or if they're not able to provide me with a sub during work hours, they will reimburse me for my time." Mixed/Unsure/Low level of support. Two participants reported that they received a low, unclear, or inconsistent level of support from their district administration.
One participant described the difficulties in the school district as follows, "Again, the STEM coordinator has been, you know-this is his first year going around. His first or second year. So, he's still feeling out the district…But other than that, there's really no other program focusing on STEM initiatives in this building... by the district." District and site administration are well aligned. This individual reported that their district and site administration are particularly well-aligned with each other.
Specifically, the respondent spoke about how the support from the district extended through the building principals and then to the staff "Well I think it extends from them, it starts there and then the principals in our buildings do what they want us to be doing. So that's also been good." The third probe asked participants the question "How is support provided in terms of kits, activities, and supplies?" Table 15 represents the three categories that emerged.

Table 15
How is support provided in terms of kits, activities, and supplies?

(40%)
Limitations in support identified 2 (20%) Note. Total is not equal to 10 (100%) given that some participants endorsed more than one response category.
High support from district. The majority of participants reported that the support for kits, activities, and supplies from their district was very positive. For example, one participant described the experience in the following quote: Yeah, they started a new program -which, the first year, they did it this year-but, Adopt A Scientist. They provided a list of topics that you could focus on and this past year we focused on global warming. So, we had a connection throughout the year as we were studying the human impact of global warming on the Earth. And the end of the year, in the spring, in May, the scientists actually came into the classroom and did a follow-through activity with the kids. So, it was kind of nice to see real-life applications based on what we had been studying all year.
High support from [university-affiliated program]. Nearly half of participants specifically mentioned the high level of support they received from [university-affiliated program] in terms of kits, activities, and supplies. For example, one participant shared her experience with [university-affiliated program]: Okay... so, we are partnered up with [university-affiliated program]. So, we get the bee's knees. I mean, there is no better support... When I first started teaching... when I first got my teaching job, ever... I was placed in first grade. The first day I walked in and there's eighteen little faces looking to me, on what to do. I mean, you're first at student teaching and everything like that... and you know, you start student teaching, you don't start on the first day of school. So, what the heck do you do with these kids on the first day, you know? The only area that I felt incredibly prepared to teach was science. And that was because of the training through [university-affiliated program]. The support, the kits that we get, are lengthy, they're rigorous, they're great. They have outdoors incorporated, so outdoor lessons. They have indoor lessons. They have a writing component. The new kits have a reading component. So, they really are really rich. But, they send us, any time you teach a new kit, or anytime you teach a new grade level, they send you to their workshops. And you'll take a day, and you'll go to the Bay campus, and you'll learn all about your kit, from not only the people from the [university-affiliated program] program but they also pull in some teacher-leaders. And I happen to be a teacher-leader for [university-affiliated program], so sometimes I'll even go to the workshop as a facilitator and help teach other teachers how to... you know, what to expect with the kit. And I think that is something that really differs from other workshops, or PD [professional development], or support things that we have in place because we as teachers learn best by seeing other good practices and talking with other teachers. And I think sometimes you will go to a workshop, or you go somewhere, and there is somebody up there, and they're not an educator, or if they are, they haven't been in many years in the classroom, and they're telling you about these ideas that are great... but when you put it in the classroom, you know, sometimes things come up that are unforeseen. So, having a teacher leader there at the workshops, for the trainings for the kits, they can tell you, "oh by the way, when we did this stuff, make sure you have..." even something as simple as "make sure you have an extra roll of paper towels on hand, because this is a water lesson and it's gonna get messy." You know, that's something that the manual might not say, that you get to hear at that training.
Limitations in support identified. Two participants described the limitations they received in the support that they receive in terms of kits, activities, and supplies. One participant noted the following limitation "Science kits are always supplied because I see the science teachers packing up their kits. But they are the only ones that I know of that have science kits… It's not really integrated with a math component or a science component." The fourth probe asked participants the question, "Does your district supply a science specialist? If so, how has this benefitted you?" Participants' answers fell into three categories, as illustrated in Table 16.

Does not have a science specialist, but identifies other source of support.
Three individuals reported that their district does not supply a science specialist, but they described another source of support. In the following quote, one participant described how her school makes use of teacher-leaders: Our district does not provide a science specialist... We do not. We have... teacherleaders in the building, which are kind of our go-to people. Like, that's not something that every building has. But if you happen to have a teacher-leader in your building, I would say that those buildings benefit a lot more.
Identifies as the science specialist for the district, describes benefits. Two individuals identified as science specialists and described the ways that their positions have benefited their districts. One individual shared how the role has been beneficial to the teachers in the district especially with all the recent curriculum changes: It's been an integral piece of our whole plan. Teachers, especially elementary school teachers, do not have the time to figure it out. Especially since the last few years, we've started new math, new reading, new writing, everything aligned to common core as well. So, they've had someone to lead and guide them for this entire process. Because of that, our test scores have gone up...significantly and our students are much more adept to teaching, learning science, and talking about science, and inquiry-based investigations.
The fifth probe asked participants the question, "Does your district supply a mathematics specialist? If so, how has this benefitted you?" Participants' answers fell into four categories, as illustrated in Table 17. Has a math specialist, receives direct support and benefits 2 (20%)

Note. n = 10
Does not have a math specialist, identifies other support. Three individuals reported that they do not have a math specialist, but identified a different form of support.
For example, one participant shared the experience of receiving support from a STEM coordinator: No. The STEM coordinator kind of is the mathematics specialist, if you... "specialist." We do have math coaches that do coach with teachers on how to deliver instruction. There is no curriculum since we've gone to the common core standards.

Has math specialist, received indirect/mixed/low support and benefits.
Three participants shared how they do have a mathematics specialist, but receive inconsistent or low levels of benefits. One participant shared how the mathematics specialist for the district is stretched across five different elementary schools and offered some advice on how a mathematics specialist might have a stronger impact: Yeah. So, we have a mathematics specialist in our district, but unfortunately, I feel as though this has not benefited us as much, because (a) it's only one person, and servicing K-5 in five different elementary schools, and you know, do all of the stuff that she has to do, it's kind of difficult... so I haven't seen, I haven't felt a lot of support from this person to no fault of her own, I think just because of the work load that's on her. Especially when we moved to common core. Then it's all new math, and everybody needed help, and here's one person trying to run around and give help to a lot of different people… She does an awesome job as much as she can, but you know, it can definitely be difficult just with the caseload that she has in front of her, and her being only one person. I would like to see either a grade-span math interventionalist, or math specialist, as a grade-span. Or ideally, one in each building. But that... forget about it (laughing). When it comes to paying, I understand they have to consider those things too, but I do think that there's room for improvement in that aspect.

Does not have a math specialist and does not identify another source of support.
Two individuals briefly shared how their district does not have a mathematics specialist and did not mention receiving support in this area. One participant responded "Not really.
It's more of a math resource. I actually don't touch the math as much. I do the science, technology, and engineering, and integrate the math, and analyzing data and stuff like that." Has a math specialist, receives direct support and benefits. Two individuals reported that their district did supply a mathematics specialist and they receive direct support and benefits from their services. One participant described the support that the math specialist has for the district: Yeah, we do. It has been tremendous. We also create our own math curriculum, K-12. And she is the lynchpin to all that. It has taken us 5 years to get it pretty comprehensive. So after all those units have been created, once again those are created with lab classrooms. She will go into 4th-grade classrooms, if they are studying fractions, she will go into that classroom once a week and teach some model lessons, they'll video tape those model lessons, she'll write up the lesson plan. The teacher that she is working within that model classroom will come up with supportive materials. All of this stuff is in a google doc and the rest of the 4th-grade teachers in the district is following along, we follow along two or three days after them. She's really about creating the curriculum and matching it with the common core and the best practices using thinking mathematics, the AFT thinking mathematics is a big part of the paradigm in which we teach math. She is really amazing, we make our own worksheets, in the google docs and share it amongst one another. I'm going to give that one an A+. Also, I don't want you to think that she is coming in and teaching math like the science specialist, where they come in and teach your science. This person doesn't come in and teach the lesson, she is more of the driving force of the curriculum and what we should be doing and each unit of study and each grade level. With working with the teachers in that grade level.
The sixth probe asked participants the question, "Does your district supply a technology specialist? If so, how has this benefitted you?" Participants' answers fell into four categories, as illustrated in Table 18.

Note. n = 10
Has a technology specialist, receives direct support and benefits. Four individuals reported that their district did supply a technology specialist and that they receive direct support and benefits from this position. One participant described a positive experience with the district's technology specialist and described particular services that made the position very valuable: Yeah, so we do have a district technology specialist. Again, it's one person... and she's phenomenal. I think, you know, when you think of her having to service the same number of kids and teachers and stuff as our math person does, but just it being in a different content area, just it being technology and not math... Basically, she is very, very supportive. Anytime you have a question, you know you can email her, and she gets back to you right away, and usually has... like, if its, "how do I do this on a google doc?" it's like you'll get an example from her. You won't just get a list of instructions. She'll actually give you like screenshots, so you can see exactly where you need to click, or exactly where you're looking on the screen. So that's really helpful... She'll also, like if you needed anybody to come in and do a video casting in your room or something, she's often the person that will come in and help, even just setting up. So like some classrooms have Skyped with one another, or they have done some Google hangouts with a middle school in our district and then a kindergarten class in our district, and she'll come and make sure the connectivity is good and all that.

Does not have a technology specialist, but identifies other areas of support.
Three participants reported that their district does not supply a technology specialist, but that they received support from other sources. For example, one participant described the support from the building's librarian: We have our librarian, has become the technology... she teaches the technology course. I've seen her talk... use Animoto with the kids. I've seen her use Buncees, and she's done the comic strips, so she does-and she also does internet safety as well. But I'm not sure that it was a position specifically for a technology specialist. It just… kind of carried over from librarian to technology.

Has technology specialist, receives indirect/mixed/low support and benefits.
Two participants described that their district does supply a technology specialist, but that they receive an inconsistent, or low level of support. For example, one participant shared experiences with a particular specialist whose assistance never quite fit with the classroom: There is a technology specialist that we can meet within the district for secondary. To be honest with you I've met with him a couple times, he wasn't effective for me. I definitely would love some more technology help, I'm older. I'm not old, but I'm older. And I would love to have more technology understanding. It just wasn't the best person for the job for me. They were available to come into the classrooms and answer questions, set things up for lessons. Anything that you wanted. It never gelled well with what we were doing in the classroom, it was more like wanting to do emojo or class dojo or whatever. That's another thing I didn't really need, I needed more content specific stuff, it was never content specific.
Identifies as the technology specialist for the district. One individual shared that she was the technology specialist for the district. This person shared how focusing on integrating technology with class projects improves learning: I am. But we have two of us. We have one for K-3, and I do 4th and 5th-grade technology integration. So, I go into the class once a week and integrate projectbased learning. I think it's very important. I think a lot of the teachers are not as familiar with the tools that we have. We're a one-to-one district. As far as technology, our 3rd, 4th, and 5th graders have Chromebooks. Our teachers never really had the training to do all of that, so they stay; when I go in and teach, they stay in the classroom, so I am modeling it for them as well.
The seventh probe asked participants the question, "What type of support do you receive to attend training opportunities to improve your instruction in STEM areas?" Participants' answers fell into four categories, as illustrated in Table 19. Table 19 How is support provided in terms of kits, activities, and supplies?
Categories n (%) Provides opportunities for training outside of district
Provides opportunities for training outside of district. Half of the participants reported that they are provided with support to attend training opportunities outside of their district. One individual shared about the training opportunities available during the summer "So last summer, I went to Houston, Johnson Space Center, and spent a week with some teachers out there looking at space stuff. This summer, I'll go and do two more trainings on computer programming and maker kinds of stuff… Generally, summer training for us." Another participant talked about how the time of the academic year and support from administrators affected their ability to attend workshops.
Yeah... good question. So, we, I think it depends on the year, and what's available. If there's something that you want to go to, let's say it's something that you feel you want to learn more about or whatever...if you find like an area that you want to learn more about, you can definitely bring that to your principal and say, "Hey, you know there's a workshop here, you know, it costs this much. Can I go?" And they'll kind of decide. Most of the time, they want it to align somehow to our school improvement plan. So, when we do fill out a professional development form, we do have to list how that relates to our school improvement plan, and how our students are going to benefit from it. So, you know, most of the areas that we want to learn more in, there's definitely opportunities to do that. And then sometimes, if it's an area that your principal sees that you either could use more support or more development in, or if it's an area that you're doing pretty good in and she wants you to learn even more about that, they'll often seek you out and say, "Hey, would you like to go to this workshop or this whatever," and you can go.
Provides opportunities for training within the district. Four out of the ten participants described the training opportunities that they were provided within their district. Specifically, one individual described how the teachers in their district approached professional development and how students were involved in this process.
We would get pulled out of school. That unit of study that I did on the code and drones, the 4th-grade teachers would have had a substitute for the day and left school and met at the administrative building and we would have done the PD [professional development] there. Speaking of that PD, I had two of my students join us for that PD, and they helped us run that PD. It was a virtual field trip for them. We put them on the school bus and they went over to the administration building and helped us run it. Why? Because they know at least as much as we do, or more. Once again, showing again that we are all in this together, from students to the teachers, to the PD office. We can learn something from everyone. I thought that it was really interesting that we used the kids for that PD. So, the answer to that is we would get pulled out of school for that. In addition to that, I just have to go back to all of our curriculum being in Google Docs, so we are always collaborating. It is not a dead document. If someone wants to text me or something, they can do it. Then I would give them that kind of feedback.
Anything we want/Very Supportive. A few participants reported that they received a very high level of support to attend any training opportunity. One individual discussed the level of support and funding they received from their administrator to attend workshops related to STEM: Anything we want. Yeah. Like I said, our assistant superintendent is constantly sending the department all kinds of workshops and seminars and trainings and website that would be a benefit. So anytime we want to go somewhere, they find funds for it.
Unsure/Low support. These two individuals reported that they receive a low level of support or were unsure about the level of support they received in regard to training opportunities. One participant described having to seek out training opportunities independently and how training in STEM was not a high priority: There might be a few flyers that are pushed through that... as a school. But nobody really has approached me and said, "Hey, this looks like a great opportunity for you to incorporate more STEM activities." Right now, the drive is blended learning, and so it's pretty much: we're trying to move to a blended model. So… training for STEM is really not a priority. So not too many people have approached me. Anything that I do is on my own. And so, unfortunately, right now because of the blended learning initiative, I've kind of put STEM on the backburner.
The eighth probe asked participants the question, "What opportunities for professional development in STEM have been provided to you by your school district?" Participants' answers fell into four categories, as illustrated in Table 20. The curriculum publisher provided a onetime training 1 (12.5%) District has invested in supplies rather than training 1 (12.5%) Grant covers summer training 1 (12.5%) Note. N = 8, Total is less than 10 (100%) given that not all participants responded to this question and some participants endorsed more than one response category.
Training is covered within the district. Three participants reported that their school district provides professional development related to STEM. Specifically, one individual talked about how STEM training is a frequently included in their professional development, "Again, same thing… STEM is usually, on our in-service days, a focal point. So there's always something being offered even on our own personal, professional development days that we're required to attend, there's always something available." [University-affiliated program] covers training opportunities. Two individuals specifically reported that [university-affiliated program] provides training related to STEM for professional development. One participant discussed how this partnership with this program was very beneficial for her school: Countless. Through [university-affiliated program], every year, you get your training in your initial kit, so your first-so, it it's a new kit, or a new grade level, you get training in that. And then, what's really cool about the way they do it, our partnership with [university-affiliated program], is that they have follow-up training, so even the teachers who have been teaching third grade for eighteen years, they get to go to those follow-up trainings and dig a little deeper into the content, and ask more of those, "Hey this came up in a lesson" questions... So, I think that that's really supportive.
Discussion and collaboration with other teachers at the school. Another two individuals described that they benefit from discussion and collaboration with teachers at other schools. One participant described how the combined interest in STEM from another teacher and a librarian were very helpful for training: Other than what I've mentioned I can't really think of any. I will say we had two people, the librarian and one other teacher, who got very involved with the idea of technology and science and math. We would hold in-house meetings that were voluntary to go to. They would call them an "Appy hour", we'd meet and share, mostly it was teachers sharing with each other what they are doing in the classrooms, which is really helpful. It's probably the best way to do it.
The curriculum publisher provided a one-time training. One participant discussed the training they received from the curriculum publisher, but that it was limited. "They had the STEMscope trainers come to the school and gave us the opportunity to work with them for four days, so they gave us free time, time out of the classroom, to do that with the specialists from STEMscope." District has invested in supplies rather than training. Another participant discussed how the district chose to allocate its funds into supplies rather than specific trainings or professional trainings for teachers. This participant also mentioned the option of working with the Rhode Island Science, Technology, Engineering, Art, and Mathematics (STEAM) Center: No, we haven't, that would cost a lot of money. I work in an urban district, or it used to be considered urban, REDACTED (school district) is in the same category as REDACTED, REDACTED, and REDACTED (school districts). That would be the financial state of our schools. We are on a budget, we are on a shoestring. And yeah, I told you we just spent all that money on drones, tablets, and things of that nature. Whatever money you have, that shows what you value. So no, we haven't gone outside the district. What we are talking about is doing more work with our high school teachers, tapping into their expertise. We don't have to go out of district, we can first go right into the people in our own school system. Like we had another unit on physical sciences, it was a motion and design type. I could have used some high school teachers to help us work on that. We haven't used the Rhode Island STEAM Center directly. But I think going down the road we will share with them what we are doing and see what kind of support we can get from them.
Grant covers summer training. One other participant reported that a grant covered summer STEM training, "Whatever weeks we need, we get, as far as the one they want to implement. I don't know if you know: in Rhode Island, the whole CS [Computer Science] for Rhode Island initiative has some grant money; and so, they're paying for us to do the app creators class and the maker class for this summer." The ninth probe asked participants the question, "How would you describe the support that you receive from other teachers and school staff?" Participants' answers fell into four categories, as illustrated in Table 21.

Table 21
How would you describe the support that you receive from other teachers and school staff?

(10%)
Note. Total is not equal to 10 (100%) given that some participants endorsed more than one response category.
School level support. Four participants reported that they received support from their school staff was consistent and reciprocal. One individual discussed how the school staff had established a culture of support for each other: That's great, yeah. I am in a very, very, very much... like, the culture of our school is very caring, and we are very much so a community. You know, we call ourselves Team [Redacted]. So, we definitely have that "we're all in this together" mentality. So, anytime you have a question, you can just even pop into someone's classroom and ask her about it, and people are more than willing to help you. Yeah, it's really fantastic. Department/Team level support. Three participants specified that they receive support from their department or a team. One individual discussed the support from the science department and a student-oriented approach: I have a dynamic science department. Everybody in there loves science, and everyone in there is there for middle school students. So, we collaborate a lot, grade level and across grade level, all the time. Always sharing and suggesting things that we've tried in the classroom.
District level support. Three individuals discussed how the support they received extends throughout the district. One participant specified how the utility of technology has helped all the teachers in the district troubleshoot projects and activities with each other Ever since we started creating curriculum on the shared doc, people have started popping in on it. It was kind of cool because I did it at 7:30 in the morning and I would be checking what math I would be teaching for the day and you'd see teachers all across the district popping in on it. It is like a sense of comradery, it wasn't just the people in my hall, but the people down the street… Ever since we really bought into google apps for education, there is just a lot more support throughout the district. It is not linear support its everywhere. Whatever you need, you have the tools to get it now We look for right in time support. We don't want the answer a week from now, we want it now. If I ask you a question, I want it now. That might be asynchronous, so if I talk to our elementary ed, curriculum guy, who is actually our technology director, different from specialist. If I have a question for him I'll either use google hangouts, or text, or email. He will answer me as soon as he can, a lot of the time it's instant. That is the kind of answer we get around the district too. Anyone can email anyone, google hang anyone or use a chat.
Sources of support are suboptimal/fair. Two participants shared that the support they received was either fair or suboptimal. One individual discussed how other school staff are typically very busy and do not have much focus on STEM "I think everyone is so busy that they don't necessary-you know, if I ask for something, they'll definitely do it if they can, but I don't know that there's a lot of focus on STEM or what we do." Provides support to others through professional development. Finally, one individual reported how she is a source of support for others and provides professional development. Specifically, this individual discussed how she provided training for several districts within the State of Rhode Island and worked with teacher leaders: We did some inter-district work as well where 5 or 6 districts got together and we spent time looking at science notebooks per grade level across the state, so that was different teachers working together, which was a huge support…Yes, and teachers were typically open to that if they had questions. They know that the teacher leader in the building, they would often reach out and say how were you doing and that sort of thing. Just sharing things that they were doing.

Research Question 2: How would you characterize your approach to teaching STEM content areas?
To answer this research question, participants described their approaches to teaching STEM content. This question yielded five response categories. Specifically, a majority of participants supported an inquiry-based or constructivist approach to teaching. The next largest type of response was using a hands-on approach followed by student-centered. Some participants specifically mentioned using a project-based approach to assignments. Other participants provided a variety of miscellaneous responses to this question, including teaching with enthusiasm, front-loading vocabulary, or working with organized chaos. Table 22 provides a list of this information.

Table 22
How would you characterize your approach to teaching STEM content areas?
Inquiry-based/Investigative/Constructivist. Eight participants described themselves as approaching STEM by giving their student a chance to explore and question content. For example, one teacher shared how she encourages students to ask questions "I very rarely told students anything. I always told them "teaching is not telling", (both laughing) so they would use each other a lot throughout the day, not just in science." Another participant discussed how she steers her students away from false assumptions but fosters their independence, "They get kind of frustrated about that, but my science teaching background, sort of with investigating, is kind of the same way I approach them. I try to make sure that they don't make false assumptions or go down the wrong path. But it's trying to encourage them to kind of pull it out of their own heads instead of me just reading it to them."

Hands-on.
Six individuals reported that their teaching style was more of a handson approach. One teacher described the process with a robotics class, "Or if we have something we're doing, even the robotics-like the gears, gear mechanisms and things-I'm kind of explaining what the gear is supposed to do and what it's kind of supposed to look like; and then I'm just giving them the materials and letting them kind of hands-on work with it/ play with it, figure it out for themselves." Another teacher mentioned the importance of students being able to interact with physical objects "I believe in kids actually doing the work and getting their hands dirty…Once again, that's why we got the drones, they get to touch stuff too." Student-centered. Four participants approached teaching STEM with a studentcentered. One teacher discussed an approach to work with students at their pace: I like to be more of a coach, instead of a teacher. I don't do whole class instruction. This past year, students were given a playlist, and within the playlist they had links. They would watch a video and actually, a student once said that it's better to have videos because if you tune out the video, you can re-watch it, but if you tune out the teacher, you can't ask her to repeat herself because she'll get upset. So, it's actually turned into more of a coaching position than it has been a teaching position. The kids move at their own pace.
Project-based/Collaborative. Three individuals discussed their approach to teaching STEM content by primarily utilizing projects, experiments, and small groups.
One participant talked about how students work together in small groups and then share their findings with the rest of the class, "I typically will start a unit with some type of engineering challenge and let them muddle through it. Then math would be the same way, just give them a problem and let them muddle through it together. They can do it by themselves and then we kind of share out." Miscellaneous: e.g., Teaching with enthusiasm and motivation, Front-loading vocabulary, Organized Chaos. Three participants shared a unique approach to teaching STEM content. For example, one participant spoke about the order of teaching content, "I tend to, I do front load vocabulary" Another participant shared the importance of having enthusiasm while teaching: Okay, well, number one, STEM is my favorite area to teach. I always have in any of my STEM classes, number one is enthusiasm. If you don't seem like you're excited about the content, then there's no way that you're going to get them to listen up. And especially with the age level that I teach. Anytime I'm teaching them about anything, I have to really make sure that I'm selling it. And I have to be enthusiastic and motivated. And a lot of times that comes with having the background knowledge of what you're teaching, is important to have.
In order to obtain more information regarding teacher's approaches to STEM content, participants were probed with six further questions. The first follow-up question asked participants about whether they used a specific curriculum. Four categories emerged from participants' responses (see Table 23). Most of the participants described using a kit-based curriculum, others used a district created curriculum, and the remaining participants mentioned the different curricula used by their districts. [university-affiliated program] 2 (20%) Note. Total is not equal to 10 (100%) given that some participants endorsed more than one response category.
Kit-based. Half of the participants described a specific kit-based curriculum for teaching STEM content. One participant shared opinions regarding one of these curricula: Sure. We used the FOSS, the Full Option Science System and when I was in the classroom we also had some STC, and I don't remember what that stands for, modules, but I can speak to the FOSS probably best, but I feel like the guides were/are fantastic. They lay the lesson out step by step and are supportive in that way.
Did not specify/Used district created curriculum. Three participants shared information about a curriculum that was created by their district or did not specify the name of the program. One participant mentioned "We do have a curriculum. They're based on the NGSS standards." Another participant described the district's reasoning for creating and using their own curriculum: We create our own curriculum because we want to control our own destiny. We are content with being confident in the direction in which we are going. We don't believe we need a publisher from California or Tennessee telling us what we need do. We are professional educators and we can work together and do the hard work. And it is hard work, so I mean teaching during the day is one thing. Then doing this other work that you and I have been talking about kind of happens all around us. But I will say, although there is more work in this district than other districts, I will say the moral is high. I think the morale is high because they have stake in that curriculum, we are creating it together. As people are muddling through it and have questions we're the ones who wrote it and created it so we are there to support one another. Morale is high.

Miscellaneous.
A third of individuals mentioned specific curricula not mentioned by other participants. For example, one participant talked about the curriculum used for science-writing activities "We use Betsy Fulwiler's Writing in Science curriculum, for how we answer our inquiry questions." Another participant mentioned the curriculum used for math content: With math, we use the Eureka EngageNY math program, which is common core based and that's been developed with the curriculum and common core over the past few years. It's been a long process, it's always changing.
[University-affiliated program]. Two participants specifically mentioned that the curriculum they use is from [university-affiliated program]. One participant stated, "For science, it's the [university-affiliated program] based on the standards that we have now. They just came out with new standards, are you familiar with those?" The second probe asked participants how they would characterize the strengths and limitations of their approach to teaching STEM. Responses fell into six different categories that have been summarized in Table 24.

Table 24
How would you characterize the strengths and weaknesses of your approach?
Categories n (%) Benefits for improved student learning 6 (60%) Benefits based on ease of use with other teachers
Benefits for improved student learning. Six teachers provided an example of a benefit that focused on improved student learning. One teacher stated "A strength would be that it is inquiry-based, so the students are learning the 21st-century skills and learning problem solving, engineering. They are doing a lot of solutions on their own." Another teacher shared "They are more... not so much content focused as skill focused. Where kids are applying skills rather than just feeding back memorized facts." Benefits based on ease of use with other teachers. Five participants discussed that the strength of the curriculum was having support from other teachers in their grade level or district who also were using the same curriculum. One teacher shared how having common goals was beneficial "So that's wonderful, those are all great things, we are all on the same page as far as grade level is concerned. We are working towards the same goals. We are kind of teaching around the same time frame, and yet we can put a little different spin on it. So that's some wonderful things." Recognition of the quality of the curriculum. Four individuals discussed the quality of the curriculum they used. One participant mentioned the national recognition of their curriculum "I think the strengths are that it's sort of vetted nationally and that it really sort of is based on some expertise of people who know a lot about this stuff and talk to people around the country about what's needed." Another participant discussed the quality of the kits for the curriculum, "The second strength would be the quality of the kits, you know, the materials that they give you. The inquiry lessons. The outdoor extensions that they give you get us outside." Weakness: Logistics and timing. Six participants reported that the primary weakness of their curriculum was related to logistics and timing. One of them commented on the difficulties of scheduling and organizing all the materials: With science, it's always the time, the logistics of organizing the materials and all of that. It can often depend on the makeup of your class, how that all goes. That's the challenge. Keeping track of the materials, inventorying the materials. (Both laughing) I just did that yesterday, that takes time.
Weakness: Poor localization. Three individuals discussed the lack of adjustments and flexibility in the curriculum for different levels of students. One teacher stated how the material in the curriculum had to be adjusted for struggling students: I think the other bad thing is that the level of vocabulary, I teach in an inner-city school system and their reading levels are very low for a large portion of students so they struggle a lot. So, we rewrite things for the students so they understand. I spend a lot of my time taking the lessons and scaffolding them in order to make it accessible for my students. We spent a lot of money on the curriculum that I feel it should have those things imbedded into it... I would love to see them have more outlining books for the students or vocabulary where they have more of the fill-in. They definitely need more scaffolding activities embedded into it for teachers so we don't have to rewrite everything for our students.
Another teacher discussed how the curriculum was not as advanced as the science writing skills of the students and that the difficulty in making connections with the local community to foster learning outside of the classroom: I guess the limitation is the writing piece, I feel like our writing in science is more advanced. We supplemented with the Writing and Science Program and now we're kind tweaking that to go with common core. There is a writing piece in FOSS, it's just I feel like our districts and when I was learning how to write in science it was a little bit stronger. Then the other limitation is that it is not localized because it's a national program, so we were forced to make local organizations to kind of take over learning in the classroom and use it in the school year or in our community.
Weakness: Poor assessment tools. Three participants stated that they felt the assessment component of their curriculum was weaker than desired. For example, one participant thought that the assessment questions included in the curriculum were often disconnected or too vague: Some of the things that are not so good about it, is that there are some vague areas that seem to, some questions are like where did that come from? It came from left field. That is a question that I would take out or I would have to go back and find something to teach it. All the teachers find that there is something that sometimes doesn't connect very well.
The third probe asked participants to consider the spectrum of direct instruction versus constructivist teaching and then to describe which side their style of teaching leaned towards. Responses fell into two different categories that have been summarized in Table 25. Table 25 On the spectrum of direct instruction versus constructivist teaching, a more experiential form of learning, which do you feel your style of teacher leans towards?
Categories n (%) Experiential/Constructivist/Student-Centered 8 (80%) Mixed depending on the class 2 (20%) Note. n = 10 Experiential/Constructivist/Student-Centered. All but one participant endorsed that their style of teaching leaned mostly towards an experiential, constructivist, or student-centered style of teaching. One teacher described the process for taking this approach with students: Definitely experiential. The content is... I try to-I usually work backwards. So, the kids are presented with the performance expectation in the beginning, and it's more an exploratory approach, where the science groups that are working together kind of work backwards from that final product and it's through that process that they gain the content. So, it's not so much that we're assessing... we do assess the steps along the way, in the content. But the goal is to let them know right up front -this is where we're headed, how will you get there? So, they kind of come up with the plan, and I'm just kind of there as a guide to help them get there.
Mixed depending on the class. Two individuals reported that they had a mixed approach to teaching that depended on the class. One of these teachers discussed how the changes to the approach depended on the students in the class: It depends on the class. So, I'm gonna have to say it's going to be a 50-50 right down the middle and that's because of the students that I have. For my accelerated students it's more of a hands-off approach and my ACL approach it is a hands-on approach. So, it really depends on the class. I would love to be where they have more control, it really depends on your students.
The fourth probe asked participants to describe an example of a classroom activity that matches their style of teaching. Responses fell into four different categories that have been summarized in Table 26.

Table 26
Could you provide an example of a classroom activity that matches your style of teaching?
Categories n (%) Teachers developed activities as they go through the semester 5 (50%) Teachers provided an activity through an online platform 4 (40%) Teachers provided a program that was scripted/guided 3 (30%) Students shared their work through a showcase 2 (20%) Note. Total is not equal to 10 (100%) given that some participants endorsed more than one response category.
Teachers developed activities as they go through the semester. Five participants provided examples of an activity that they develop throughout the semester depending on the class's level of understanding or specific interests. One participant described how an experimental mistake with brine shrimp led to a learning experience for the whole class.
Okay, so I would definitely say, that was something that happened recently, so that's definitely fresh in my mind. So, you know, that experiment of learning that it was cross-contamination, totally seeing what was happening, you know, I'm totally seeing him pick up the salt spoon and stir that non-salt-water cup, but I just have to bite my tongue, and that's it, you know what I mean... and, so that's definitely very much me. Just kind of sitting back and, you know, one kid will even say to me, "I have a question," and then like he'll walk in, "aw, you're not going to give me the answer anyways." That's right! You're going to figure it out! You know what I mean. It's not that I don't want you to ask me questions when you need help, but I'm going to come back with a "well, what do you think?" You know, and really try to get them to dig into that a little bit.

Teachers provided an activity through an online platform. Four individuals
reported that they use an online platform for their students to work together on larger projects or assignments. One individual specified the different steps that their students went through from the beginning of the assignment until the culmination of a video: One of the projects I just finished this week with 4th grade was... I created the assignment for their learning about renewable energy resources. So, I created on a Google doc, an assignment that they had to break into groups and study one form of renewable energy. So, once they were in their groups, suppose the biomass group, they had to start off by collaborating on a Google doc and doing the research with the questions that I laid out. Then, they had to write a script and we green-screened videos about why their renewable energy... what the challenges, the strengths... and that was one of the technology/science projects I just did... Then, they had to upload them to my YouTube channel, share them, and then critique each others.
Teachers provided a program that was scripted/guided. Three participants specified that the activity they provided was scripted or guided students through a series of steps. One participant described how this scripted approach allows students to be more self-directed: Since the curriculum is pretty scripted, I'll have the students read the directions and walk themselves through it for my accelerated students. I don't sit there and read it with them and answer the questions or probe them. I love for them to have those conversations with each other. So, I'll walk around the room and monitor their conversation and listen for that science talk that I love to hear. And that's where I can leave them and guide them. Then sometimes I might have to come as a whole classroom and say let's review something because I see that a few kids have some misconceptions. Then I call them all back in and set them back off again. That is kind of how the hands-off approach is. I might have to give direct instruction to begin with obviously and then set them off and then regroup at the end.
Students shared their work through a showcase. Two individuals explained that their students worked on a project that eventually led to a showcase or exhibition that is shared with parents. One participant described how the project was developed as an exhibition that was shared with parents "And then we set up a museum type place for the other grades to come through, and their parents to come through and take a look at it." Another participant described the development of a coding and drones program that eventually was shared with parents: We were using Tinker, this is the program that we used that actually uses Bluetooth to pair with the drones. So, we ended that unit with a drone showcase. And we simply said, your code that you write has to have at least one conditional, one function, all the different principles of computer science and coding. It was a blank slate. Kind of like Facebook, Tinker has a showcase, so when you are done with a project, you can put it in the class showcase and other kids can kind of test out your code and see, we are looking for a unique flying path of their drone. Using those conditions of coding. It was very wide open…Then we brought parents in and now we are going to ramp it up and give them a little bit more incentive to make it really authentic and put a little extra effort into it. We shared those drone showcase code with parents.
The fifth probe asked participants to describe their approach to differentiated instruction. Responses fell into five different categories that have been summarized in Table 27. Accommodating assignments based on student needs

(30%)
Note. Total is not equal to 10 (100%) given that some participants endorsed more than one response category.
Differentiated teaching depending on student learning. Eight participants discussed their process of differentiated instruction based on the progress that students make as the lesson continues each day. One participant described differentiated instruction in a variety of the courses they teach and how the experience has made this easier: Typically, if I give an example in math, so if we start our new unit of math that we created. We all start lesson one on day one. But as you go through the lessons, after day two and three kids are already starting to separate. Sally doesn't need this anymore. Johnny needs this. I might still have kids still building decimals with place value blocks. I might have other kids using virtual manipulatives on their Chromebooks. I have some kids doing it conceptually, mathematically in their head. As the lessons go on I have to identify where kids are at and giving them what they need… They could be studying decimals, it could even be the same lesson, but in different ways. Also, if I'm teaching a reading and writing workshop, which I have been teaching for 26 years. That's differentiating. If I've got 26 different readers, they are reading 26 different books. I have been managing this thing for a long time, that kind of a thing. You just kind of adapt to it in science and math and engineering.
Grouping choices by the teacher. Five individuals described how they group students in a way to foster student learning. One participant described how they balanced students of different ability levels across groups: It's a little tricky, but we do a lot of group work. That helps a lot because we can sort of create the groups that work well together. Sometimes that means putting multi-level students in one group, and sometimes, or other times we'll just control the groups, so that maybe students who need a little extra help in one particular area are all in the same group so that we can assign someone to work with them, or something like that, so that they can...and sort of leave the other group with an extra challenge or an independent activity that they can do. We also do things like we use code.org, I don't know if you're familiar with that.
Technology making differentiation/blended learning easier. Four participants identified how technology made differentiating instruction much easier. Specifically, one participant described the ease of differentiated instruction and adaptive learning with technology supports: Having this one to one environment, where we have these Chromebooks allows us to differentiate more than ever. People talk about blended learning, blended learning is the great differentiator. It's about giving kids as many tools as you can. Basically, here's what we did find with differentiating with one to one Chromebooks. Screen time led to face time. The more kids were using their Chromebooks, the more face time they had with me. Would allow me to conference and really identify where they're at. And more face time with each other. So, the last thing you would ever find in our classroom would be 26 kids on 26 devices and no one talking and interacting with the computer. The other thing we found was, it leads to more engagement, not less. It seems counterintuitive with screens and every kid has their own computer the private sector would love because we are the next big market, money makers. Ed-Tech is huge. Who wants to make the next buck. There's a lot of these adaptive learning platforms out there. An adaptive learning platform is where you are going to have 26 kids interacting with the screen one on one with the device.
Agency/Choices for students to demonstrate learning. Four individuals shared how they allow students to demonstrate what they have learned in a way that suits them best. One participant described how they are flexible with how students present their findings: There's always a choice in how they're going to demonstrate what they've learned. So, they're offered a variety of methods to show what they've learned. It could be something visual where they put together a presentation, or some kids just like the basic give me a regular kind of assessment with writing or things of that nature. Some kids like to set up a debate based on a certain topic. So, I always like to try to differentiate not only the approach but how they're going to demonstrate what they've learned.
Accommodating assignments based on student needs. Three participants discussed how they adjusted the assignments based on particular needs of the student.
One teacher discussed how modifications to assignments are based on the students' needs, "And then some kids who just can't complete an assignment, it can be modified based on their needs. So, if they are not working at their level per their IEP [individualized education program], they are... their grade level assessment will reflect something that they're working with at their level, per their IEP." Another teacher shared how the resource teacher for their classroom is very helpful for assisting with the IEPs of their inclusion-special education students: In addition, we have a resource teacher that works with some of the students, but she helps everyone actually. They have IEPs, but they have different amounts of work that they need to complete. Some of them have extended time, they will have fewer questions that they have to complete. It's a little bit different for the different students.
The sixth probe asked participants what personal factors or qualities have helped them teach. Responses fell into six different categories that have been summarized in Table 28. Patience/Appreciating differences. Five participants stated that their patience and ability to appreciate the differences of their students was beneficial to their teaching.
One participant stated, "I think, patience, in knowing that the kids aren't going to all get it at the same time, and providing opportunities for them to take several attempts before they demonstrate mastery of a certain skill or content." Another teacher shared growth as a teacher: Learning to let things go. That they are what they are… It's not a... I think the biggest thing was realizing the "one size fits all" doesn't work. You don't see it in real life. You don't buy a shirt that's a "one size fits all" and expect to look great in it. So, I have students who don't have access to computers, and so if they can finish their work in here for my curriculum, then absolutely they can work on something else for another class. Or if they have a student who emotionally is not there-and I've also been looking at mindfulness and how your brain worksif emotionally they're not there, I can't expect them to perform with the rest of their peers because they have other stuff going on that inhibits them to access their curriculum. So, understanding that a child is not a sponge, they're not going to all learn the same way, they're not going to reach a certain level at the same time and accepting that... and just going in and treating each kid as an individual instead of a subject that's supposed to sit in front of you every day to learn something... That's just not right.
Past Experiences. Four teachers shared how their prior experiences have helped them in their teaching careers. One teacher discussed how their experience they earned while pursuing different degrees has been helpful for approaching differentiation, "I have a degree in Art, so that was very helpful. I have two degrees in anthropology, so from the social science point of view I can work better with the differentiation and I also just got a Master's degree in English as a second language, so that is very helpful too." Another individual shared how their experience with a mentor shaped their interest in becoming a teacher: I've always liked science. I liked science in high school, but I also worked in my early years, in 4th grade, with a teacher who loved it. She was really on the forefront of science and math and she actually got the presidential award one year for math and science. She was amazing. She's just one of those people, like gosh I wish I could be like her when I grow up. (both laughing). She enthused me and got me involved with going to some workshops at that time and hands-on stuff. Seeing how it motivated kids and nature walks and bird feeders and all that. That was probably the biggest factor for me. The fact that I like it and the fact that I had that good mentor in the early years.
Risk-taking/Not afraid to learn with students. Three individuals discussed how they have developed an attitude of not being afraid of taking risks and to help their students share this attitude as well. One teacher responded with this approach: Risk taker, not afraid to fail, I'm really good at it actually. I'm not afraid to ask questions. I'm not afraid to learn side by side with my students. I'm not afraid to learn from my students. I move out of my comfort zone. I don't feel like I have to be an expert in anything to teach my kids, particularly when it comes to technology. All I need to know is a little bit more than them. Then once the tool gets in their hands, we will figure out if it is a worthwhile tool or not.

Sense of humor/Positive attitude.
Two participants responded that having a sense of humor and positive attitude were especially helpful for teaching young students.
One of the participants stated "Number one, humor. You've got have to a sense of humor in middle school." The other participant conferred a personal perspective: Yes! So, I am naturally very enthusiastic positive person. It's a choice, though, I will say that. It's not something that you wake up feeling every day (laughing). So, you need to make that choice to shine that way for the day, instead of letting little things get you down. So, I think that has definitely helped me. I have wanted to be a teacher since I was little, so I always knew what I wanted to do and the grade levels, knowing I wanted to be in elementary ed, was very easy from the start… Is it going to be a good day or notyou're actually in control of that. You can't control the little things that happen along the way. But you can definitely control how you're going to react to those things. So that's usually what I try to keep in perspective.

Problem-solving/Fostering independence. One individual reported about the
importance of teaching students about problem-solving and letting students work on their own towards a solution: My whole focus is on problem-solving…I was always solving problems, and I just felt like students don't really have that ability. They just like to be instructed to all the time, or they're used to it, or whatever. I think that kind of forcing that side is what motivates me, but it's also sort of what I bring to the students that they're not as used to, I think.
Being an older teacher and having more time available. One teacher shared how being older allowed her to have more time to devote to teaching. She discussed how much more time she has for teaching after-school programs and her future goals for teaching STEM: My personal quality that helps me teach… I think because I'm a little older and my children have grown and I have a little more free time to dedicate to my profession. I don't know how I did it when there were three young children (both laugh) and get my Master's degree, I don't know how I did that. But I feel like I do dedicate a lot of time to my classroom. I'm also the PD [professional development] facilitator, a teacher mentor, and an after-school SMILE teacher. I don't know if you know Smile from [your university]. It's a science and math club after school. I've done it for 5 years now. That's a lot of engineering right there for you. If you had time to experience, you would love that. So, I'm able to have exposure in different areas to keep my interests peaked as well. And grow myself professionally. I want to become nationally STEM certified. I was just offered a scholarship through STEMscopes and through my district to become a nationally STEM certified teacher. So, I think that all those things, becoming a life-long learner helps me to become a better teacher.

Research Question 3: Two words that have been frequently used in the literature to
describe integration are multi-disciplinary and inter-disciplinary. Lederman and Niece, in 1997, used the metaphor of chicken noodle soup versus tomato soup to explain the fundamental differences between multi-disciplinary and inter-disciplinary approaches to integration. In their description, multi-disciplinary was characterized as a bowl of chicken noodle soup, where each ingredient maintained its identity without direct

mixture, yet came together to make a whole. On the other hand, tomato soup represented an inter-disciplinary approach to integration in which all ingredients and subjects were mixed together and could not be easily separated. How would you describe your style of integration?
To answer this research question, participants described their approaches to teaching integrating STEM content. This question yielded four response categories. Specifically, some participants described themselves as being inter-disciplinary, others as multi-disciplinary, and some stated that it depended on the activity. A subset of participants shared that they thought the interdisciplinary approach was the best for teaching elementary school students and it allowed for the most creativity. Table 29 provides a list of this information.

Table 29
How would you describe your style of integration?

Inter-disciplinary (Tomato Soup). Five individual's style of integration was
characterized as inter-disciplinary, meaning that all ingredients and subjects were mixed together and could not be easily separated. One participant related the question to a topic that is taught in science: I have to think on that for a minute. All I was thinking of is: we always teach homogenous and heterogeneous mixtures in science (both laugh) and that's exactly the same thing! ... I would say probably more interdisciplinary. I mean, we're definitely not identifying different components, like, "today we're doing the technology part," and "tomorrow we're doing the math part." It's just like, here's a project and there's probably a math component, a science component, a technology component.

Multi-disciplinary (Chicken Soup). Three participant's style of integration was
identified as being multi-disciplinary, meaning that each ingredient maintained its identity without direct mixture, yet came together to make a whole. Interestingly, each participant that identified this way also mentioned a strong aspiration to be interdisciplinary. One participant shared a view on integration as gradually moving towards an inter-disciplinary approach with more comfort with the curriculum: I would have to say, I'm probably more of a chicken noodle soup person right now. Looking towards making some tomato soup eventually. I think with the curriculum, I have to get a little more comfortable with what is expected of us and to carry out. They definitely have shown us how to do more inquiry-based lessons and but that's us developing them from STEMScopes, and with that, I think I could pull in a lot of different things.
Both/It depends on the project. Two participants characterized their style of integration as being dependent on the activity. One participant briefly shared "I think it depends on the project really. I don't know... I think some things, I am multi-disciplinary.
And some things, I think inter-disciplinary is really important to... be able to do." Miscellaneous: Inter-disciplinary is best for fitting everything together, allows for more creativity. Two participants, one identifying as multi-disciplinary and the other as both, declared how they thought inter-disciplinary was the optimal approach to integrating various STEM topics. For example, one of the participants stated: I think in an elementary world, the inter-disciplinary is your best bet because you only have so many hours in the day and you have every subject to teach. So, I try to talk to my teachers about "well, you're reading.. and you're learning informational texts and you're working with headings and captions... and use your science nonfiction reading during reading," and try to build it that way instead of, "okay, we're going to do this obscure article that doesn't go with anything, and then we're going to do the science after because that's science.
In order to obtain more information about teacher's style of integration, participants were probed with four further questions. The first follow-up question asked participants about how science and math were related in their class. Five categories emerged from participants' responses (see Table 30). Most of the participants described particular math concepts or collecting, analyzing, and graphing data. The remaining participants mentioned the ease of integration with science and applied problem-solving.

Table 30
How are mathematics and science related in your classroom?

(80%)
Science is easily integrated/Topics should be mixed 6 (60%) Applied problem solving 2 (20%) Difficulty with integration due to new curriculum 2 (20%) Note. Total is not equal to 10 (100%) given that participants endorsed more than one response category. Median, getting the median score... Mode score. It really does tie in so nicely to what we teach in math. For sure. And also, measurement conversions. I love that in science, you know, I think one of the neatest things that I remember even hearing about in my own schooling, about, oh the metric system. Well, the United States said, "oh yeah, we're gonna get there, we're gonna get there." Well, here we are, we're not there yet. But, our entire science community in the entire world uses the metric system. So, when you're teaching that, it's such a nice conversation to have with the students and to get them to start to do those conversions. And to get them to talk about weather in Celcius. Because if you're talking to a scientist from England, and you're comparing the dreary rainy day that we have in front of us, they're not going to know when you say it's 50 degrees, they're going to be shocked. So, it's always nice to have those conversations. I mean they go hand in hand so well together that that's a piece of cake for integration.
Data collection, analyzing data, graphing and figures. Eight teachers described how math and science were related in regards to data collection and analysis. One teacher discussed the utility of graphing to help children understand trends in data through a visual format: Oh, always. Anytime we have any data collection. But in terms of like graphing results on a data table, or even having students-like the brine shrimp lesson, you know-we had them take observation notes of what they were noticing and how many brine shrimp were hatching and things like that...And then we were able to do/we were able to graph that, and kind of look at the trend of that and find out where that sweet spot is for brine shrimp to be hatching. That's going to really help them. And I would say, in terms of that frequency chart, when you're looking at doing repeated trials and doing those multi-trials, getting some frequency charts in there.

Science is easily integrated/Topics should be mixed. Six individuals expressed
that science lends itself easily to integration and that STEM topics should be mixed as much as possible to get the most out of the school day. One teacher shared how they are responsible for teaching many units of study each day and that integrating them makes it easier: That's the goal, make them connected. There are times where they just don't. There are times when we aren't teaching science, there are only so many minutes in the day. As a 4th-grade teacher in our district, I'm responsible for teaching two units of study of science, two units of study for social sciences, and we do integrate those into our English language arts block as much as we can. The goal is to get our math and science connected.
Applied problem-solving. Two participants specified that applied problem solving incorporates science and mathematics in the classroom. One teacher talked about how the students would not be aware that they had done math because it was integrated into a science lesson: I would apply it in our science, what we had done in the classroom so that it could be related to science where kids would say "we didn't do math today," but we actually did do math as well as math practices and the persistence and problemsolving. Like when I give my math, if I had centers I would integrate that into what we were doing in science. So that the skills could be applied, but in a context that the kids knew well. Difficulty with integration due to new curriculum. Two teachers experienced difficulties with integrating mathematics and science due to a new curriculum. One of the participants shared that certain topics did not lend themselves well to integration: We've just been doing ecosystems a lot, and there hasn't been much math discussion. But I didn't get through our weather stuff and some other things. I think there would be more math if I was able to cover some more topics. But being a new curriculum, I didn't get to add that.
The second prompt asked participants if they include engineering topics in their teacher of math or science, and to provide an example. Three response categories were generated which have been summarized in Table 31. No, limited resources 2 (20%) Note. Total is not equal to 10 (100%) given that participants endorsed more than one response category.

Yes, includes design process activities (some component of design & redesign).
Eight participants included engineering topics, specifically design process activities, which included an opportunity to re-design. One teacher shared about an assignment, which allowed students the opportunity to redesign, "Yes, one example would be to design a cooler to maintain the integrity of an ice cube for as long as possible. They have to design it and then test it and then they can redesign it. That is more of an engineering viewpoint." Another teacher discussed realistic situations encountered by engineers: Even we do a lesson, too, where even any time the chapter builds something, we always talk about the engineering behind it, and then I'll kind of go over and I'll say, "Oh! Budget cut!" And I'll take half their materials. And they're like, "Wait! Wait! I need it!" I'm sorry, you got a budget cut, sorry. You know, you're not getting the money for your project. So now you have to make it work with that much. And that's what real-life engineers face. It's always... it's a great time.

Yes, includes concepts, challenges, or career exploration.
Four teachers talked about how they incorporate engineering concepts, challenges, or careers into their teaching. One participant shared about how her class learned about challenges faced by other cultures: My fifth grade, for example, is learning about density, so they watched and learned about the Plastiki, some boat that was built out of plastic bottles, and they're engineering boats made out of just recyclable materials; they have the constraints and the challenges... We try to have multiple engineering projects for every different grade level.
Another teacher discussed how there is a rising interest in engineering careers among young students: So, I think that the engineering part really ties in well with our real-life kind of connection of that... you know, any time we ask those questions, how does this connect to our real world? Everybody always brings up engineers. I've heard more and more, through the years of teaching, of kids saying they want to be engineers when they grow up. And I'm wondering if that is because of all the discussion that we have, not only me as a teacher, but I know my colleagues discussing it too with them throughout the years.
No, limited resources. Two individuals mentioned that they did not include engineering topics in their teaching due to limited resources. One teacher shared concerns about taking risks with engineering activities: Unfortunately, no engineering. I haven't... I don't have the time or the space to do it, and I don't have the training to do it, or the materials. Again, that's a bunch of excuses, which is pretty unacceptable. But it would be nice to have the flexibility and the time to create a bridge. we used to... In high school, we made spaghetti bridges. Then we'd have what was known as "the Crusher" from [a local university] come in and crush the bridges to see how sturdy they were. That was when school was fun…Yeah, it would be nice, but then I'm also afraid of doing that because if it flops in my class, then I'm afraid to be compared to the other teachers who are sticking with the curriculum and the expectation that I should be doing the same thing because we're a department. So, I'm afraid to try it because I don't want to get in trouble if it fails and test scores go down. So, I'm afraid to take that risk.
The third prompt asked participants to describe how they have integrated technology into their courses. Four response categories were generated which have been summarized in Table 32. Google programs 6 (60%) Miscellaneous: (e.g., SmartBoards, computer programs, 3-D printing, probes, green screen, and drones)

(50%)
Note. Total is not equal to 10 (100%) given that participants endorsed more than one response category.

Personal electronic devices (1:1 Chromebooks, laptops, iPads). Eight
participants integrated technology through the use of personal electronic devices including one-to-one Chromebooks, laptops, and/or iPads. One of the teachers discussed her enthusiasm for integrating technology at a district-wide level: Let's see... countless... So, I am super enthusiastic about technology, I kind of always have been. But, I am our school's technology lead teacher. So, we don't actually have technology classes, but what our district does is they have one tech lead teacher in each school. It's a stipend paid position. You have to apply for it every year. I think by stepping up to that role, I've been in that role for about five years now, four or five years, and that definitely helps me learn a lot more about integration technology and getting that in. I also attend a lot of the professional developments that are centered around technology because people know that I'm really into that. They often will send me, and I always will come back and bring something back to the table-present at a faculty meeting, present at a district meeting-whatever I have to do. Integrating technology has always been a love of mine... but has definitely gotten so much easier because this year we have rolled out one-to-one devices, in terms of Chromebooks, all the students in grades 1-12.
Online websites for simulations, videos, and activities. Eight teachers integrated technology into their teaching by making use of online websites for simulations, videos, and activities. One teacher shared about how these resources allow students to broaden their experiences beyond a classroom: So, they do online lab experiences and do things that are maybe not feasibly acceptable, like looking down into a volcano, or checking out earthquakes in different parts of the world. So, it kind of brings the world closer to them.

Google programs. Six individuals specifically mentioned using web-based
Google programs for their instruction. One teacher discussed receiving professional development on Google programs: So actually, right now I'm working on PD [professional development] courses, and the course I'm doing right now is the Google classroom with the Google Gsuite and all that stuff. So, I'm incorporating right now Google forms, Google Drive, Google Sheets we just did today…I also kind of want to use Google Earth, because there's a lot of math and engineering in Google Earth, and the kids use it in their social studies class too. So, it can bridge math and social studies component. That's pretty much how technology is used in this class.
Miscellaneous: (e.g., SmartBoards, computer programs, 3-D printing, probes, green-screen, and drones). Five participants integrated a variety of technology, such as Smart Boards, computer programs, computer-aided drawing, 3-D printing, probes, green-screen, and/or drones. One of the teachers shared a program that student enjoys, "Like there is a Coaster Creator program from the Jason Group that goes with kinetic energy and potential energy. It's sort of a game, but it teaches potential and kinetic energy very well." Another of the teachers indicated that the technology was not always as reliable: We have a 3D printer; it was donated. But it is not really great quality, so we end up with... it uncalibrates itself quite a lot. If you breathe on it the wrong way it's not covered so. The students are often disappointed by it, so I actually don't use it as much as I would like to, because it ends up being a flop more than it is getting them excited about it.
The fourth prompt asked participants if they were able to integrate other subject areas. Three response categories were generated which have been summarized in Table   33. English language arts/Reading and writing. All but one participant mentioned integrating English Language Arts (ELA), or Reading and Writing into their STEM topics. One participant discussed the overlap of science and ELA coursework: It was really cute, cause I was doing a writing lesson, and one of the little girls raised her hand and asked, "can I use the science sentence starters?" And I realized, oh jeez, they think that's only for science. Yes! Yes! Of course! So then, starting to teach them stuff about, you know, you can use "furthermore" even when you're not talking about science you have more to say, and you're not writing in science, you can still use that! So that's the way to integrate that in there.
Another teacher shared expectations for students when communicating their findings: Well, English is always integrated because my expectation is that they write in complete sentences. I don't take off points for that, but I do expect, when they are answering a question, they write in a complete thought. When we are doing communicating as a scientist, we need to write in paragraphs. We need a topic sentence, we need a closing sentence. Another thing that is big in science and also in STEMScopes is Claim, Evidence, and Reasoning, which is a CER. They have to state their claim, they support it with evidence and have to have their reasoning as to why they believe the claim is this way by looking at the evidence, so there is a lot of writing involved with that.
Art. Three teachers integrated art into their work. One teacher discussed working alongside art teachers to coordinate coverage of STEM topics, "Then for art, I worked very closely with all of the specialists to make sure they knew what was happening in the classroom." Another teacher described how art fits well with math topics: There's the art. I like to use... for enrichment pieces, especially with geometry, you can use fractals... We've studied Pascal's triangle. The geometric components to that [...] So I try to use-especially in geometry, there's... and actually, even in graphing, you can do a whole bunch of... You can make an art design, find the slope of a line in a coordinate plane, and so you can do a whole bunch of stuff with art and math.
Music. Two individuals discussed integrating music into their teaching. One teacher shared how a class activity in making a ball maze carried over into a music class, "In music, they would use different sounds for throwing or rolling the ball differently. If you rolled the ball you made one sound or throw the ball and make a different sound." Social Studies. One participant integrated social studies into teaching. This individual shared a focus on teaching about different climates, "You know, social studies, any time we're... weather patterns, things like that... That's a great way to, you know, learn about climates in different regions of the U.S. Yeah, it's pretty cool." Physical Education. One teacher integrated science into the students' physical education class by collaborating with the coach. For example, both the teacher and coach worked on using consistent vocabulary: For example, in P.E. [physical education] that was a pretty easy one, when they were doing balls they would use the same vocabulary as we were using force and friction like you said. Putting more force and less force rather than just kicking harder and kicking softer, so we work with that same type of language.

Research Question 4: Describe your familiarity with the Next Generation Science
Standards.
To answer this research question, participants described their approaches to teaching integrating STEM content. This question yielded four response categories. Specifically, the participants endorsed different degrees of familiarity with the NGSS ranging from very familiar to limited familiarity. Table 34 provides a list of this information. I would say I am somewhat of an expert with my position now that I live, eat, and breathe NGSS. I provide professional development for teachers around NGSS. We have specific workshops that focus on specific practices or crosscutting concepts and we are also continually modeling lessons in the classroom and using NGSS framework in our workshops.
Pretty familiar. Three teachers described that they were fairly familiar with NGSS because of the extensive trainings they have attended. One participant discussed how training through [university-affiliated program] provided information about NGSS and related concepts: So, I am pretty familiar with them, just because of my work with [universityaffiliated program] that I do as a teacher-leader…I would say that I'm pretty familiar with them in terms of the work that I've done with [university-affiliated program], looking at the standards and studying them a little bit deeper. We have a summer academy that we attend. It's a two-day at URI in the summer with our [university-affiliated program] team. Our teacher-leaders go on one of them, so I'm able to kind of get a little bit more guidance on those summer days as well…what I like about that particular summer workshop, is we study and we do a lot of the stuff that we're going to be using in our classroom.
Not very familiar. Two individuals described their limited familiarity with the NGSS. One teacher's description implied awareness of them through the trainings that they attended, but did not provide much more detail beyond this: I went to a presentation by some gentlemen who came to [a local university]. This was probably two or three years ago, they weren't quite finished yet, but he was coming to do a presentation on them. It was sponsored by [university-affiliated program]. I went to that. We hear about them through our [university-affiliated program] training sessions, so that's pretty much it. I think I have a paper in my room some place that's got them on it.
Limited familiarity. One participant described having limited familiarity with NGSS, with the only knowledge acquired coming from personal meetings where the standards were explained. Specifically, the inclusion of NGSS into the curriculum was handled by someone at a district level: They are incredibly hard to read. The document's a beast, if you don't have someone who has gone through the training, it is really hard to understand…We happen to have one of the NGSS writers in our group. He is really helpful. Having said that, I've talked to most teachers from my school, they can't make heads or tails of it. They don't necessarily have to know because when it comes to writing those units of study, somewhere at the district level, we have a K-12 science scoping sequence for lack of a better term. NGSS runs what we are doing and why we are doing it…I'll say that my level of NGSS is when I first saw them like four years ago and tried to read them, I couldn't make heads or tails, but then over the last two years, we have taken them out of the hands of teachers really.
The first prompt asked participants if they made efforts to orient their class towards the NGSS standards. Three response categories were generated by the participants and these have been summarized in Table 35.

Table 35
What efforts have you made to orient your class towards the NGSS?
Categories n (%) Nothing specified 3 (30%) Focus on standards/concepts 3 (30%) Increased hands-on activities 2 (20%) Miscellaneous 2 (20%) Note. n = 10 Nothing specified. Three participants indicated that they have not made many efforts to orient their classes towards the NGSS. One teacher stated, "I can't say specifically. I mean I haven't had that on the forefront of my mind in terms of when I'm doing things. I guess I would say whatever the kits are providing us. I am assuming those are all in there." Focus on standards/concepts. Three participants mentioned orienting their classes towards the NGSS by focusing on addressing standards with students as well as focusing on the cross-cutting concepts and disciplinary core ideas. One teacher discussed how these concepts were shared with students in a manner that was easily understandable: I have up on my board the class curriculum. I have posters of them so the kids know. I have the engineering topics and the standards all on mini posters. So, when I introduce a new topic. I point to the mini-poster for a visual and so it is written out. We talk about what they are learning and that these are the standards that they're learning. I also have it listed on a board, with what we are learning and the standard that they're learning in kid-speak, so they understand it and also the cross-cutting concepts and the science and engineering practice.

Increased hands-on activities.
Two instructors attempted to orient their classes towards the NGSS by increasing hands-on activities and making models. For example, one interviewee shared having students move beyond simply learning and reciting information: I think a lot of the NGSS stuff is about making models and sort of implementing, and I think that pretty much everything we do here is more about hands-on and implementing more than it is sort of just being able to regurgitate content…what I know about the NGSS and some of the engineering standards that they've created and stuff, like nationally taking it and doing it in the classroom kind of fits with a lot of it. Sort of the "making models" and all of that kind of stuff is what we do, and what we talk about all the time.
Miscellaneous: (e.g., sharing NGSS expectations with parents, adjusting curriculum to make it more grade appropriate). Two teachers attempted to orient their classes towards the NGSS by sharing educational expectations with parents or by adjusting the curriculum at the end of the year to make NGSS content more grade appropriate. One participant discussed the importance of obtaining support from parents for quality STEM education by using NGSS parent guides: I think where I like to kind of start is with the parents. I think a lot of times if you…you know, you could start with the kids, which is great, but then sometimes when they relay information at home, it can be kind of be very confusing, and then you're fielding a ton of emails. So, I like to kind of talk with the parents about it. One of the things that we have…some of the parents that are educators themselves, and I have a lot of science educators actually this year, and last. They knew a little bit more about it, they were able to talk. They had suggested about the parent guides that are out there, through NGSS, and so I think that is definitely a good way of starting to kind of get that out there, is letting the parents know what is going to be the expectations. But then as a teacher, you know, just trying to look at: okay, what were we doing in the past? And what is the little bit of difference that is going on now that we can kind of tie in a little bit? And starting small-that's always my thing, is always just start with one thing. You know? Start small. You don't expect to be an expert on everything all at once. So that's usually where I begin.
The second prompt asked participants how they changed their curriculum in response to the NGSS. Three response categories were generated, which have been summarized in Table 36. Table 36 How have you changed your curriculum in response to the NGSS Standards?
Categories n (%) Large shift, district has changed the curriculum to align with NGSS 6 (66.6%) Currently uses aligned curriculum 2 (22.2%) Not really 1 (11.1%) Note. n = 9 Large shift, district has changed the curriculum to align with NGSS. Six participants' curriculum was changed in a large, district-wide, shift to make it align with the NGSS. One participant described this shift as follows: Completely. Our science curriculum, like I said, there was a three and a half year lap where we struggled. Teachers were like now what do we do. We didn't have much direction from the district, they just told us not to panic, so we kind of did our own little thing for a while. But now, it's getting back up and built. Not to say it's perfect because some of these units are first generation. We need to go back and look at them again with some experts consulting us.
Currently uses aligned curriculum. Two instructors described that their curriculum already aligned with the NGSS. One of these instructors described their gratitude for this: Well, I have to say I'm quite lucky in that the curriculum that I have is aligned. So, it's great to have those lessons kind of be-not that they're done for youbut you have that support. You know that because we are doing the new NGSS kits, in class, and with [university-affiliated program], those are integrated in there…It has been quite seamless for us…I don't even know if half the people know that they're teaching NGSS… but I do, and those new FOSS [Full Option Science System] kits.
Not really. One teacher shared not making any specific efforts to orient the class towards the NGSS.
The third prompt asked participants how standardized testing affected their curriculum. Four response categories were generated which have been summarized in Table 37. Table 37 How has standardized testing affected your curriculum?
Categories n (%) Curriculum adjusted to improve test scores 5 (50%) Strong negative impact 3 (30%) Minimal or no impact 2 (20%) Future testing will align with curriculum 1 (10%) Note. Total is not equal to 10 (100%) given that one participant endorsed more than one response category.
Curriculum adjusted to improve test scores. Half of the participants stated that the curriculum was adjusted with the intent of improving student standardized testing scores. One instructor discussed how standardized testing creates expectations for the district: Part of the reason FOSS [Full Option Science System] was chosen was that it had an assessment system built in, so there are formative assessments as well as summative assessments and those pieces are aligned to NGSS. The standardized testing NECAP [New England Common Assessment Program] has always been one of the reasons I suppose some of the district's support science education and STEM education. I think there is a piece to standardizing testing. If you are tested for it then you have the expectation of teaching it and the district has supported that.
Strong negative impact. Three of the teachers discussed how standardized testing reduces time with students due to test preparation, and is a source of test anxiety for students. One instructor elucidated feelings about standardized testing and the impact it has on students: Oh boy…This question (interviewee laughing). I love…you know…I'm a teacher, okay. Love it. I understand the need for testing. However, I feel as though it is a little bit much! And I…so this year, I teach fourth grade, and my fourth graders took the Science NECAP [New England Common Assessment Program ], and this is kind of where it all started. So we, have a PARCC [Partnership for Assessment of Readiness for College and Careers] assessment, you know, our standardized ELA [English-Language Arts] and Math, and that started after April vacation, and we just finished our testing the last week in May. So, between we had PARCC-two weeks of PARCC/three weeks of PARCC; and then we had a week of our NECAP testing, and then we had going into our…just the districtbased testing that we do, you know on our level we do the STAR [Standardized Testing and Reporting] assessment. So, you know, putting those out there. You know, standardized testing, I know that there's definitely a place for it. I do think that we can scale back a little bit. I've seen students, in terms of test anxiety, increase. And that's the part that is concerning to me. Because these students are feeling inadequate, because they're feeling stressed without realizing that stress is a natural part of the world…but, before any type of standardized test, I always do a little bit of test prep. And my test prep involves not content, more test-taking skills. How to…if you're feeling anxious, how can you calm yourself down, and how can you kind of see what it's going to look like.
Minimal or no impact. Two participants stated that standardized testing had little to no impact on their curriculum and that their curriculum prepares the students well for standardized testing. One of the participants stated that the NECAP testing did not really match the NGSS: No, because we standardize tests with NECAP, and I started this position when NGSS first came out the first year, and we have kind of said we're going to not pay attention to the NECAP science assessment, we're going to go forward with the standards that have been brought down. And if our kids can do the things required in the NGSS, then they'll be capable of doing well on standardized tests. We have a very proficient district anyway, so on the multiple-choice part of NECAP, we always do well. It's the inquiry part that we always kind of fell down on, and that's only gone up since this position started, because they really understand science, they can apply it more. But we haven't used that to drive any instruction.
Another participant shared views on teaching towards a test: Fortunately, I work in a system that doesn't value... they have value, but we aren't going to create everything we do around those tests. They are just there. They are a measure. We focus on the work, the day to day work, day to day grind. Yes, we want our kids to do well. Yes, they partake in it. There is virtually no practice, we don't do anything to get ready for it. We just do the work. So, I would say, minimum. We want our kids to be scientists, social scientists, we're not getting them ready for a test. We trust that if we give them these kinds of experiences they'll continue. If all we do is get ready for a test, who cares honestly?
Future testing will align with curriculum. One individual stated that the curriculum did not align well with the NECAP testing, but future testing will be better aligned with the NGSS. This teacher explained that with RI moving away from NECAP testing would result in testing being closer to the curriculum currently used by the curriculum: We finished our final year this year of the old science NECAP standardized tests. This was our last year, thank God, because since we moved to NGSS five years ago, it didn't really align. The type of testing in that standardized test didn't align well with what our approach was, and it was very different than before. Our kids still performed very well on it, but I'm looking forward to this year, they're implementing the first NGSS standardized test in our building for grade 8, so I'll be interested to see what that's going to look like.
The fourth prompt asked participants how has their district responded to NGSS.
Seven response categories were generated which have been summarized in Table 38.

Table 38
Has your district addressed the NGSS or required you to teach it?
Categories n (%) Indirectly 3 (33.3%) Yes, early adopters 1 (11.1%) Yes, able to share knowledge with department 1 (11.1%) Yes, slow shift 1 (11.1%) Yes, did not specify 1 (11.1%) No 1 (11.1%) Not currently, plans to 1 (11.1%) Note. n = 9 Indirectly. Three teachers stated that their district has addressed the NGSS indirectly through adhering to their NGSS aligned curriculum. One individual shared, "They've addressed it. Every grade level now has two units of study. They overlap if we have physical science in one grade. It does spiral, they do a pretty good job of doing that.
We are right on target." Yes, early adopters. One individual indicated that the district adopted the NGSS "as early as they could." Yes, able to share knowledge with department. One interview shared that the district had addressed the NGSS and was able to send an instructor back to train the rest of the department: Yes, and I brought that knowledge back to the department and had PD [professional development] for my teachers. That was my job as PD facilitator. I brought that back to them and I taught them the standards and how to read them after I learned how to read them. What the students should be doing and what it should look like. So, I brought that back to the teacher, so we can look at that together. That was great.
Yes, slow shift. One person discussed how their district addressed the NGSS by requiring their instructors to teach it, but that this shift was gradual: We started changing about 3 years ago, was the shift. I guess the supportive piece of that is the shift was done slowly rather than overhauling everything and saying here everything is new. We did in a slower fashion where instead of teaching all three courses, we used to teach three, we started with two and spread them out when we had them for a longer period of time so that teachers could really learn at the mechanic level. Before having to do it all and being expected to do it all.
Yes, did not specify. One teacher shared that the district adopted NGSS, but did not specify how it did so.
No. One participant stated that the district did not require the teachers to adopt the NGSS.
Not currently, plans to. One participant stated that the teachers do not currently teach to the NGSS, but that there are plans to do so in the future: Well, we haven't done much with it yet. I think we're going to go forwardmoving forward I think we'll definitely see more. I mean we have had information provided to us by [university-affiliated program] in our trainings, and workshops, the kits that you teach-they're definitely talked about in there. But I'd be willing to say that we're about a medium for that. I think that going forward we're going to see a lot more of support in that area.
The fifth prompt asked participants if their district has provided any training or professional development in NGSS. Five response categories were generated which have been summarized in Table 39. Table 39 Has your district provided any training or professional development in the NGSS?
Categories n (%) The district provides multiple opportunities for training and development in NGSS 2 (25%) Lessons are set to meet NGSS standards, but there is no additional training 2 (25%) The district provided training for science, but not for my discipline 2 (25%) Yes, the district provides training and development for NGSS 1 (12.5%) There is not as much training and development as instructors would like 1 (12.5%)

Note. n = 8
The district provides multiple opportunities for training and development in NGSS. Two instructors described how the district provided multiple opportunities for training for the NGSS. One interview shared an experience, "Yes, but that was voluntary.
[university-affiliated program] invited people and a few of us went up to hear that. We've also had [university-affiliated program] people come into our district and talk to us at faculty meetings or you know small groups on professional development days." Lessons are set to meet NGSS standards, but there is no additional training.
Two participants discussed that their daily lessons were organized to help students meet the NGSS, but there were no additional trainings received for this purpose. For example, one teacher shared how the standards have been identified for the teachers and that they focus on the delivery of the content: That's because they've identified where in grade 4, what NGSS standard and what core concept they want to address. We don't sit with teachers and say "This is the standard," we don't care. We just want to do the work. That's just our roadmap that says okay, what does balance science program look like and what kind of experiences do your kids need. Oh, you gotta have inquiry in here, you've got to have some engineering in here. The tools are just the technology. We're doing it and I think we are doing it really well. But the teachers don't know what standard they are working on and that's okay. They need to focus on the nuts and the bolts.
The district provided training for science, but not for my discipline. Two teachers shared that the district does provide NGSS training for the science instructors, but that other disciplines, such as math and STEM, do not receive training. One teacher discussed that upon transitioning from a science teacher to a STEM teacher, she no longer receives training, "Not since I've been a STEM teacher; but as a science teacher, we definitely did a lot with NGSS. There were different generations of it... So, the district, yes. Not necessarily STEM-related." Yes, the district provides training and development for NGSS. One individual simply stated that the district provided training for its instructors, but did not elaborate.

There is not as much training and development as instructors would like.
One person talked about how the district provided training and development for the NGSS, but not as much as the teachers at her school would like: Not as much as we would have liked. Quite a few of us were on the writing committee. In that sense, they gave us time out of school and supplied substitutes, so that we could go to the writing curriculum, days that we needed. So, they were supportive of that.

Research Question 5: In your view, how do you feel about the importance of creativity in the classroom?
To answer this research question, participants described their feelings about the importance of creativity in the classroom. This question yielded two response categories.
Specifically, the participants endorsed either that creativity was very important, or that they did not have as much creativity in the classroom as they would like. However, given the nature of responses provided, the category "Very important" was further divided into three subcategories, including "self-expression," "leads to life-long learning," and "fosters creativity in the classroom." Table 40 provides a list of this information.

Table 40
In your view, how do you feel about the importance of creativity in the classroom?
Categories n (%) Creativity is very important 9 (90%) Self-expression 3 (30%) Leads to life-long learning 2 (20%) Fosters creativity in the classroom 2 (20%) Important, but not as much creativity in the classroom as desired 2 (20%) Note. Total is not equal to 10 (100%) given that participants endorsed more than one response category.
Creativity is very important. All but one participant indicated that they felt that creativity was very important in the classroom. Seven of the nine participants went on to expand on their reasoning with additional information. The two responses that were not divided into three subcategories simply acknowledged the importance of creativity in students. For example, one shared about how creativity should be a core component in the classroom: It's huge, I mean creativity, every student is creative in some way. It might be in math, in engineering, in art, it might be in writing. I mean creativity is what drives our uniqueness and what makes us all richer. If someone is a really good writer, they can be as creative as they want. Their creative genius only makes us smarter because then we look at what the do and we're like "wow" and start emulating that. Well it's the same thing in math. So in math, we might have a tradition algorithm we want to teach, but we never start with that. We start with well how did you solve it. Then you've got that creative math student, that you know you always want to hear from. And they push your thinking. So, creativity has got to be one of the core principles in the classroom at school.
The first subcategory of responses specified that teachers recognized its importance and felt that creativity is important for self-expression. One of the three participants commented on being pleased when students create novel solutions: I think it's very important. It's important for me to be creative, to be able to reach the students, but I love when they aren't afraid to personalize and create... be creative about what they're doing, instead of-so many of them want a recipe for how to make something-and I love when they go off the map and just go for it. I think it's really important.
The second subcategory of responses acknowledged the importance of creativity, but also specified that it is important for leading to life-long learning or engaging assignments. One of the two participants elaborated on the importance of choice for students: So, I think providing students with the opportunity to be creative, to have a little bit of voice and choice and some agency in their learning is the only way we're gonna be creating lifelong learners. Or else they're going to feel like they have to fit into a box that they don't fit in, and they're gonna shut down.
The third subcategory of responses also identified the value of creativity, but also elaborated the about the importance of students thinking flexibly. One of the two teachers discussed the importance of divergent thinking with multiple solutions rather than the default style of convergent thinking: It's very important. I think the general belief is that science is just one answer type thing, and there's no other -I think kids need to be allowed to use their creativity to solve some of the problems that are in our world today regarding science. So, it's very much encouraged. And again, whenever I look at the performance expectation of where kids are headed in a certain unit, I try to make it an experiential. The kid can be a little creative. And it's not just one answer is the answer. We gotta give kids the opportunity to explore a little, be creative, and even if it's a failure, we always go back and reflect on, well what do you think went wrong? So, I think creativity plays a big part of science, even though it may not seem like it fits that way.
Important, but not as much creativity in the classroom as desired. The second category of responses acknowledged the importance of creativity, but focused more on the loss of opportunities in the classroom. One of the two participants expanded on how the responsibility of closing achievement gaps leaves little room for creativity in the classroom: It's a business. This school-public education-is a business. It's not creative, in my opinion. I don't know. It might be different in maybe some of the other classes. But from the math perspective, you're given a set of standards, you need to teach to those standards, all students have to reach those standards. If not, you need to provide interventions so that they do meet those standards. So, we don't have time in a school day to go find something, any enrichment components to boost those levels up because we're supposed to be closing gaps.
The first prompt asked participants about the opportunities for creativity they have included in their courses. Five response categories and one subcategory were generated, which have been summarized in Table 41. Do not feel there is enough creativity 2 (20%) Divergent/Convergent thinking 2 (20%) Interdisciplinary collaboration 1 (10%) Note. Total is not equal to 10 (100%) given that participants endorsed more than one response category.
Project-based, students have a lot of autonomy in design. Six teachers described using projects that allow their students to have a lot of freedom to design. One of the respondents shared a specific creative exercise that that was part of their science curriculum to design toys: One of the Project Lead the Way initial tasks is that they have to design a foot orthosis for a child with cerebral palsy. It has to meet certain criteria and that kind of thing. Beyond that, they can do pretty much whatever they want. I've gotten some very creative approaches to how to solve that problem for students. The final project is a similar kind of thing where they're supposed to design a toy for a child with cerebral palsy that meets certain criteria, and that way they get very creative…It is that particular thing I think is a change in Project Lead the Way for this year, and it is... they are really good projects, cause the students learn a lot about cerebral palsy, but then they're incorporating design and drawing.
The first subcategory of responses specified that three of the six teachers allowed students a great deal of autonomy with design and also specified an engineering component. One of the three instructors discussed how engineering lends itself more easily to being creative: But, I think a lot of the engineering projects that we do will be themselves a great creativity….A lot of the science experiments are not as creative, but when they're designing their own experiments, they are creative./ …Then we do invention conventions, so the kids are allowed to create their own inventions and, so, lots of opportunities I think to be creative.
Students are allowed to use whatever means to meet a standard. Four instructors stated that they allow students the ability to complete their assignments or meet a standard in many different ways. One of these individuals described an openended approach to assignments: I try to be a constructivist, so I don't give them, "you have to do x, y, and z" So there are always opportunities for them to tackle what they do in different ways like I said with the Chromebooks. So, my students typically use 60-80 apps, web 2.0 tools, extensions, those tools that are available in that Chromebook. By the end of the year, they are exposed to 60 or 80 of them and they might use 20-30 of them a lot. When we have a task, I don't say "use this tool." No, you just give them the task or the problem and before you know it, there are 3-4 different ways kids are doing it with 3-4 different apps or extensions. That's how we do it. We don't make them follow directions. We give them a problem to solve. Whether it's reading, writing, math, engineering, science. It's all about solving problems.
Do not feel there is enough creativity. Two participants stated that they make attempts to allow creativity in the classroom, but do not feel there are enough opportunities yet. One of the teachers recognized limitations and talked about plans for the future: I tried the Buncees this year and I did the comic strips. So now that I know the kids are familiar with it, I can write up rubrics so that they hit... I can assess them and check off the standards that they meet. But I do want to find more ways for kids to be creative. Perhaps, maybe even blogging next year... Have kids blog. Oh! We've used Padlets too. So, with the Padlet, what they did was they would come up with questions, and they can edit each other's questions to make it better. I also want to do Google Slides. I definitely want to look at Slides next year, too. For next year, especially the technology pieces, I want-for creativity-would be Slides, the Buncees. Yeah, there definitely needs to be more creative stuff. And not on paper-pencil. Divergent/Convergent thinking. Two interviewees promoted creative thinking by fostering both divergent and convergent thinking. One of the teachers elaborated on a specific situation involving creative thinking: But we do want efficiency. Today, as a matter of fact, I did a multistep problemsolving lesson and how they solved it was amazing, their thinking was everywhere. They had these huge whiteboards. I'd say two-thirds of the students were able to accomplish it, but when we were done we had to come up with some kind of constructs. So, we weren't able to follow your thinking, so we need to use precise labels. So, from now on we will use precise labels. We kind of build this together. It was amazing, wish you could have been there.
Interdisciplinary collaboration. One individual mentioned that working with other teachers and to create more overlap between their courses was an enjoyable opportunity: So, it was kind of nice to work with the art teacher and the music teacher and have their input. And when they went to those art and music classes or even the robotics class, they were able to use what we were doing in the science class to work on a project through that method. So, there's a lot of good collaboration going on as far as the creative aspect goes.
The second prompt asked participants how they balance opportunities for creativity with the potential for disruptive behavior. Four response categories were generated which have been summarized in Table 42.

Table 42
How do you balance the opportunities for creativity with the potential for disruptive behavior with conformity and the need for guided instruction?
Categories n (%) Sets clear but supportive boundaries and expectations 6 (60%) Uses creative work to minimize disruptive behaviors 3 (30%) Has sensitivity towards students to manage behavior 2 (20%) Has a disciplinary/behavioral support curriculum 2 (20%) Note. Total is not equal to 10 (100%) given that participants endorsed more than one response category.

Sets clear but supportive boundaries and expectations.
Six interviewees explained that they created clear but supportive boundaries and expectations for their students. One teacher explained the importance of the first weeks for establishing these boundaries: Developing a community of learners, that's what happens in those first 2-4 weeks, well it happens all year long, but those weeks are crucial to develop protocols and procedures to encourage creativity and socialization and choice. But all of that takes responsibility. Sometimes you have to go slow to go fast. That's how we do it. We develop procedures as a class, what works for us, what doesn't work for us. I'll give an example, the first or second day of school you got 26 kids, they got 26 Chromebooks and I want their attention. And they all have a screen in front of them and they are ten years old…. Good luck!!! So, I want their attention and I'm their teacher and I want to give them feedback, I want to give them a little tidbit, a morsel of information. Immediately I learned from one of the other students that I want to share with everyone else. On the first or second day of school, there will be those students that can look at me with the screen up, and others where their eyes are on the screen. You learn real quickly. And I say, now listen boys and girls, we now live in an environment where you've got a screen in front of you all day. So, if I want your attention or someone else does, now if you don't have the attention control to look at the teacher or the other student in the class that's talking, then what would be the logical consequence, oh "close the Chromebook", simple. So those little kinds of procedures that you develop as a class because every year is different, but in the end, you want to be a community of learners. You don't want a community of followers or doers. I don't want to assign something and they do it, no, we want a community of learners. We just kind of muddle through it.
Uses creative work to minimize disruptive behaviors. Three participants discussed how creative activities minimized disruptive behaviors. One of the teachers shared how having students sit longer for class-wide instruction tends to lead to more disruptive behaviors: Woah that's a big question and it's a challenging thing to do with teachers, especially elementary, it's always time, time, time trying to get everything in. But I think when they are doing creative things it often takes away the disruptive behaviors. I think you get more disruptive behavior when you're expecting kids to sit still and do things all in a rigid kind of way. Especially children who have learning problems with reading and writing. I find that if it can be managed well that keeps them to a minimum.
Year to year is different, what your class is, but I think in general that they like science, they like hands-on, they like to do those things. I think you get fewer disruptive problems during that. Now there's always the exception and there's always the serious problems that can come up and those you just have to figure out how to manage.
Has sensitivity towards students to manage behavior. Two instructors stated that having sensitivity towards student's personal needs has been helpful. One teacher discussed how being mindful of the students' abilities helps guide instruction: In the beginning, it's a little challenging because I don't know the students. Once I know the students, they're... I can regroup them based on what might trigger their behavior and try to be proactive before they get out of hand. So, if I have a student who... Actually, I have a student who... has trouble focusing, and so if he was given-if I had an assignment that was long, lengthy, written-he can communicate what he knows orally, and I can write his answers down that way. Other students, same thing. If they have any attention issues, anything that's lengthy, if it's broken down into pieces, chunked, and if they can give me their answers in another medium other than pencil, that's fine with me. And that has alleviated a lot of behaviors. Especially with tedious calculations, specifically when we were finding mean, if they could explain the process to me, that was completely acceptable; because if they have any attention issues, calculating a bunch of numbers, they're going to come up with a calculation error, and it's not because they don't know what they're doing. It's just because they can't attend to a certain task for long periods of time. So, I don't... I've learned to not punish them for that. And that's alleviated a lot of behaviors because now they're starting to feel successful. Some of them have been coming into class and not giving me a hard time, where I hear some reports haven't been the same in other classes.
Has a disciplinary/behavioral support curriculum. Two teachers shared that they had a disciplinary/behavioral support curriculum to guide student behavior. One of the participants described having a social/emotional program in place at the school, "we have an excellent social/emotional curriculum called Open Circle. And these are twiceweekly meetings for fifteen minutes that teach them skills. Anything from bullying, to calming down strategies, to problem-solving, people-problem strategies, things like that." The other participant explained how the instruction for science is a strong incentive for students: Well, we have a sort of discipline program where they are given instruction as to their correct behavior, a model behavior, and if they do anything unsafe they have to be excused from the classroom. Mostly, they want to have the enjoyment of the activity, so it's an incentive to behave and they do. I have really noticed that they really want to be a part of the activity and all I have to do is say "If you're not doing what you need to do then you will have to leave" that keeps them in line. The punishment will be no school!

Qualitative Comparison
In order to examine differences in categorical support between teachers affiliated with a university network and teachers outside the network, a subset of questions and probes were analyzed. These questions and probes were selected based on the variability of responses. One question was selected from each area of the interview: support, style of teaching, STEM integration, NGSS knowledge, and attitudes towards creativity. These investigations were made for discovery purposes rather than verification purposes in order to assess whether teachers and non-affiliated teachers tended to agree or disagree across the different codes that were endorsed across interviews.
The first selected question asked participants to characterize the support they received to attend training opportunities to improve their STEM instruction. The comparison of response categories was generated which have been summarized in Table   43. The curriculum publisher provided an one time training 1 0 1 Our district has invested in supplies rather than training 1 0 1 Other; Grant covers summer training 1 1 0 Note. N = 10.
University-affiliated teachers tended to endorse their curriculum as providing them with training and collaboration with other teachers within their school compared to their non-affiliated peers. The endorsement of the remaining categories was relatively similar by teachers from both groups.
The second selected question asked participants to characterize their approach to teaching STEM content. The comparison of response categories was generated and have been summarized in Table 44. Note. Total is not equal to 10 (100%) given that participants endorsed more than one response category.
Non-affiliated Teachers tended to endorse their teaching style as constructivist or inquiry-based more so than the affiliated teachers. Endorsement of the remaining categories was relatively similar between both groups.
The third selected question asked participants to characterize their approach to integrating STEM content. The comparison of response categories was generated and has been summarized in Table 45. Note. Total is not equal to 10 (100%) given that participants endorsed more than one response category.
Affiliated Teachers tended to endorse an interdisciplinary style more than their peers outside of the network. Non-Affiliated teachers tended to endorse using both styles of integration more and supported interdisciplinary integration as a superior approach than their affiliated counterparts. Endorsement of the multidisciplinary approach was relatively similar between both groups.
The fourth selected question asked participants to describe their familiarity with the NGSS. The comparison of response categories between affiliated teachers and nonaffiliated teachers was generated and has been summarized in Table 46. Limited familiarity with the NGSS standards because one-on-one time was provided to explain how to use them 1 0 1 Note. N = 10.
Non-affiliated Teachers tended to endorse being very familiar with the NGSS more than their affiliated peers. Affiliated teachers also tended to be not very familiar with the NGSS compared to their colleagues out of the network. Other levels of familiarity were relatively similar across both groups. However, the non-affiliated teachers had the most variability in their knowledge of the NGSS standards with some members having a great deal of familiarity and others not having received any training.
The fifth selected question asked participants how they balance opportunities for creativity with the potential for disruptive behavior. The comparison of response categories was generated and has been summarized in Table 47. Uses creative work to minimize disruptive behaviors 3 2 1 Sensitivity towards students and their personal needs to manage the classroom 3 2 1 Has a disciplinary/behavioral support curriculum to guide student behavior 2 1 1 Note. Total is not equal to 10 (100%) given that participants endorsed more than one response category.
Both groups of teachers had similar levels of support for setting supportive boundaries for students. Endorsement of the other categories was also relatively similar between both groups. Overall, both groups of teachers tended to view student creativity and behavioral expectations similarly.

DISCUSSION
The current study used a quantitative analysis to examine differences in NECAP achievement performance between schools that belong to university STEM network and schools that do not over three time points. This study also used semi-structured interviews to examine supportive factors, styles of teaching, STEM integration, NGSS standards, and attitudes towards creativity among teachers from university-affiliated schools and schools outside the network. The interviews yielded rich descriptions regarding the experiences of teachers in the state in New England. In addition, teachers provided valuable information that may be of use to other educators in the elementary and middle school level.

Academic Achievement Differences
In order to address the first research question about differences in academic performance of students within an integrated network compared to other schools, a series of t-tests were used to compare scores across 2007-2008, 2009-2010, and 2014-2015. The results indicated that literacy of science knowledge among students did not seem to differ between the university-affiliated schools and those without an affiliation across the three observed years. This trend was similar for inquiry scores across the three years; however, inquiry scores did significantly differ in the last observed year of 2015. The NGSS was released in 2012, and university STEM network was an early adopter of these standards.
Perhaps the adoption of the NGSS and commitment to its integration is what led university-affiliated schools to outperform their matched peers at the 2014-2015 academic year on inquiry practices, but this hypothesis was not directly assessed.

Quantitative Summary
Although science literacy was not different among students from either group (but with a notable trend in the final year), inquiry skills were found to be different in that final year. In other words, students from both school curricula learned content material at least as well as each other. However, university-affiliated students either performed similarly or outperform their peers when it came to using scientific skills or the process of scientific thinking. An attempt was made to investigate possible reasons for this difference by interviewing teachers from schools in the university-affiliated curriculum compared to schools outside of the network in the following key areas: support, style of teaching, STEM integration, knowledge of NGSS, and attitudes towards creativity.

Supportive Factors
Interviewees were asked about the support they received at their school and were subsequently probed about different sources of support. At least half of participants endorsed the support they received as excellent. Seven of the ten participants stated that there was a commitment to professional development and support for new training and curriculum from their district administration. When asked about support from their site administration, support was not as strong, with four of the ten participants stating that there was coverage for training, professional development, and resources and another four participants stating that their site administrators were only nominally involved. In terms of support for kits, activities, and supplies, seven of the ten participants stated that there was a high level of support from their district, and four specifically mentioned a specific university-affiliated program as providing a high level of support.
When asked about specialists within their district, most participants reported to not have a science specialist or a math specialist, but four of the ten mentioned having a technology specialist and recognized this as a beneficial source of support. Half of the participants stated that they received opportunities to go outside the district for professional development. When specifically asked about training opportunities within their district, responses were more varied with only two teachers specifically mentioning the training opportunities provided by a university-affiliated program. Support from other teachers and staff at their school was rated by four teachers as being quite strong throughout the school by four teachers. Differences of support between universityaffiliated teachers compared to other teachers outside the network revealed that affiliated teachers endorsed their curriculum as providing them with more frequent training and opportunities for collaboration within their school.  (Scott, 2012), having a supportive leadership from the administration (Bryk, Sebring, Allensworth, Luppescu, & Easton, 2010), and having a school culture for learning and safety (Bruce- Davis et al., 2014).

Teaching Styles
Interviewees were then asked about their approaches to teaching STEM content and were probed about their curriculum, strengths, and weaknesses of their curriculum, style of teaching, an example of a classroom activity, approaches to differentiated instruction, and any personal factors that have helped them teach. Eight participants endorsed an inquiry-based/investigative/constructivist approach to teaching STEM content suggesting a strong interest in students exploring STEM content and learning through their experiences with the material. These responses were similar to a follow-up probe that specifically asked about whether teachers leaned more towards direct instruction or constructivist teaching, with nine out of ten instructors supporting an experiential/constructivist/student-centered approach. showcase. This would imply that teachers tend to create their activities based on student feedback and interests that fit their curriculum.
Eight of the ten participants mentioned differentiating instruction for their students based on the student's learning and five of the ten specifically mentioned using grouping strategies based on different student abilities. Finally, when asked about personal factors that have helped them teach, the teachers had a variety of responses, but half stated that having patience or an appreciation for individual differences was a helpful factor. Other less frequently endorsed responses included previous experiences, not being afraid to learn alongside students, having a sense of humor, having a problem-solving orientation, or being an older teacher with more time available.
When investigating the differences in style of teaching between universityaffiliated teachers compared to other science classes, non-affiliated teachers tended to endorse their teaching style as constructivist or inquiry-based more so than the affiliated teachers. Endorsement of the remaining categories was relatively similar between both groups. This may suggest that teachers from affiliated schools may have been relatively unaware of the constructivist approach of their curriculum. However, when asked in a subsequent probe whether teachers specifically lean towards direct instruction or constructivist approaches, all but one teacher identified as having a more constructivist approach.
Overall, the findings for teaching yielded that teachers strongly endorsed having a constructivist or project-based approach to teaching. Teachers cited the strengths of their respective programs as improving student learning, but that the logistics of organizing materials and supplies for activities were the most significant drawback. Also, most teachers tended to make up activities as the semester progressed for their students, which may have contributed to the frequently cited weakness of organizing materials.
Differentiated instruction was acknowledged as important and the interviewed teachers discussed approaching working with the different ability levels of their students with patience and an appreciation for individual differences. Lieberman (1995) suggested that teacher involvement in a new curriculum was an important predictor of its success. In regards to adopting a new curriculum, it has been advised to adopt a continuous learning approach where the material is reinforced among educators across multiple training days rather than a one-off training day (Sugai & Horner, 2006;Stokes & Baer, 1977). It is necessary that a district has the resources to support ongoing training with their teachers rather than a one-time event, which typically fails to result in successful implementation. National Council of Teachers of Mathematics, 2000;National Research Council, 1996;2000). Project-based approaches to teaching STEM have also been found to be effective at fostering learning among students (Harris et al., 2015;Varelas et al., 2014). Most teachers interviewed supported using an inquiry-based approach, which indicated that these STEM teachers were following best practices.

STEM Integration
Participants were provided with a description of multidisciplinary and interdisciplinary approaches and then asked to describe their approach to integrating STEM content and were probed about how math, science, engineering, technology, and other subjects are integrated into their courses. Four participants identified as having an interdisciplinary approach to integrating topics and felt that the different subjects could not be teased apart. Interestingly, three participants identified as having a multidisciplinary approach but aspired to be more interdisciplinary.
When asked about how science and math were integrated, all but one participant stated that math concepts such as measures of central tendency tended to be involved in both areas and eight of the interviewees also mentioned how data collection and graphing were frequently involved in both. Participants were then probed about engineering, and surprisingly all but two teachers mentioned having activities that included a design and re-design component. Technology was also commonly integrated as well. Eight participants stated that each of their students had a personal electronic device to use in the classroom such as a Chromebook and also that online websites for simulations were used as well. Finally, when asked about how other subject areas were integrated, the most popular response by all but one teacher was that English Language Arts was strongly related to their coursework. Many of the teachers described the importance of having their students practice reporting their findings through various journals or worksheets or even through writing prompts. Art, music, social studies, and physical education also were included to a lesser degree.
When investigating the differences in STEM integration between universityaffiliated teachers compared to other teachers, university-affiliated teachers tended to endorse an interdisciplinary style more than their peers outside of the network. Nonaffiliated teachers tended to endorse using both styles of integration more and supported interdisciplinary integration as a superior approach than their affiliated counterparts. It could be the case that the university-affiliated curriculum and the support of the program helped teachers feel more comfortable using an interdisciplinary approach to integrating STEM material.
Overall, teachers mainly approached integrating different STEM topics through an interdisciplinary style, combining materials in a way that it would be difficult to divide the activity back into separate subjects. Teachers discussed that science, technology, engineering, and mathematics were commonly integrated along with English-language arts. In the article by Liederman and Niess (1997), from whom the chicken soup versus tomato soup question was taken, it was described that students who were taught through a multidisciplinary/chicken noodle soup method would be able to identify science from other disciplines in a particular activity. The interdisciplinary/tomato soup approach fits closer to the NGSS's cross-cutting concepts, which seek to link several disciplines together in a single activity such that students would not be able to tell them apart (Mobley, 2015). Once again, most teachers ended up endorsing or aspiring for an interdisciplinary approach, which aligns well with the NGSS despite the variability in familiarity with NGSS by the interviewed teachers. Curricula that make the effort to integrate all subjects help students have opportunities to make sense of the world (Basham, Israel, & Maynard, 2010). Teachers in this study provided a variety of disciplines that they integrate, including physical education, music, and art. Making efforts to increase integration beyond STEM is important, especially as it relates to English-language arts to provide students with improved scientific literacy and better expression (DePaul Science Working Group, 2013). Teacher training that focuses on integration activities has also been found to be helpful for fostering teachers' sense of comfort and self-efficacy with integration with STEM content (Adams, Miller, Saul, & Pegg, 2014).  2018). Regardless of the source of the science assessment, participants acknowledged that standardized testing was one of the reasons that there was such a strong level of support from the district to improve STEM education. The accountability of student's scores created the need for support from the district, although three of the teachers mentioned that the testing has a negative impact on the students in terms of stress.

Knowledge of NGSS
When investigating the differences in attitudes towards the NGSS between university-affiliated teachers compared to other teachers, non-affiliated teachers tended to endorse a higher level of familiarity with the NGSS more than their affiliated peers.
However, as a whole, the non-affiliated teachers had more variability in their knowledge of the NGSS standards. Two of the non-affiliated teachers mentioned being writers for drafts of the NGSS whereas another teacher mentioned that NGSS decisions were handled at a district level and did not receive training. The teachers that belong to a university-affiliated network may not be aware that the content they are learning is aligned with NGSS. However, the content that they are teaching appears to be having an impact on their students' inquiry skills. Also one of the teachers in the universityaffiliated group was primarily a math teacher who taught STEM topics during summer classes and after-school groups. As a math teacher, she may not have had the same level of exposure to the NGSS as would be expected by science teachers in the universityaffiliated network.
The NGSS was released in 2013 and functions as a present standard for K-12 science education and engineering practices. The U.S. education is a complex system that is affected by decisions at the national, state, and local level. Bybee (2014) described three main routes that student learning is influenced including curriculum, teacher development, assessment, and accountability. The NGSS was designed with coherency in mind and that student understanding would progress from K-12 through the use of grade bands. Three core dimensions of the NGSS include science and engineering practices, cross-cutting concepts as a way to bridge other disciplines, and disciplinary core ideas that reduce the volume of content for students allowing them to go more deeply into key principles (Bybee, 2014). However, policy change and standards by themselves may have a limited impact on teaching and professional development is needed to accompany this change.
Another study of NGSS was conducted by Nollmeyer and Bangert (2017), who assessed elementary teachers' perceptions of their understanding of the NGSS framework. Their study yielded five factors including science and engineering practices, cross-cutting concepts, disciplinary core ideas, integration of the three dimensions, and best practices/connections to Common Core. The responses by interviewed participants in the current study aligned the most with science and engineering practices and integration. A couple of teachers also specifically discussed cross-cutting concepts and disciplinary core ideas. Interviewed teachers did not discuss best practices or connections to Common Core. Bybee (2014) recommended that training programs attempt to align themselves more with the NGSS and replace components as needed, akin to replacing parts of a vehicle. It was argued that science-teacher educators should have the most knowledge and understanding of NGSS in order to have the largest impact on student learning.
Another option would be to reform science-teaching education based on NGSS, which would be similar to purchasing a new vehicle. However, Bybee acknowledges this approach as being relatively rare due to the costs associated with designing new programs. The preparation of classroom teachers, as aligned with NGSS would include basic competencies with science literacy, cross-cutting concepts, and engineering practices as well as the integration of all three practices. Bybee (2014) goes further to recommend certain personal qualities such as having adequate personal relations with students, and enthusiasm for teaching.

Attitudes Towards Creativity
The interviewed teachers were asked about the importance of creativity in the classroom and then probed about what opportunities they have provided, and how they manage behavioral issues in the classroom that might co-occur with students having more autonomy. All but one participant supported that creativity was very important, with the remaining individual reporting not as much creativity in the classroom as one would like.
Teachers recognized the importance of creativity in different ways, such as sharing that it supports students' self-expression, leads to life-long learning, or fostering creative thinking.
Teachers were then asked to describe how they included opportunities for creativity in the classroom and six of the ten participants described using an approach that allowed their students freedom to design a project; half of these six participants went on to specify including an engineering component that required a redesigning component.
Another four participants mentioned allowing students different ways to demonstrate their learning.
Finally, participants were asked about their classroom-management skills during creative activities. Six of the ten teachers specified that they set clear but supportive boundaries at the beginning of the year and three teachers shared that creative activities actually reduced disruptive behavior. When examining the differences in teacher attitudes towards scientific creativity between university-affiliated teachers and other teachers, both groups of teachers had similar levels of support across all categories suggesting no real difference in their views on creativity.
Overall, teachers tended to view student creativity as being very important for fostering creative thinking, improving self-expression, and as being a core component of engineering activities. Teachers also shared that creative activities tended to reduce behavioral disturbances and emphasized the importance of setting clear boundaries and behavioral expectations.
Although creativity is often recognized as being a valuable resource, teachers have been found to value creative students less due to an association with nonconformity, impulsivity, and disruptive behavior (Cropley, 1992;Dawson, 1997;Scott, 1990;Torrance, 1963). However, this study found that the interviewed teachers fostered creativity through a variety of means and that having creative assignments often led to a reduction in unwanted behavior. It could be the case that classrooms that emphasize learning through creating products are less likely to see creative students as a nuisance as these students are allowed to channel their creativity into their work while still learning important concepts.

Qualitative summary
School support was recognized by both groups as being important and the strongest sources of support identified were from district administration and other school staff. Site administration was not recognized as being as supportive. Leithwood, Harris, and Hopkins (2008) recommend that principals can become important leaders for improving a commitment to STEM education through their advocacy, allocation of funds, and organizing professional development opportunities. Most teachers across both groups identified that an inquiry-based/investigative/constructivist approach described their teaching style, which was consistent with best practices. In terms of integration, studies have demonstrated that integrated curricula produce results that are either as well or better than traditional instruction (Czerniak, Weber, Sandmann, & Ahern, 1999;Hill, Kawagley, & Barnhardt, 2006;Vars, 2001;Weilbacher, 2001). Integrated learning also leads to improved interest, engagement, and motivation (Czerniak, Lumpe, Haney, & Beck, 1999;Erlandson & McVittie, 2001;Hinde, 2005). The findings from this study demonstrate that teachers of STEM content across both groups take integration seriously, which is beneficial for student learning.
In terms of NGSS, efforts could be made to improve teachers' understanding of key concepts such as cross-cutting concepts, engineering practices, and disciplinary core ideas. Most teachers mentioned making an effort to include engineering topics into their courses, but few participants mentioned the other components during their interviews.
Finally, teachers were able to support their claims of creativity being important by providing several vivid examples of creative assignments in their classroom. They also reflected how opportunities for creativity lead to less behavioral disruption. Moreover, they emphasized the importance of having behavioral expectations established on the first day of school.

Overall Summary
It appears that despite the significant difference in inquiry skills among students from university-affiliated schools in the year 2015, attempts to discern an answer for this finding at the teacher level were limited. It was offered that an earnest attempt at integrating the NGSS might have contributed to increases in student learning. However, differences in teachers' familiarity with NGSS or their attempts to align to these standards did not seem too different between the two groups. Possible reasons for the difference may come elsewhere, such as differences in curriculum, or support or from site and district administration. Teachers from both groups acknowledged the importance of professional development; however, university-affiliated teachers tended to endorse their curriculum as providing them with training and collaboration with other teachers within their school compared to their non-affiliated peers. These teachers have more frequent training in a supportive network that utilizes teacher leaders to teach how to perform upcoming class activities and how to troubleshoot potential problems. The network component of university affiliation could be a contributing factor to the differences observed at the student level, because teachers may feel more secure in their teaching by belonging to a larger supportive network. This study suggested that evidence that the benefits of a membership to university STEM network includes teachers and also extends to improved performance among students' inquiry skills.

Limitations
An important limitation to address is the sample size of the quantitative study. One important consideration is that the comparison of qualitative data between the groups was exploratory and each group only consisted of five teachers. The findings should be considered with a great degree of caution given the small sample size of teachers and the average difference in the number of years in a classroom between the two groups. A larger sample would have potentially provided additional information not covered by this study, but it is important to note that qualitative research typically has a small number of participants (Patton, 2002;. Another potential limitation of the qualitative comparison would have been differences endemic to teaching elementary school science versus middle school level science. Findings from the overall qualitative study should be considered with greater assurance than the exploratory investigation of differences between the affiliated and nonaffiliated teachers. Exploratory approaches are often more useful for generating hypotheses rather than confirming hypotheses. The main objective of the qualitative comparisons was to focus on the discovery of information rather than verification of existing differences. Another limitation is that this study focused specifically on a state in New England, it may have limited utility generalizing to other states.

Directions for Future Research
As states in the New England area transition away from using the NECAP as a standardized measure of science literacy and inquiry skills, an attempt could be made to continue to compare student performance with the new science measure to monitor the impact of having an affiliation with a university on science literacy and inquiry skills.
Another worthy comparison would be to compare performance of schools from different states in New England with affiliations to universities performance to other schools without affiliations to examine differences to other curricula used in other states by standardizing achievement scores before analyzing them statistically.
This study analyzed differences at the school level and at the teacher level; it may be of use to examine how site and district administrators differ in their support of STEM education given their positions of authority. It is important to mention that an attempt was originally made by the researcher to examine differences in scientific creativity among students from the different groups examined in this study; however, every attempt to reach out to a superintendent was responded to negatively or not at all. It might be worthwhile to examine student -level differences if a researcher had a larger connection with administrators and stakeholders at a number of schools within New England. It would also be interesting to inquire how teachers support girls and students of color through their experience with STEM content given that these groups tend to have higher rates of dropping out of the pipeline as they get older.
Finally, future research could also monitor teacher's knowledge of NGSS using the measure created by Nollmeyer and Bangert (2017). This measure, the New Framework of Science Education-Survey of Teacher Understanding (NFSE-STU), would be useful for monitoring the quality of professional development that is specifically oriented towards improving teachers' NGSS literacy. It appears to be an effective instrument that would be useful to curriculum developers and those responsible for monitoring professional development. b.
On the spectrum of direct instruction versus constructivist teaching, a more experiential form of learning, where do you feel your style of teaching leans towards? c.
Could you please provide an example of a classroom activity or assignment that matches your style of teaching? d.
How do you approach differentiated instruction? e.
What personal factors or qualities have helped you teach?
Thank you for sharing that information with me. Now, I would like to ask you some questions about your opinions of teaching STEM topics in your class.

STEM Integration
3. Two words that have been frequently used in the literature to describe integration are "multidisciplinary" and "interdisciplinary." Lederman and Niess (1997) used the metaphor of chicken noodle soup versus tomato soup to explain the fundamental differences between multidisciplinary and interdisciplinary approaches to integration.
In their description, multidisciplinary was characterized as a bowl of chicken noodle soup, where each ingredient maintained its identity without direct mixture, yet still came together to make a whole.
On the other hand, tomato soup represented an interdisciplinary approach to integration, in which all ingredients/subjects were mixed together and could not easily be separated.

How would you describe your style integration?
Probes include: a.
How are science and mathematics related in your classes? b.
Do you include engineering topics in your teaching of science and/or mathematics? If so, please describe. c.
What efforts have you made to integrate technology into your course? d.
How have you integrated other subject areas besides science, technology, and mathematics? Thank you for responses. Now, I would like to ask you some questions about the NGSS Standards...

Describe your familiarity with the NGSS Standards?
Probes include: a.
What efforts have you made to orient your class towards the NGSS Standards? b.
How have you changed your curriculum in response to the NGSS Standards? c.
How has Standardized testing affected your curriculum? d.
Has your district addressed NGSS or required you to teach it? e.
Has your district provided any training/professional development in NGSS? Thank you for your time.
What will be done: If you decide to participate in this study, you will take part in an audiotaped in person interview lasting about 30-45 minutes.

Risks or discomfort:
The possible risks or discomforts of the study are minimal.

Benefits of this study:
If you choose to participate, your answers will help increase the knowledge base about teachers that provide coverage of STEM content and how to best support them.

Confidentiality:
Your participation in this study is strictly confidential. This means that none of the information will identify you by name and only pseudonyms will be used. All data will be maintained in a locked and secure facility.

Decision to quit at any time:
If you decide to take part in the study, you may choose to withdraw your participation at any time. There are no consequences for not participating in the study and you are free to refuse to answer any questions.

Rights and complaints:
If you have any questions, or if you are not happy about the way in which this study is conducted, you may discuss your complaints with Ryan Holt at (909)  Enclosed are two copies of this consent form. Please keep a copy of this form and return a signed copy to Ryan Holt via e-mail to ryan_holt@uri.edu.
Thank you for your time and help in this study. Your assistance is greatly appreciated!