UNDERSTANDING TEACHERS’ CONCURRENT KNOWLEDGE OF ASSESSMENT LITERACY AND CURRICULUM-BASED MEASUREMENT

As a result of the standards-based movement in education it is important for teachers to be knowledgeable about and understand the use of assessment in the general education setting. Prior studies have investigated teachers’ understanding of sound assessment known as assessment literacy. This study explores teachers’ concurrent knowledge of Curriculum-Based Measurement and sound assessment, as measured by the Assessment Literacy Inventory. More specifically, the extent of the relationship is examined by the Pearson product-moment correlation coefficient. Additionally, the relationship between training and knowledge about CBM is described. Results of the current study revealed a moderate, positive relationship between scores on measures of knowledge about CBM and assessment literacy. It is unclear if training is related to knowledge about CBM or assessment literacy. Implications of these results and the conceptualization of assessment literacy, as well as limitations of the study, are discussed.


Statement of the Problem
The current study explores to what extent teacher assessment literacy, as measured by the ALI (Mertler & Campell, 2005), is related to teachers' knowledge about CBM. The adoption of CBM as a tool for assessment in the general education classroom has increased since the promotion of accountability and evidence-based assessing students' basic skills (Deno. 1985). Now, general education teachers are expected to utilize these measures to make instructional decisions.
In schools, testing and assessment are common practice. The dissemination of the 1990 Standards for Teacher Competence in Educational Assessment of Students incited interest in examining teachers' knowledge and understanding of assessment, and in 1991, Richard Stiggins coined the term 'assessment literacy' (AFT, NCME, NEA, 1990;Stiggins, 1991). According to Stiggins (1991), an individual who is literate in assessment activities has the knowledge and skills to administer, score, and interpret assessments with high quality. In general, the literature has indicated low levels of assessment literacy and gaps in both inservice and preservice teachers' knowledge (Plake, Impara, & Fager, 1993;Mertler, 2003;Volante & Fazio, 2007).
Regardless of the empirical evidence supporting the reliability and validity of CBM (Good & Jefferson, 1998) and the positive effects on student achievement when the measures inform data-based decision making (Stecker, Fuchs, & Fuchs, 2005; Tindal 2013), the implementation of these measures by teachers in schools varies (Ysseldyke & Bolt, 2007;Bolt, Ysseldyke, & Patterson, 2010). Recent research has indicated that variability of the implementation and interpretation of CBM is not related to teacher characteristics including teaching experience, and years of experience with CBM (Wayman et al., 2011). One possible explanation for the variability of implementation is the function of different levels of knowledge and understanding about CBM.
In a data-driven system, where decisions about student progress and achievement are based on assessment results, it is important that teachers are assessment literate. To make valid decisions, teachers need to use assessments with fidelity, which means they need to be knowledgeable and confident interpreting results in addition to the implementation. The current study explores the relationship between an established framework for conducting sound assessment and more specific assessment information comprising CBM. The nature and specifics of the identified relationships will provide implications on how training for adoption and use of CBM could be more effective.
The following primary research question was examined: 1. How is teacher assessment literacy, as measured by the Assessment Literacy Inventory, related to teacher knowledge of Curriculum-Based Measurement?
The following secondary research question was examined: 1. To what extent is the amount of teacher training in assessment related to knowledge about Curriculum-Based Measurement?

Gap between Research and Practice
The goal of educational research is to influence practice by discovering and promoting the use of evidence-based practices. As educational research becomes more advanced, dissemination of effective practices into the field setting is integral. After a direct relationship between a program, or intervention, and positive student outcomes is established, then the degree to which a program generalizes to a typical school setting may be explored (Dzewaltowski, Glasgow, Klesges, Estabrooks, & Brock, 2004). However, it has been documented that educational research has been undervalued by teachers because of insufficient communication of how to replicate, in natural settings, those practices specified in journal articles (Greenwood, 2001). As a result, oftentimes research is not translated into practice as intended by the researchers (Greenwood & Abbott, 2001). A benefit of research conducted in the field setting is that it may elicit if and to what extent modifications to the implementation are necessary while preserving the outcomes (Shulte, Easton, & Parker, 2009). Potential consequences of a gap between research and practice may be the inaccurate use of or the absence of evidence-based practices in schools.
Evidence-Based Practices. Current school reform efforts focus on the use of evidence-based practices to improve the quality of core programs, targeted programs, instructional strategies, and professional development to enhance student academic achievement. National laws and policies, including the No Child Left Behind Act (NCLB, 2002) define evidence-based practice as "research that involves the application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs" (NCLB, 2002, p. 540).
Methods that are supported by scientifically based research may have a greater opportunity of impacting student learning and achievement because a relationship with positive student outcomes has been documented (Dzewaltowski, Glasgow, Klesges, Estabrooks, & Brock, 2004). Typically, evidence-based practices are manualized, or at the least have specific procedures detailing the implementation of the practice. To maintain the relationship between evidence-based practice and positive student outcomes, it is essential that the practice is implemented as intended by the researchers (Sanetti & Kratochwill, 2009). Such implementation is known as "treatment integrity".
Treatment integrity. Treatment integrity refers to the extent to and level of skill in which the procedures of an evidence-based practice are implemented (Shulte, Easton, & Parker, 2009). This is dependent on how well the implementers were trained and how well they understood the training. Currently, a universal research based method does not exist for documenting adherence to implementation procedures (Shulte, Easton, & Parker, 2009). Without this information it cannot be determined if student outcomes are a result of the evidence-based practice or how the evidencebased practice has had to be altered in order to obtain results (Sanetti & Kratochwill, 2009).
The intricacies of a school setting increase the risk of implementing a treatment inaccurately (McIntyre, Gresham, DiGennaro, & Reed, 2007). Teachers have numerous responsibilities in the classroom that may limit their attention to treatment integrity, including but not limited to setting demands, classroom management, behavior management, and limited resources. Drifting from a protocol may interfere with outcomes if such drift results from factors unrelated to student needs (Sanetti & Kratochwill, 2009). However, some adaptations may be necessary to accommodate the context or individual. The American Psychological Association (2005) considers adaptation to be a natural part of implementing evidence-based practices in schools, if the modifications are determined by an individual with expertise in the area. Individuals who implement evidence-based practices should be knowledgeable about how to implement the practice and how to implement it with high quality.

Response to Intervention/Instruction
The primary issues noted, regarding a research to practice gap, the need for evidence-based practices in schools, and the importance of treatment integrity for obtaining expected results, are relevant to the contemporary area of educational practices known as Response-to-Intervention (RtI). In 2004, IDEA incorporated a new regulation in regard to identifying students with specific learning disabilities. The new regulation permits local education agencies to consider a student's response to scientifically-based intervention as a procedure for determining eligibility for a specific learning disability (P. ). In other words, the national policy encourages schools to use evidence-based practices for instruction and intervention. Further, documentation of how the practice is implemented (i.e. treatment integrity) and student progress during implementation is necessary to determine students' response to the treatment.
RtI is a data-driven prevention, intervention, and problem solving model (Burns & Gibbons, 2008). In essence, the goal of RtI is to increase the number of students who are successful in the general education setting, while simultaneously decreasing the number of students referred to special education. One core feature of RtI is data-based decision making; at each stage of the problem-solving model data are used to determine students' response to instruction and/or intervention (Fuchs & Fuchs, 2006). Academic assessments are one method for collecting data. Ysseldyke, Burns, Scholin, and Parker (2010) review the characteristics of instructionally valid assessments within an RtI framework. First, assessments should be precisely matched to student individual needs. In other words, the evidence collected needs to provide information specific to the instructional goals or outcomes for each student. Second, all students' progress are monitored on a frequent basis, from three times a year for students attaining grade-level expectations, to two times a week for students receiving intensive intervention services. As such, multiple, equivalent forms are necessary.
Finally, to perform the function of progress monitoring, or to determine students' responsiveness, it is essential that assessments are sensitive to change. Data collected by an assessment method that possesses these characteristics would support decisions made within an RtI framework.

Assessment
Assessments are measures used to determine what students know and are able to do before, during, and after instruction (Green & Johnson, 2010). According to Pellegrino, Chudowsky, and Glaser (2002-3), first, information is gathered and second, information is used to make inferences about students' knowledge and understanding. An example of one type of assessment teacher's use is an end-of-unit exam, or a chapter test. The purpose of making judgments about student learning is to enhance future outcomes.
Research has indicated that teachers may spend a large portion of their instructional time engaged in assessment-related activities (Stiggins 1991). Given the amount of time spent on assessing students and the required use of assessments within an RtI framework, it is important for teachers to be aware of and understand why they are assessing, how the assessment is administered and scored, how to interpret the results, and how to use the results for decision making. Under the NCLB act, and for the purpose of data-based decision making within an RtI framework, a well-supported, evidence-based approach to assessment is warranted. The extent to which teachers are knowledgeable about assessments has been discussed in the literature as 'assessment literacy'.
Assessment Literacy. Assessment literacy is an understanding of the basic principles of sound assessment (Stiggins, 2002). In other words, an individual who is literate in assessment activities has the knowledge and skills to administer, score, and interpret assessments with high quality. The primary impetus for examining teacher's assessment literacy was due to the dissemination of the 1990 Standards for Teacher Competence in Educational Assessment of Students (AFT, NCME, & NEA, 1990), delineating the competencies necessary for teachers to conduct sound assessments.
Research following the dissemination of the standards explored teachers' level of assessment literacy using multiple-choice content related measures, self-efficacy measures, and a combination of the two measures. In general, the literature has indicated that preservice and inservice teachers have low, insufficient levels of assessment literacy (Plake, Impara, & Fager, 1993;Mertler 2003;Gotch & French, 2013). More specifically, studies using multiple-choice content-related measures, directly associated with the standards developed in 1990, yielded common areas of weakness including developing assessment methods appropriate for instructional decisions (Quilter & Gallini, 2000;Mertler 2009), developing valid pupil grading procedures which use pupil assessments (Mertler 2003(Mertler , 2009, and communicating assessment results to students, parents, and other audiences (Plake, Impara, Fager, 1993;Mertler 2009). Additionally, two common strengths emerged including choosing assessment methods appropriate for instructional decisions and administering, scoring, and interpreting the results of both externally-produced and teacher-produced assessment methods (Plake, Impara, & Fager, 1993;Mertler, 2003).
However, further analysis of teachers' interpretation of results from externallyproduced assessment methods revealed teachers' level of knowledge is inconsistently developed within this domain (Gotch & French, 2013).
Further, studies comparing inservice and preservice teachers' level of assessment knowledge have reported mixed results, suggesting it is unclear if teaching experience in the field influences assessment knowledge (Mertler, 2003;Alkharusi, Kazem, Al-Musawai, 2011). In fact, two studies that examined the effects of teaching experience on knowledge of assessment reported mixed conclusions. Alkharusi, Kazem, and Al-Musawai (2011) found that teachers with less than seven years of experience had higher levels of knowledge than inservice teachers with more than seven years of experience, and further, inservice teachers who completed a preservice course in educational measurement had higher levels of knowledge than inservice teachers who did not complete a preservice course in educational measurement.
Conversely, results from Gotch and French (2013) indicated no relationship between the number of years of teaching experience and knowledge or the completion of professional development in educational measurement within the past three years and knowledge.
The Assessment Literacy Inventory is one instrument that was developed to assess the strengths and weaknesses of teacher's application of the seven standards for teacher competence in educational measurement. The instrument includes five scenarios mimicking real-life experiences with assessment in the classroom (Mertler & Campbell, 2005). The internal consistency of the measure, r KR =.74, indicates acceptable reliability within the measure (Mertler & Campbell, 2005). The information gathered from this measure may suggest areas for refining knowledge and skills. A limitation of this measure is the focus on teacher-made assessments because the inventory does not address adherence to the guidelines for valid instructional assessment within an RtI framework (Ysseldyke, Burns, Scholin, Parker, 2010).
Based on the assessment literacy literature, it can be concluded that teachers' have variable levels of assessment knowledge and inconsistent areas of strengths and weaknesses have emerged. It is possible that the noted variability may be a result of inconsistencies between assessment requirements in teacher education programs, state standards for assessment education, and the culture of schools. More than a decade of research on assessment practices, in addition to standards-based reform efforts, led the Assessment Training Institute to develop an updated list of competencies necessary for teachers to understand, to conduct sound assessment. These competencies include: 1) assessment processes and results serve clear and appropriate purposes, 2) assessments reflect clear and valued student learning targets, 3) learning targets are translated into assessments that yield accurate results, 4) assessment results are managed well and communicated effectively, and 5) students are involved in their own assessment (Stiggins, Arter, Chappuis, & Chappuis, 2004).
Curriculum-based measurement (CBM) is one approach to assessment that possesses the characteristics necessary to fit within an RtI framework (Ysseldyke, Burns, Scholin, Parker, 2010;Ball & Christ, 2012). Evidence suggests that CBM is precise, frequent, and sensitive to change (Shinn & Bamonto, 1998;Deno, 1985). A limitation of the assessment literacy research is the exploration of teachers' knowledge and understanding of CBM. This information would be particularly revealing because CBM was originally developed for use by special education teachers and specialists (Deno, 1985). Special education teachers and specialists typically have more in depth training in educational measurement than general education teachers. As a result, problems with treatment integrity may arise when teachers use these measures to make educational decisions.

Curriculum-Based Measurement (CBM)
CBM is a standard approach to assessment that allows for the efficient measurement of skills aligned to the existing curriculum taught in a classroom (Deno 1983;Shinn & Bamonto, 1998). The technical adequacy of CBM is well-supported in the literature (Good & Jefferson, 1998). (For a detailed review of the available literature on CBM since its' conception in the 1970s, the reader is referred to Tindal (2013). Current research has increasingly focused on examining the appropriate and inappropriate uses of CBM for determining students' response to instruction and intervention (Ball & Christ, 2012). More specifically, there has been criticism about the use of CBM for the purpose of progress monitoring (Ardoin, Christ, Morena, Cormier, & Klingbeil, 2012). However, CBM is currently being implemented in schools to support decisions about students' response to instruction/intervention. Similar to other approaches to assessment, the use of CBM alone is not sufficient; rather data-based decision making must accompany the use of CBM (Tindal, 2013).
Following is a discussion of CBM broken down by the competencies identified by the Assessment Training Institute for conducting sound assessments (Stiggins, Arter, Chappuis, Chappuis, 2004).

Assessment processes and results serve clear and appropriate purposes.
The three intended purposes for the use of CBM include screening students for academic difficulties, measuring student growth, and recognizing a need for a change in the instructional program when the current program is ineffective (Shinn & Bamonto, 1998). The first purpose for CBM is screening, or the process of identifying the students that may need additional educational support and the students that are reaching grade level expectations (i.e. benchmarks). All students are screened, rather than a selected set of students suspected of having difficulties, to reduce the chance of overlooking any student who is at-risk. Multiple screenings throughout the year allow teachers and principals to evaluate and determine if students are making progress toward an end-of-the-year goal. Screening measures inform teachers that classrooms have a group of heterogeneous students. Scores on these measures may assist teachers in differentiating instruction to meet students' various needs. CBMs provide a quick and efficient method for assessing large groups of students (Deno, 2003).
The second purpose for CBM at the individual level is to monitor student progress. Unfortunately, it is unknown ahead of time whether or not a student will respond to a specific instructional program or intervention. A student's rate of progress during an instructional program is an indication of their responsiveness to and the effectiveness of the instructional program for that student (Fuchs & Deno, 1991). By collecting data points during instruction, or intervention, it can be determined whether the student's performance and/or learning is improving or not. If the graphed relationship between number of weeks and the students' scores does not indicate growth, as determined by the teacher or team, changes to the instruction would be indicated. However, if the student is making adequate progress, the current instructional program should remain unchanged.
In recent years, an examination of the technical characteristics of CBM for progress monitoring has been prominent in the literature. More specifically, passage equivalence (Christ & Ardoin, 2009;Betts, Pickart, & Heistad, 2009), decision rules (Ardoin, Christ, Morena, Cormier, & Klingbeil, 2012), domain sampling (Shapiro, 2013), probe-set development (Christ & Ardoin, 2009), and standard errors associated with commonly used CBM (i.e. DIBELS and AIMSWEB) (Christ & Ardoin, 2009) have been explicitly studied. An empirical base for the technical features is paramount to the standardization of CBM, and further, may lead to improvements in consistent decision making within an RtI framework. In other words, it may increase the level of confidence for making special education eligibility decisions.  (Shinn & Bamonto, 1998). Second, the severity of the problem will validate if special services are required and what level of support is necessary to meet student needs. The severity of the problem is verified by the magnitude of discrepancy between a student's measured skills and a normative measure of those same skills.
Third, CBM is used to explore solutions and set goals. To determine a goal, Shinn (2002) recommends that the local norms for the school are considered for typical performance. This means that the goal is specific to the curriculum, instruction, and the environment because it is based on students within a specific context. Fourth, after a goal is set CBM is used to determine if the student is making adequate progress toward achieving that goal. Finally, the initial screening tool will reevaluate the discrepancy between individual student performance and local normative performance to determine if the problem continues to exist or if the problem has been resolved.
Assessments reflect clear and valued student learning targets. CBMs are tools used to indicate basic skill level development (Deno, 1985). Similar to a thermometer in the medical profession, CBM measures the 'vital signs' of a student's ability in different areas of academics (Shinn & Bamonto, 1998). The purpose is to demonstrate if a problem exists using an efficient method. However, these measures are limited because they are skill based.
CBM is also known as a general outcome measurement approach to assessments.
That is to say that Fuchs and Deno (1991) describe CBM as using a long-term measurement approach. This means that the skill/s students are expected to have at the end of the year are being assessed. This is contrary to common mastery measures that assess specific skills currently being taught. Instead, the tasks students perform on CBMs require students to apply numerous subskills learned throughout the year (Hosp, Hosp, & Howell, 2007). For example, oral reading fluency is a reading achievement measure by CBM. To read fluently, some of the skills students apply are decoding, vocabulary knowledge, syntax, and background knowledge (Adams, 1990).
Poor reading fluency would indicate that the student is struggling with one or more subskills or the integration of subskills, and further diagnostic information is necessary. Results from a meta-analysis of 41 correlational studies examining CBM oral reading fluency as an indicator of reading achievement demonstrated that CBM oral reading fluency was a significant predictor of state-specific tests of reading standards, a significant predictor of third grade reading outcomes, and a significant predictor reading comprehension (Reschly, Busch, Betts, Deno, & Long, 2009).
Learning targets are translated into assessments that yield accurate results. CBM has standardized administration and scoring procedures. The purpose of standardization is to ensure that the procedures are consistent across students and testing periods, which minimizes error. Prior research has demonstrated CBM to be a valid and reliable approach to measurement (Good & Jefferson, 1998). Numerous studies have demonstrated the concurrent validity between CBMs and standardized measures of student achievement. For example, Fuchs, Fuchs, and Compton (2004) found that CBM word identification fluency was highly correlated (i. Further, a strong link between the use of CBM in the classroom and student achievement has been documented (Fewster and MacMillan, 2002;Fuchs, Fuchs, and Hamlett, 1989). For example, Fuchs, Fuchs, and Hamlett (1989) examined reading achievement for students whose teachers used CBM. The study compared three groups of teachers: one group of teachers who used CBM only for measurement, one group of teachers who used CBM for measurement and instructional decision making, and a control group who did not use CBM at all. The results demonstrate that for teachers who used CBM for both measurement and decision-making, students had higher rates of growth than the measurement only group with a medium to large effect size of .72.
No reliable differences were found between the measurement only group and the control group. A limitation of the study was the lack of random assignment. This study indicates that to achieve the most accurate results with the use of CBM, it is essential to use the results for instructional decision-making.
Assessment results are managed well and communicated effectively. The academic skill areas CBM measures are reading, written expression, spelling, and mathematics computation. CBM is valued for its sensitivity to intra-individual growth over time, repeatability of use, access to multiple equivalent forms, inexpensive cost, time efficiency, easy-to-learn administration procedures, and reliability of the measures (Deno 2003). These design characteristics were an integral feature of the development to provide teachers with a simple way to monitor student achievement and inform instruction (Deno, 1985).
To communicate results effectively, an accurate understanding of how to interpret results is necessary. Scores can be compared to two different types of normative scores: local or national. Normative scores provide a criterion for evaluating student success. National norms are available for both AIMSWEB (AIMSweb National Norms Technical Documentation, 2012) and DIBELS, and both sets of norms are based on research studies conducted with nationally representative samples. This provides an indication of how students' scores, or overall school scores, compare to other students, or schools, in the nation. Conversely, local norms provide an indication of how students' scores compare to other students in the district, or school. To understand the comparison between a student's score and a normative score, scores are attributed to a percentile rank. A percentile rank represents the percentage of scores that fall at or below the student's score. Students' scores may also be compared to a benchmark, which is a desired level of performance that indicates proficiency.
Additionally, teachers should be knowledgeable in sharing information from progress monitoring graphs. Wayman et al. (2011) examined special education teachers' interpretations and understanding of progress monitoring data using a thinkaloud approach, which means that teachers looked at a progress monitoring graph and expressed their thinking process. Results revealed strengths in teachers' understanding of goal attainment, function of the goal line, and setup of the graph and weaknesses in teachers' understanding of the slope, baseline data, and the meaning of the words read correctly measure (Wayman et al., 2011). Although these results are informative, it is important to note that general education teachers would not be expected to have the same depth of knowledge as special education teachers.
Students are involved in their own assessment. CBMs were originally developed as a formative evaluation tool (Deno, 1985). Formative assessments are administered during instruction, or an intervention. Learning occurs through the use of formative assessment for both students and teachers. Moreover, teachers can use formative assessment to enhance their instruction to meet student needs (Green & Johnson, 2010 p. 97 This study tested test the following primary hypothesis: 1. It is hypothesized that CBM related subareas of knowledge would be differentially, not similarly, related to Assessment Literacy competencies for conducting sound assessments.
The second research question was exploratory because prior research reported mixed results. Consequently, a specific hypothesis was not examined.

Participants
For the primary research question, the sample is based on respondents who completed both the CBM instrument and the ALI. The total sample size was 27 participants. Approximately 3% of the teachers who were contacted completed both instruments. Participants were 24 inservice teachers and three school psychologists from elementary schools in the Northeast region of the United States. All participants were female and the majority (i.e. 89%) of participants identified with Caucasian ethnic backgrounds. Almost half (i.e. 44%) of the sample had ten or more years of teaching experience, 33% had between five and nine years of teaching experience, and 22% of the sample had less than five years of teaching experience. More than half (i.e. 63%) of the inservice teachers identified themselves as currently teaching general education and the remaining ones identified themselves as reading, or literacy, specialists (i.e. 19%) and special education teachers (i.e. 7.4%). Further, individuals teaching in the general education setting taught kindergarten (i.e. 11%), first grade (i.e. 19%), second grade (i.e. 11%), third grade (i.e. 7%), or fourth grade (i.e. 19%). The majority (i.e. 82%) of the sample earned a Master's degree as their highest level of education. All participants taught at a school that implements CBM. Finally, half of the participants (i.e. 48%) attended two or more trainings on CBM, whether the training was during their preservice or inservice experience (see Table 1). The sample demographic characteristics are not likely representative of the general teacher population. Greater diversity would be expected within the population. Additionally,  Table 2).
Forty-seven participants completed the ALI, which is approximately 6% of the teachers who were contacted. All participants were female and taught at a school implementing CBM. The majority (i.e. 92%) of participants identified themselves with Caucasian ethnic backgrounds. Approximately half (i.e. 53%) of the sample had 10 or more years of teaching experience, 32% had between five and nine years of teaching experience, and 15% had less than five years of teaching experience. Over half (i.e. 55%) of the participants identified themselves as currently teaching in the general education setting, and the remaining ones identified themselves as reading specialists (i.e. 17%), special education teachers (i.e. 9%), other (8.5%), school psychologists (6%), and a math specialist (2%  Table 3).

Framework Conceptualizing Subdomains of Curriculum-Based Measurement
Knowledge. The CBM instrument was designed to measure teachers' level of knowledge and understanding of the use of CBM in the classroom. Three major dimensions characterize the use of CBM: administration and scoring, interpretation, and data-based decision making. Knowledge of administration and scoring procedures are necessary, but not sufficient for using assessments in the classroom. Given CBMs principal use as assessment for formative evaluation, described by Shinn and Bamonto (1998), results must be applied to instructional decision-making. To accomplish this, knowledge of interpretation and data-based decision making are necessary requisites.
Items were developed to assess knowledge and understanding specific to all three domains. Please refer to Appendix B for a visual representation of the framework for item construction.
Validation Procedures. The first version of the instrument was reviewed by an expert in educational measurement in the teacher education department, at the University of Rhode Island. Specifically, items were reviewed to determine appropriateness and relevance for inservice teachers. Following feedback and discussion two items were removed, two items were added, and nine items were reworded for clarification. Additionally, two items were modified into applied scenarios (i.e. given the provided information, what would the next step be). As a result, the second version of the instrument, a total of 48 items, consisted of 43 singleselect multiple-choice items, including four scenarios, and five multi-select multiple choice items.
Next, ten graduate students in school psychology, who completed a course in CBM and assessment one year prior, completed the instrument and were encouraged to provide feedback on the clarity and appropriateness of items. As a result one item was removed for poor wording and two items, both addressing the same term, were carefully reviewed. To review the items, numerous state department of education websites were explored to find a more appropriate term; however, it was noted that there were inconsistencies among states. The two items were retained and noted to discuss with potential respondents, inservice teachers. Further, the five items following the second data were reviewed one at a time and the teachers provided feedback regarding the clarity, appropriateness, and difficulty of the items. In general, teachers thought the instrument challenged them to think critically. More specifically, teachers suggested they were unfamiliar with terminology and theory, but they understood the basic concepts of CBM and how it is used in the classroom. In other words, teachers felt they possess the knowledge of how to implement and use CBM in the classroom, but are less familiar with why it is used. Additionally, teachers expressed concern with the length of item response options and how it influenced the overall time to complete the instrument. In response to teacher feedback, three of the multi-select multiple-choice items were removed, fourteen items were reworded for clarification, and three basic concept questions were added. With the noted revisions, the fourth and final version of the instrument consisted of 47 single-select multiple-choice items, including four scenarios, and two multi-select multiple-choice items (See Appendix C). In the present study, the internal consistency coefficient for the CBM instrument was r KR20 = .469.

Assessment Literacy Inventory
The Assessment Literacy Inventory (ALI) was designed by two experts in the field of educational assessment to be aligned with the Standards for Teacher Competence in Educational Assessment of Students (AFT, NCME, & NEA, 1990), and questions were developed to mimic real-world applications of the competencies.
The ALI is a 35-item measure of teacher's assessment knowledge. The measure includes five classroom-based scenarios that present teachers facing numerous assessment-related decisions. Participants are asked to answer seven questions for each scenario; one question pertaining to each of the seven standards (AFT, NCME, & NEA, 1990). Each correct answer is worth one point and an overall score out of a possible 35 points is awarded. When used with preservice teachers the internal consistency (KR20) of the measure is rKR20= .74, demonstrating sufficient reliability within the measure (Mertler & Campbell, 2005). The item difficulty values range from .212 to .992 providing an effective range of difficulty (Mertler & Campbell, 2005;Chase 1999). Further, the mean item discrimination value .313 demonstrates that the majority of the items on the ALI are good quality questions (Mertler & Campbell, 2005;Chase, 1999). Further, the ALI was a modified, shorter and easier to read, version of the Teacher Assessment Literacy Questionnaire (TALQ; Plake, Impara, & Fager, 1993). Prior studies employing the TALQ with inservice teachers have reported internal consistency coefficients ranging from r KR20 = .5 (Quilter & Gallini, 2000) to r KR20 = .57 (Mertler, 2003). Moreover, the first, original study using the TALQ (1993) reported an internal consistency coefficient of r KR20 = .54, which was based on a nationally represented sample of inservice teachers (Plake, Impara, & Fager, 1993). In the current study, the internal consistency coefficient for the ALI was r KR20 = .525.

Qualtrics Survey Software
The final version of the CBM instrument was uploaded onto Qualtrics, an online survey software system. Maintaining participant anonymity was a priority for producing an electronic version of the instruments. Multiple actions were performed to maintain participant anonymity. First, Qualtrics offers a feature to anonymize participant responses, which means an IP address was not recorded and therefore it was unknown to the researcher who chose to participate. To utilize this feature, the anonymize results option was selected in the survey flow and in the survey options during instrument development. Second, to associate responses on the two measures, a random number generator was used. Participants were assigned a random four-digit number at the beginning of the first instrument and were asked to enter the four-digit number during the second instrument. Accordingly, participant responses were associated with the random number and the researcher could not associate responses to any individual. Third, participants did not report information about their school or district, and individual responses were not shared with building principals or district superintendents.
Two versions of each measure were generated on Qualtrics to allow for counterbalancing the distribution of instruments. Therefore there was an ALI part one, CBM part one, ALI part two, and CBM part two. For purposes of distribution, ALI part one was paired with CBM part two and CBM part one was paired with ALI part two. The part one and part two versions were not identical. Part one included assignment of the random number and the demographic questions. Part two included entering the previously assigned random number and a hyperlink to the incentive survey. The incentive survey offered participants the opportunity to win one of eight twenty-five dollar gift cards to amazon. Participants were asked to enter their email address to be eligible to win. A brief thank you note was added to the end of each instrument. Additionally, an email repertoire was created including the following emails: an initial invite to participate, a reminder to complete part one, an introduction to part two, and a reminder to complete part two. The student teacher versions were comparable in all aspects, except the demographic questions. Two districts requested to meet with the researcher in person to discuss the project.
The researcher met with the Director of Special Education Services in two school districts that requested a meeting. The first meeting occurred during the first month of recruitment. The Director proposed minor changes to the project to make it more accessible in their district. First, he suggested the researcher meet with building principals to explain the study and answer questions. Second, he requested clarification for the term Curriculum-Based Measurement. More specifically, he recommended using Star (i.e. Renaissance Star) as an example of CBM because teachers were currently implementing it. Next, he recommended shortening the interval between the distributions of the two instruments from two weeks to three days. Finally, he selected online distribution rather than hard copy distribution. Three weeks following the initial meeting the researcher met with building principals to explain the project and answer questions. The principals agreed to speak with their teachers at their next faculty meeting and the first instrument was sent out immediately following.
The second meeting occurred during the fourth month of recruitment. The Director supported the purpose of project; however, he expressed concern regarding the expectation for teachers to complete both instruments and the time commitment it entailed. He volunteered to approach principals, special education teachers, and general education teachers to develop interest in the project. If interest was expressed, he would distribute the instruments via email to volunteers. Immediately following the meeting the researcher sent the Director the invite to participate with hyperlinks to both instruments for distribution to the teachers.
Following approval from superintendents, fifty principals were sent an email requesting permission to contact teachers in their building. As a result of low response rates, principals were sent a follow-up email two weeks following the initial email.
Approximately forty-eight percent of principals contacted responded to the email.
More specifically, forty-four percent granted permission and four percent denied permission. Further, some principals (i.e. 27%) preferred to distribute the assessment instruments to their teachers, a few principals (i.e. 9%) collected names of volunteers to send to the researcher, and the remaining principals (i.e. 64%) preferred the researcher to distribute the instruments. Following principal requests to distribute, the assessment instruments were modified for this purpose. Originally, the assessment instruments were designed for distribution by the researcher to simplify the process for teachers. To permit principals to send the instruments while maintaining private access (i.e. invitation only), a password was added to all versions of the assessment instruments.
Upon receiving approval from principals, teacher emails were obtained from district and school websites. Teacher emails were first added to an excel file and then uploaded as a panel onto the Qualtrics website. Once uploaded onto the website, one email could be sent to all individuals within a panel. Teachers contacted by the researcher received a total of four emails each sent at one-week intervals. The four emails included an initial invitation to participate, a reminder to complete part one, an introduction to part two, and a reminder to complete part two. Teachers contacted by their school principal received one email that included a hyperlink to each instrument.
Following three months of recruiting, response rates were significantly lower than expected. In particular, numerous participants completed the first instrument and failed to complete the second instrument. Further, although participants were allowed to exit and reenter the instrument for two weeks, a number of participants responded to fewer than half of the items. In pursuit of recruiting additional participants, the assessment instruments were modified for distribution to student teachers. Two Universities were contacted requesting permission to conduct research and one approved the project. The director of teacher education, at the University of Rhode Island, granted permission to contact students in their junior and senior years of the teacher education program. A total of 264 students were contacted at the end of the fall semester. Students received a follow-up, reminder email approximately one week following the initial invitation to participate. Approximately 2% of student teachers completed the first instrument and 1.5% of student teachers completed both instruments. As a result of the low response rate from student teachers, these respondents were not included in the analysis.
At the beginning of the study, there were three inclusionary criteria for participation. First, participants were required to be currently working in a school implementing CBM. Second, participants were required to have completed a preservice, either undergraduate or graduate, course in assessment or educational measurement. Third, participants had to have attended an inservice presentation or training on the use of CBM. These three questions appeared immediately following the notice about informed consent. If participants responded 'no' to any of the three questions, they were redirected to the thank you note at the end of the instrument.
After approximately six weeks of data collection it became evident that a large portion of participants were answering no to either the second or the third question. As a result, the researcher modified the questions so participants could continue with the instrument after responding no. However, the first question was retained as the only inclusionary criteria.

Informed Consent
Informed consent was obtained electronically on the Qualtrics website. When volunteers followed the hyperlink from the invite email to the instrument, a notice regarding informed consent (Appendix E) appeared. Participation in the study was voluntary and anonymous. Once informed consent was obtained, by participants selecting 'yes,' the instructions to the instrument appeared. If an individual selected 'no,' the site was redirected to the thank you note at the end of the instrument.

Design
The strength of the relationship between measures was examined using Pearson's product-moment correlation coefficient. Pearson correlation is a method for exploring the direction and strength of a relationship between variables (Pallant, 2010

Primary Research Question
Descriptive Statistics and Preliminary Analyses. A total of 27 respondents completed both assessment instruments and were included in the correlational analysis. Thirty-five participants completed part one, but did not complete part two.
Multiple methods were used to assess the normality of the distribution of scores on both instruments (see Table 4). First, skewness and kurtosis were acceptable for both measures according to guidelines presented by Harlow (2005  hypothesized relationships between competencies of assessment literacy were made, and a correlation matrix was expected to provide insight on the relationships. Following initial analyses, it was concluded that the subcategories (i.e. competencies) were not independent from one another, and as a result a correlation matrix was not further investigated. In the current study, the sample size was too small to run a correlational analysis, or to make statistical comparisons between groups, so data were examined using descriptive statistics. As a result, it was unnecessary to run preliminary analyses regarding the normality of the data.
The following descriptive statistics are based on 52 participants who completed both, or one, of the two instruments. All participants were teaching at a school that was implementing CBM.As demonstrated in Table 6, approximately half of the participants (i.e. 25 out of 52) did not complete a preservice course in assessment or educational measurement. More specifically, as shown in Figure 1, all special education teachers (N=4), half of the math specialists (N=1), half of the literacy specialists (N= 4), and three out of four school psychologists completed a course in assessment. Notably, 16 out of 29 general education teachers did not complete a course in assessment despite spending an average 21% (range 10-50%) of instructional time engaged in assessment related activities (see Figure 2). This means that more general education teachers did not complete a course in assessment than those that did complete a course, which is consistent with previous studies that found less than half of teacher education programs require a course on assessment (O'Sullivan & Chalnick, 1991). Additionally, all participants teaching for less than five years reported completing a course in assessment. Thus, individuals who did not complete a course in assessment had more than five years of experience teaching (see   Approximately 38 out of 52 participants reported having attended an inservice, professional development, presentation or training, on the use of CBM (see Table 6).
More specifically, as demonstrated in Figure 4, all special education teachers and specialists reported attending professional development on CBM; however, only 13 out of 21 general education teachers reported attending professional development on CBM. This means that some teachers implementing CBM in their classroom may not have received training specific to the use of CBMs. Interestingly, all 15 participants with 10 or more years of experience teaching attended a training compared to 3 out of 6 participants with between one and five years of teaching experience (see Figure 5).
Further, 15 out of 19 participants who indicated they partake in the interpretation of CBMs have attended a professional development presentation or training.   Interestingly, the highest mean CBM score was found for participants who attended an inservice three to four years ago, which may be a result of learning from experience using CBMs in the classroom (N= 3; M= 75.89; SD= 9.83). Further, some teachers may collaborate with colleagues to learn more about the uses and implications of the measures. The difference in means between the groups is approximately five items.
No difference was found for the mean score between participants who attended an inservice less than a year ago (N= 13; M= 73.32; SD= 7.8) and participants who attended an inservice one to two years ago (N= 6; M= 73.05; SD= 6.41).    Notably, teachers with the least amount of teaching experience and the most amount of teaching experience had the highest mean scores. which ranged from 56% to 68% of items answered correctly (Quilter & Gallini, 2000;Mertler, 2003;Mertler 2009) Data were used to investigate if training may be related to assessment literacy. Figure 11 presents a comparison of mean ALI scores based on participant completion of a preservice course in assessment. Participants who completed a course in assessment (N= 23; M= 59; SD= 8.7) scored similarly to participants who did not complete a course in assessment (N= 24; M= 58; SD= 12) and the difference, approximately one item, does not appear to be meaningful. Figure 12 presents a comparison of mean ALI scores for teachers who attended an inservice on CBM and teachers who did not attend an inservice on CBM. Teachers who attended an inservice training on CBM scored similarly (N= 33; M= 58.7; SD= 10.2) to teachers who did not attend an inservice (N= 14; M= 58.6; SD= 11.4) and the difference does not appear to be meaningful. Further, Figure 13 demonstrates that mean scores on ALI do not differ based on how long ago teachers attended an inservice. Although a majority of teachers who attended an inservice (i.e. 22 out of 33) did so one or two years ago, their scores were not greater than those who attended an inservice more than three years ago.    completed one course or training, followed by teachers who completed two courses and/or trainings (i.e.11 out of 47). Moreover, teachers who completed two courses and/or trainings (N= 11; M= 64; SD= 7.27) and five or more courses and/or trainings (N= 2; M= 62.9; SD= 4.04) scored highest on the ALI instrument. The greatest difference in means was noted between teachers who completed two courses and teachers who did not complete any courses or trainings (N= 2; M= 55.7; SD= 6.07) in CBM; however, the difference is equivalent to approximately three items and does not appear to be meaningful. Notably, there appeared to be a small spike in the mean for two courses; otherwise, a linear relationship emerged as expected. The total number of courses and/or trainings completed on CBM may be related to performance on the ALI and the relationship should be explored further.

Figure 15
Other information related to Assessment.
Information is based on 52 participants who completed both, or one, of the two instruments. Over half (i.e. N= 29) of the participants were teaching in a general education classroom. The remaining participants, as demonstrated in Figure 16, were special education teachers (i.e. N=4), reading specialists (i.e. N= 8), math specialists (i.e. N=2), school psychologists (i.e. N= 4), and 'other' which was mainly defined as speech and language pathologists (i.e. N=4). A majority of teachers (N=48) reported using premade CBMs including web-based programs or materials from a curriculum.
The most commonly identified premade CBMs included Renaissance Star, AIMSWEB, and DIBELS. Approximately half of participants (N=27) reported they develop their own teacher-made CBMs (see Table 7). Teachers reported a wide range of time spent engaged in assessment related activities (i.e. less than 10% to 50%). On average, general education teachers reported spending 19% of instructional time engaged in assessment related activities.
Additionally, the average amount of time spent on assessment related activities was 26% for special education teachers, 14% for literacy specialists, 25% for math specialists, 32.5% for school psychologists, and 20% for participants who identified as 'other.' In summary, the correlational analysis revealed a positive, moderate correlation between assessment literacy, as measured by the ALI, and knowledge about CBM (r= .505, p < .01). The relationship is significant, meaning the two instruments are measuring a similar construct, and the relationship indicates that as scores on the ALI increase, it is likely that scores on the CBM instrument will increase. The extent of the relationship could not be examined by exploring relationships between subareas of knowledge because the tools were not susceptible for this analysis. When examining the extent to which training influences knowledge about CBM, meaningful differences between groups did not appear to emerge and the difference between groups could not be examined statistically due to a small sample size. Overall, knowledge about CBM is related to assessment literacy, as measured by the ALI.

DISCUSSION
The were not differentially related as expected. An examination of the internal consistency of the instruments provided further insight into the relatedness of items and predicted subareas of knowledge (see Appendix F for a correlation matrix containing the interitem relationships for the CBM instrument).
The internal consistency of each instrument was below the generally acceptable level of .7; however, this was expected due to the similar focus of the instruments (Nimon, Zientek, Henson, 2012). In the current study, the Kuder Richardson coefficient for the ALI was r KR20 = .525. Similarly, the Kuder Richardson coefficient for the CBM measure was r KR20 = .469. One reason instruments may not reach the desired level of internal consistency is because the instrument may be measuring more than one construct (Tavakol & Dennick, 2011). Although the ALI and the CBM instruments are measures of teachers' knowledge about assessment, both instruments intend to measure the breadth of competencies as an indicator of overall assessment. In other words, the ALI and the CBM instrument may be more appropriately referred to as general outcome measures (Hosp, Hosp, & Howell, 2007).
That is, the instruments measure a sample of multiple components to provide a general indication about the construct assessment knowledge, or assessment literacy.
Accordingly, the internal consistency reliability coefficients obtained in the current study are adequate based on the nature of the instruments. Despite the indication that both instruments are measuring multiple constructs, as expected, strengths and weaknesses in teachers' understanding of subcomponents of assessment could not be examined. Three possible reasons for the inability to examine a correlational matrix are described below.
First, the instruments used in the current study measured a small sample of each component and scales with a small amount of items (i.e. < 10 items) are typically not representative of a construct (Tavakol & Dennick, 2011 components, it is more challenging to understand other aspects of assessment. This is one possible reason the competencies appear to be interrelated, or not independent from one another, in the current study. Finally, rather than grouping items, or examining subskills based on the components of assessment literacy, an alternative classification system may be more appropriate. Arter (2006) argues that teachers are implementing effective practice, but they lack a conceptual framework to organize their current understanding and to incorporate professional learning. Similarly, DeLuca and Klinger (2010) used factor analysis to identify knowledge domains (i.e. components) on a questionnaire estimating teachers' assessment literacy by measuring teachers' confidence levels.
Results revealed three knowledge domains with adequate internal consistency: practice, theory, and philosophy. In contrast, another study using factor analysis demonstrated assessment conceptualized into a framework with three components: format, purpose, and use (Brown, Lake, & Matters, 2011). Based on these studies, an alternative conceptualization of assessment may be a more appropriate fit to the instruments, particularly the CBM instrument, used in the current study. This approach of alternative conceptualizations may be fruitful for use in future investigations of teachers' assessment skills and knowledge.

Examining relationship between training in assessment and knowledge about CBM.
The secondary research question involved investigating to what extent teacher training in assessment was related to knowledge about CBM. Due to the sample size obtained in the current study, statistical comparisons were not made between groups.
Instead, data were examined descriptively and meaningful differences did not appear to emerge. It is important to note that the small sample size may contribute to the lack of meaningful results. Mean scores on the CBM instrument did not differ by more than 5 out of 47 items when mean scores were compared based on training variables including completion of a preservice course on assessment, attendance at an inservice training on CBM, the number of years teaching, and the total number of courses and/or trainings completed on CBM.
Mixed results about the relationship between training and assessment literacy have been reported in the literature. Consistent with the current study, Gotch and French (2013) investigated two training variables including number of years teaching and attending professional development training, and found that both variables were not related to knowledge about assessment. Conversely, studies have found that teachers with greater than seven years of experience in the field had higher levels of assessment knowledge than teachers with less than seven years of experience in the field (Alkharusi, Karem, Al-Musawai, 2011), teachers who completed a preservice course in assessment had higher levels of assessment knowledge than teachers who did not complete a preservice course (Alkharusi, Karem, Al-Musawai, 2011;DeLuca & Klinger, 2010), and teachers with some training, or exposure, to assessment had higher levels of assessment knowledge than teachers without any training, or exposure, to assessment (Plake, Impara, & Fager, 1993;Volante & Fazio, 2007).
Therefore, it is unclear if training is related to knowledge about CBM. Another limitation of the current research is the varying levels of training prior to teaching in the field. Prior to data collection, completion of a preservice course in assessment or educational measurement was inclusionary criteria for participation;

Limitations
however, data revealed that almost half of the respondents did not complete a course.
Completion of a course was considered inclusionary criteria because it may not be appropriate to expect teachers' to have knowledge and skills about assessment without having first received training. Further, receiving formal training (i.e. teacher education program) may differ than receiving informal training in the field (i.e. learning through experience and collaboration with colleagues). Formal training provides a foundation of knowledge to build on in the field. Therefore, it is unclear if the expectation for being 'assessment literate' could be the same for individuals who have completed a preservice course in assessment and individuals who have not completed a course in assessment. The expectation for 'assessment literate' would need to be more clearly defined.
Finally, it is not possible to determine if the observations (i.e. scores on instruments) were independent from one another. In other words, it is unclear if teachers completed the instruments independently, or if they worked with colleagues to complete them. Further, teachers within a school building may have been exposed to similar trainings and information about CBM offered in their school. Therefore, it cannot be concluded if some teachers have common influences that affected their outcomes on the instruments.

Implications for Practice
The results of the current study have two major implications for practice. First, the mean scores on the ALI and the CBM instrument in the current study were consistent with mean scores on the ALI in prior research. This means that teachers are consistently answering approximately 56-68% of the items correctly. The question that remains is whether that level of understanding is adequate for practice, or if a higher number of items answered correctly would qualify teachers as assessment literate.
Although the ALI has been used frequently in the literature, it is unclear what specific scores mean. As general outcome measures, the instruments provide a general indication of what teachers know about assessment, but without cut-offs, scores are less meaningful. In the current data-based accountability reform, the expectation for teachers to use assessment tools both accurately and effectively has increased; however, is it necessary for teachers to be experts in assessment? In a manner similar to a screening measure, it may make sense to assign cut-off scores to classify levels of teachers' understanding of assessment. This could inform practice if teachers could receive appropriate, individualized supports to feel more confident with the use of assessments. Furthermore, teachers may not need to be experts in all aspects of CBM.
The definition of being literate in CBM may depend more specifically on the role teachers have, and further this may vary among schools and districts.
Second, results demonstrated minimal differences in the mean CBM scores based on the number of years of teaching experience. Due to the small sample size these results are interpreted with caution. A linear relationship was expected to occur, meaning the mean score would increase as the number of years increased.
Interestingly, differences in the mean scores (i.e. approximately 1-3 items) did not appear meaningful. In fact, teachers who recently graduated from teacher education programs (i.e. less than two years of experience in the field) and teachers with greater than ten years of experience in the field exhibited slightly higher levels of CBM knowledge than teachers with 2-10 years of experience. One possible explanation for the observed lack of a linear relationship in CBM knowledge is once teachers are in the field their role shifts from being a professional learner (i.e. a student) to the day-today activities of a teacher. Teaching is a multifaceted profession that requires knowledge of numerous areas in addition to assessment. This result has implications for professional development. Effective professional development is necessary to maintain, or increase, teachers' level of knowledge while in the field. It is unclear what is the most effective method of professional development; however, more recent studies are exploring this area. For example, Koh (2011) found that ongoing, sustained professional development more effectively increased teachers 'assessment literacy, as measured by student work samples, scoring rubrics, and focus group interviews, than a short-term, one-shot workshop. In another example, Mertler (2009) found that a two- week professional development workshop based on the 1990 Standards for Teacher Competence in Educational Assessment of Students significantly increased teaches scores on the ALI. Finally, providing a conceptual framework for sound assessment practice may assist teachers with organizing their knowledge about assessment (Arter, 2006). Professional development is important for maintaining teachers knowledge about assessment, and it is even more important to ensure that effective professional development is provided.

Future Directions
In this study, the relationship between teachers' assessment literacy and teachers' knowledge about CBM was examined. A positive, moderate correlation was found regardless of the small sample size. Future research could replicate the current study with a larger sample size to determine if the relationship generalizes beyond the current sample. Furthermore, research can examine differences in assessment literacy and knowledge about CBM based on different teaching positions (i.e. general education, special education, specialists, and school psychologists).
Additionally, future research could examine to what extent these two areas of knowledge are related by investigating alternative conceptualizations of assessment.
For example, rather than conceptualizing assessment based on the 1990 Standards for Below is an assessment instrument consisting of multiple choice questions. Please read each question carefully, some are single response questions, some are multiple response (check all that apply), and some require you to use provided information to answer the proceeding questions. This survey is focused on academic curriculumbased measurement that teachers may use in the classroom. Curriculum-Based Measurement (referred to as CBM) may include, but is not limited to AIMSWEB, DIBELS, FAIP-R, easyCBM, Renaissance Star, and teacher-made CBMs . Please answer the following questions based on your knowledge of CBM acquired from courses, trainings, and experience in the field, and refrain from searching for the answers on the internet or discussions with colleagues. This instrument is for general research purposes and results will not be distributed to anyone in your district. Some of the questions were designed to be challenging, so do not be concerned and please give your best possible response. Thank you in advance for your contribution and assistance in this study.
To help schools make effective data-based decisions, a systematic problem-solving process is used. The following questions on this page will ask you to think about this problem-solving process.
Which is the most appropriate order of stages in the problem solving process?  Problem Identification, Problem analysis, Plan Development, Plan implementation, Plan evaluation  Plan evaluation, Problem Identification, Problem Analysis, Plan implementation, Plan Development  Problem Identification, Problem Analysis, Plan Evaluation, Plan development, Plan implementation  Problem Analysis, Problem Identification, Plan development, Problem Evaluation, Plan implementation What is the purpose of the problem identification stage?  To determine if items for assessment are drawn from the local curriculum  To set goals and plan intervention  To decide whether a student's performance on academic tasks is discrepant enough from expectations to indicate a problem exists  To determine the magnitude of a problem What is the purpose of evaluating the effects of a plan (i.e. intervention or change in instruction)?  To monitor student progress toward goals and determine if growth is occurring  To tie results to local norms  To determine whether a problem continues to exist and if support services are still required  To decide whether a student's performance on academic tasks is discrepant enough from expectations to indicate a problem exists What is the purpose of the plan development stage in the problem-solving model?  To monitor student progress toward goals and determine effectiveness of an intervention  To make decisions within a problem-solving model  Describe how CBM is standardized  To set goals and plan intervention What is the purpose of the problem analysis stage in the problem-solving model?  To determine the magnitude of the discrepancy  To determine whether a problem continues to exist and if support services are still required  To facilitate continuity across special education decisions  To validate the assessment measure What is the purpose of the plan implementation stage in the problem-solving model?  To monitor student progress toward goals and make changes when appropriate  To determine the reliability and validity of the solution  To determine if items for assessment are drawn from the local curriculum  To begin using a new curriculum with the class What question or questions does the problem analysis stage address?  Is the intervention tied to the student's curriculum?  What are appropriate and effective interventions?  Did the intervention work (i.e. has the student made progress toward their goal)?  Why is the problem occurring?
What question or questions does the plan implementation stage address?  Is the intervention working (i.e. is the student making progress toward their goal)?  Is the intervention valid and reliable?  Does a problem exist?  Is the intervention tied to the local curriculum?
What question or questions does the problem identification stage answer?  Is there a discrepancy between the student's actual and expected performance?  What materials are necessary to help the student benefit from their education?  What is an appropriate long-term goal for the student?  Is the measure reliable and valid?
Which question does the plan development stage first address?  What materials are required to help the student benefit from their classroom education?  What are appropriate and effective interventions?  Is there a discrepancy between the student's actual and expected performance?  Can local norms be established?
What question or questions does the plan evaluation stage address?  Was the plan implemented correctly?  Does a problem still exist?  What is the content of the intervention?  What is an appropriate long-term goal for the student?
CBM was developed to be a simple, efficient method for assessing student achievement in the basic skill areas. What are the advantages of this method? CHECK ALL THAT APPLY.  Tied to a problem-solving model of decision making  Measures mastery of specific skills  Performance based assessment  Accurate predictor if student will graduate high school  Tied to national norms  Valid and reliable indicators of academic performance  Results used to diagnose specific skill deficits  Cost efficient in terms of time and money  Designed for simple, repeatable administration  Can be used to determine the effectiveness of instruction and monitor student progress.  Allows instructor to make inferences about student behavior beyond the behavior measured  Use in different content areas (i.e social studies, science)  Used to develop local norms  Measures student comprehension (i.e. higher order thinking)  Standardized administration Note: Some of the questions were designed to be challenging, so do not be concerned and please give your best possible response.
CBM is tied to a problem-solving model of decision making. This means:  CBM is designed to facilitate consistency across instructional decisions  All students will receive the same intervention  All students have problems and instructors are required to fix them  Instructors are guided to make the right decisions CBMs employ production-type responses. This means:  Students choose correct answer from a list of responses  The tasks students are asked to perform are similar to tasks in the curriculum  Students must actually perform the skill of concern  Students are asked to discuss the skill of concern CBMs should be reliable measures of student achievement. What is the definition of reliability?  If a student takes the assessment more than once their score will be consistent  The assessment measures what it intends to  The measure systematically samples the year-long curriculum  The assessment is given at the end of the school year CBMs should be valid measures of student achievement. What is the definition of validity?  If a student takes the assessment more than once their score will be consistent  The assessment measures what it intends to  The measure systematically samples the year-long curriculum  The assessment is given at the end of the school year CBM is cost efficient in terms of time and money. This means:  CBM involves many materials that are funded by the state  Less students need to be assessed  Instructors need less training to administer and score CBMs  Less money is spent on test materials and less time is spent on administration than published norm-referenced tests (i.e. standardized tests such as SAT and NWEA) CBM was designed for simple, frequent administration. This means:  CBMs are quick to administer and results can be easily graphed to monitor progress  CBMs are difficult to administer and score  Frequent administration allows CBM to be used for screening students  Frequent administration allows for comparison between students in the nation CBM can be used to determine the effectiveness of instruction and monitor student progress. This means:  The tasks students are asked to perform are similar to tasks in the curriculum  CBM graphs demonstrate relationship between student achievement and instructional interventions  CBMs are short duration, multiple form measures.  CBM is the only data needed to make accurate decisions regarding student achievement and interventions Standardized administration means:  Administration procedures differ for progress monitoring and screening purposes  Administration procedures are specified so teachers do not have to be involved in the administration  Administration procedures are specified so you can change the instructions based on student needs  Administration procedures are consistent across all settings CBMs can be used to develop local norms. What are local norms?  Local norms define guidelines for diagnosing skill deficits  Local norms define a score that represents a desired level of performance  Local norms define expectations for student achievement and reflect the student's learning culture and community  Local norms define expectations for student achievement so students' scores can be compared to other students in the nation At the classroom level, local norms can be used for which of the following. CHECK ALL THAT APPLY.  Identifying if a student score is below expected achievement/performance  Goal setting  Diagnose skill deficits  Determining if an instructional technique or program is effective  To discipline students  Determining if a problem still exists  Assisting teachers and multi-disciplinary teams in prioritizing students' needs  Choose which students should receive rewards  Instructional planning  Progress monitoring  Compare student achievement to same-age peers in the nation What is the definition of fluency?  A measure that reflects speed/automaticity and indicates several elements of proficiency  Assessments during instruction to inform or assessment for learning  A measure that is based on a systematic sample the year-long curriculum so that each skill is represented  Assessment that focuses on a particular set of skills The short duration of CBM facilitates  Instructional planning  Creating national norms  Monitoring student progress  Reliability and validity A general outcome measure, or long-term measurement approach, is:  Degree to which an assessment measures what it intended to measure  Assessment that reflects proficiency of specific skill  Assessment that reflects overall competence of a basic skill  Assessment conducted on a frequent basis to inform instruction Mastery Measurement is  A type of assessment that is based on a systematic sample of the year-long curriculum so that each skill is represented  A type of assessment that reflects accuracy of a students level of proficiency  A type of assessment that reflects speed/automaticity and indicates several elements of proficiency  A type of assessment to identify which students are at-risk for academic failure CBM is able to detect small amounts of growth over time. This is an advantage for:  Identifying skill deficits  Mastery measurement  Obtaining validity of student scores  Monitoring student progress Progress monitoring, or formative assessment, is:  An established process with consistency from context to context  Assessment of learning, after instruction or a summation of what the students know and are able to do after instruction  Assessment conducted during instruction or an intervention to inform instructional decisions  Scientifically demonstrated consistency of a measure In relation to assessment, the purpose of screening is to:  Identify which students are at-risk for academic failure  Assess on a frequency basis to inform instruction  Determine if an intervention will be effective before it is tried  Assess student achievement for graduation Curriculum-Based Measurement is:  A well-researched type of general outcome measurement  A well-researched type of mastery measurement  A poorly researched type of mastery measurement  A poorly researched type of general outcome measurement What is the purpose of using CBM to systematically measure student performance at grade level and the grade levels sequentially below until the student has reached the benchmark?  To identify students who are at risk for academic failure  To identify students' instructional level  To diagnose skill deficits  To use at pre-post testing  Marking a student score on a graph  Comparing student score to other students of the same age  Comparing student score to a standard representing a desired level of performance  Frequently collecting data to determine if a student is making growth How is the median score calculated?  Add up scores on all probes and divide by the number of probes  Using your own judgment about which score best fits the students ability  Put all scores in order lowest to highest and pick the middle score  Add up scores and divide by the student's grade level Hannah is a third-grade student who transferred to Red Elementary School late in the fall. Her teacher has noticed that she seems to struggle with many independent reading assignments. When the teacher administered the mid-year universal screening measure, she was not surprised to see that Hannah's score had fallen below the gradelevel benchmark. Consequently, the teacher gives Hannah additional instruction in a small group two times per week for 30 minutes and monitors her reading performance once per week for seven weeks using a measure of reading fluency. After seven weeks Hannah's scores show steady growth in her performance, but she still has not met the mid-year benchmark. It can be concluded that:  The small group instruction is ineffective  The small group instruction requires a modification  The instructional strategies used in the small group are not valid  The small group instruction is effective and should be continued Larry is a third-grade student at Purple Elementary School. His teacher administered a universal screening measure a few weeks after school began. Larry's score indicated that he may be struggling in reading. As a result, the school provides him with an intervention two times per week for 30 minutes and monitors his reading performance once per week for eight weeks using a measure of reading fluency. He is expected to improve by one word per minute each week. After eight weeks Larry's score has increased by 4 words per minute. His teacher notices that Larry scored 55 words read correct for three weeks straight. It can be concluded that:  The program is effective for Larry  The program is ineffective for Larry  The program should continue to be used with Larry  The program is not valid Use the provided data to answer the following questions. The data below was collected for Diane, a 4th-grade student who was referred by her teacher for performing "far below grade level" in reading. On grade 3 material, Diane scored in the 14th percentile. What does this mean?  Diane scored lower than 14% students in grade 3.  Diane scored higher than 14% of the students in grade 3.  Diane answered 14% of questions correctly.  Diane received a total score of 14 out of 100 Use the provided data to answer the following questions. The data below was collected for Kasey, a 4th-grade student who was referred by her teacher for performing "below grade level" in reading. The Superintendent or Research person of your district has given permission for this research to be carried out. I am hereby seeking your consent to approach a number of teachers in your school to solicit as participants for this project. More specifically, I would like to approach general education teachers teaching Kindergarten through fifth grade, special education teachers, and specialists for this study. Participation in this project is anonymous and individual teacher participation is voluntary. Teachers who choose to participate will be asked to complete two online assessment instruments with a two week interval between the first and second.

Results of CBM Survey-Level Assessment in Oral
I have provided you with a copy of my thesis proposal, which includes a copy of the consent form to be used in the research process. In addition, I have attached a copy of both measures to be used, as well as a copy of the approval letter which I received from the University of Rhode Island Institutional Review Board.
With your permission, I would like to proceed as follows: 1. Obtain permission from the Principal to distribute information to teachers 2. Distribute information to teachers for their consideration to participate If you require any further information, please do not hesitate to contact me at (914)  You are being invited to take part in a research study exploring teachers' understanding of assessment. In particular the study will examine teachers' understanding of assessment in general and teachers' understanding of a specific type of assessment known as curriculum-based measurement. The title of my research project is 'Toward a better understanding of teachers' knowledge of assessment: Examining the concurrent validity of tests of Assessment Literacy and Curriculum-Based Measurement'. The study is described in more detail below. If you have any questions, please feel free to call Paige Hamilton, graduate student, at 914-804-7621 or Gary Stoner, PhD at 401-874-4234, the people mainly responsible for this study.
You must be at least 18 years old and currently teaching in an elementary school to participate in this research project.
The purpose of the study is to explore the extent to which teachers' knowledge and understanding of general assessment is related to their knowledge and understanding of curriculum-based measurement.
If you decide to take part in this study, you will complete two online assessment instruments containing multiple choice questions. One assessment consists of 35 questions and will take approximately 15-20 minutes to complete. The other assessment consists of 48 questions and will take approximately 20-25 minutes to complete. After completion of the first assessment instrument, there will be a two week period before the second assessment instrument is sent to you.
The possible risks or discomforts of participating in this study are minimal, although you may feel some frustration while answering challenging questions. Following the completion of the study the answer key to each questionnaire will be sent to all participants to allow for reviewing the questions asked along with the answers to the questions.
Although there are no direct benefits of the study to you, it may prompt you to think and learn more about the use of assessments in the classroom. Your answers will help improve the general understanding of how general assessment knowledge may be related to the understanding and knowledge of curriculum-based measurement. This information could potentially lead to better methods for teaching teachers about the purpose and use of assessment in the schools. Your participation in the study is anonymous. That means that your answers to all questions are private. I have taken measures to ensure that your answers will not be linked with your name or email. No one else will know if you participated in this study and no one else will find out what your answers were. You will not have to identify the school district or name of your school at any time. All information will be kept private, in a password-protected file, in case someone could look at your demographics and identify you. To link your responses to both instruments, you will automatically receive a randomly generated 7 digit number from the survey website after completing the first assessment instrument. You will enter the 7 digit number at the beginning of the second assessment instrument. You are the only person that will know the number that has been assigned to you. Any scientific reports generated will be based on group data and will not identify you or any individual as being a participant in this project.
The decision to participate in the research project is up to you. You do not have to participate and if you do participate you can refuse to answer any question.
Participation in the study is not expected to be harmful or injurious to you. However, if this study causes you any injury you should email or call Paige Hamilton (914)