Developing Knowledge for Teaching Graphing of Bivariate Categorical Data

ABSTRACT Due to the increased role of statistics in current curriculum standards, there is at present a call for attention to secondary teachers’ knowledge for teaching statistics in teacher education. This article reports the results of a study that answered this call by addressing mathematics teachers’ knowledge for teaching graphing of bivariate categorical data. Novel curriculum materials to develop such knowledge were written then implemented with mathematics teachers in courses at four post-secondary schools. Results showed that prior to use of the materials, teachers’ graphical competence was limited with teachers often utilizing frequencies when analyzing the data. Following use of the materials, they were more likely to correctly use relative frequencies in their analysis and expanded their knowledge of graphs to include segmented bar graphs. They also improved in their analysis of a student’s graph and proposed response to the student, with more teachers noting the inappropriate use of frequencies and lack of label in the student-made graph post-instruction. Implications for development of teachers’ knowledge for teaching statistics are discussed.


Introduction
Statistics has become an important strand of mainstream mathematics curricula worldwide, including Australia (Australian Curriculum, Assessment, and Reporting Authority 2012), Brazil (Ministério da Educação 2006), and the United States of America (National Governors Association Center for Best Practices and Council of Chief State School Officers (CCSS-M) 2010). The preparation that teachers receive for teaching statistics, however, has generally not changed to keep up with the increased role statistics plays in secondary mathematics curricula. For example, with the advent of the Common Core State Standards for Mathematics (CCSS-M 2010) in the United States, most future secondary (grades 6-12) teachers will need to teach statistical content they never saw during their own secondary education (Conference Board of the Mathematical Sciences (CBMS) 2012a). Moreover, teachers who have taken statistics courses typically experienced curricula emphasizing procedural knowledge rather than the statistical reasoning they will be asked to teach (Rossman 2015). This has led to concern regarding whether teachers' knowledge for teaching statistics (Groth 2013) is adequate (Batanero, Burrill, and Reading 2011;CBMS 2012b;Kettenring, Lindsay, and Siegmund 2003). The CBMS (2012b) and the American Statistical Association (Franklin et al. 2015) recently developed recommendations for teacher preparation for teaching statistics to address this concern, and both groups identified statistics as a content area for which teachers' knowledge for teaching greatly needs improvement. Despite the current emphasis on teacher knowledge in mathematics education, there is an overall lack of research on knowledge for teaching CONTACT  statistics (Batanero, Burrill, and Reading 2011;Groth, 2007). Shaughnessy (2007) identified this gap specifically and named it a key area for future research.
A key topic for which teachers need robust knowledge for teaching statistics is graphing. As the adage "A picture is worth a thousand words" emphasizes, graphical representations can facilitate the understanding of statistical data. Indeed, most statisticians start analyzing a dataset by making graphs (Burrill and Biehler 2011). In addition to being useful for analyzing data, graphs are also useful for communicating information about data (González, Espinel, and Ainley 2011). Burrill and Biehler (2011) deemed the making and interpretation of graphical representations a fundamental idea in statistics. The inclusion of learning objectives regarding statistical graphs in many countries' primary and secondary curricula reflects the importance of this topic in students' statistical education (González, Espinel, and Ainley 2011) and highlights the need for teachers to have strong knowledge for teaching statistical graphs. Following their review of studies on teachers' graphical competence and knowledge for teaching graphs, González, Espinel, and Ainley (2011) noted that teachers' graphical competence is quite limited and research on how to develop teachers' knowledge for teaching statistical graphing is needed.
One type of data for which graphs can be used to analyze and communicate relationships is bivariate categorical data. While this type of data is often summarized numerically in two-way tables, graphical displays such as side-by-side bar charts and segmented bar graphs provide visual representations of the conditional distributions for each category. These graphs illuminate and reveal stories in the data and provide a tool for learning about relationships (Burrill and Biehler 2011). Standards regarding the summary, representation, and interpretation of bivariate categorical data are included in current secondary curriculum standards documents (e.g., CCSS-M (2010) 8.SP.4 and HSS-ID.B.5); hence, the knowledge for teaching graphical analysis of bivariate categorical data is needed by today's secondary mathematics teachers. However, widely available curriculum materials that comprehensively address the knowledge for teaching analysis of bivariate categorical data do not exist. To address this concern, the first and third authors wrote novel curriculum materials to develop teachers' knowledge in this area then oversaw their use with secondary mathematics teachers at four post-secondary schools during the 2014-2015 academic year. This article describes the graphical competence of mathematics teachers regarding bivariate categorical data and the impact the curriculum materials had on the teachers' knowledge for teaching graphing of bivariate categorical data.

Knowledge for Teaching Statistics
This study applied the theoretical framework of Mathematical Knowledge for Teaching (Hill, Ball, and Schilling 2008) to describe the components of the knowledge for teaching statistics. This framework has been used as the basis of conceptualizations of knowledge for teaching statistics (e.g., Groth 2013) and instruction regarding statistics in the K-12 setting is primarily done by mathematics teachers (CCSS-M 2010), so use of this framework is appropriate. Figure 1 displays the components of this framework.
This framework divides teacher knowledge into two main parts-subject matter knowledge and pedagogical content knowledge-each with three components. Studies by Hill and colleagues established that when knowledge for teaching is defined to consist of subject matter knowledge as well as pedagogical content knowledge, student achievement is tied to teacher knowledge (Hill, Rowan, and Ball 2005) and improved knowledge for teaching contributes to the overall mathematical teaching quality . This emphasizes the usefulness of this conceptualization of teacher knowledge.
Beginning with subject matter knowledge, one component is common content knowledge. This refers to the statistical knowledge needed by common citizens to be statistically literate. It is expected and necessary that teachers possess this knowledge. Knowledge of how to compute conditional probabilities for a two-way table is an example in the context of categorical association. This content knowledge alone, however, is insufficient for teachers; teachers also need specialized content knowledge. This is a knowledge of content specific to the work of teaching that is different from the type of knowledge needed in other occupations where statistics is used. The depth and detail of specialized content knowledge goes well beyond the knowledge needed by persons who only need to carry out procedures reliably (Ball and Bass 2003). It is an understanding of content that enables teachers to be able to interpret statistical ideas brought forth by students as well as to understand various representations of statistical concepts. In the context of categorical association, this could be relevant to a teacher interpreting a student-invented graph for displaying bivariate categorical data. The final category of subject matter knowledge, horizon knowledge, calls for teachers to know more about the greater discipline of statistics in which the prescribed curriculum is situated. This includes knowledge of future topics the taught topics lead to; hence, the metaphor of considering the view of the mathematical horizon. For example, a teacher teaching formal inference for categorical association using chi-squared methods would benefit from knowing how these methods relate to two-sample z-tests for proportions.
Pedagogical content knowledge addresses the interaction of content knowledge with knowledge about students, teaching, and the curriculum. Knowledge of content and students refers to the need for teachers to know what students may think about particular content topics, including conceptions and misconceptions, and learning progressions for topics. Teachers need this type of knowledge to be able to anticipate the ideas students have when they learn about a certain topic as well as standard stumbling blocks that occur during the learning of that topic. In categorical association, an example is knowing that students often struggle with proportional thinking which is relevant when computing and comparing conditional probabilities. Teachers also need a type of knowledge that mixes their content knowledge of a specific mathematical topic with pedagogical principles for teaching it. This is known as knowledge of content and teaching and is called upon when teachers are making pedagogical decisions such as sequencing of content or choice of representations. For categorical association, this could involve knowledge needed to determine the best representation to use to teach students the meaning of association of categorical data. The third category of pedagogical content knowledge, curriculum knowledge, refers to teachers' knowledge about the materials, textbooks, and instructional programs that are available for the teaching of specific content and how to sequence their use to enhance student learning. This includes knowing how and when the content topic is taught in other subject areas such as science or social studies, as well as how ideas are developed across grade levels within a subject area. Categorical association, for example, is relevant to the analysis of two-way tables produced in simple genetics studies in introductory biology classes, so teachers need to be knowledgeable about students' learning experiences concerning categorical association there as well.
Collectively, these components describe the knowledge needed for teaching statistics, and the curriculum materials were written to address all of them concerning analysis of bivariate categorical data. The present study addresses the common content knowledge, specialized content knowledge, knowledge of content and students, and knowledge of content and teaching for graphical analysis of bivariate categorical data.

Research About Teachers' Knowledge for Teaching Statistical Graphs
Minimal research has been done regarding mathematics teachers' knowledge for teaching statistical graphs, but the available studies have shown that in general teachers' knowledge is limited. González and Pinto (2008) examined four Spanish preservice secondary mathematics teachers' knowledge of statistical graphs. The teachers were unable to distinguish between different graphs and did not recognize some of them (e.g., stem and leaf graphs). They also preferred to work with numerical values from a table than from a graph, though they saw the value in using graphs to summarize information. Regarding graph construction, Bruno and Espinel (2009) found that preservice teachers in Spain struggled when constructing histograms and frequency polygons. Their struggles included inadequate labeling of the axes, not considering zero frequency intervals for histograms, and not completing the frequency polygon. When these same teachers were asked to identify mistakes in studentmade graphs, these errors resurfaced. Moving past graph construction to the interpretation of graphs, preservice teachers either found interpretation was not needed (i.e., teachers found it sufficient to construct a graph in order for the task of analyzing a dataset to be completed) (Burgess 2002) or had great difficulty interpreting statistical graphs (Burgess 2002;Espinel, Bruno, and Plasencia 2008;Batanero, Arteaga, and Ruiz 2010;da Silva, Kataoka, and Cazorla 2014). Rouan (2002) found that this deficiency had implications for instruction by inservice teachers.
When inservice teachers taught about statistical graphs, they focused on construction of graphs and rarely provided instruction on interpreting graphs. The research studies just mentioned focused on mathematics teachers' subject matter knowledge concerning statistical graphs and its implications for instruction. Research regarding mathematics teachers' pedagogical content knowledge related to teaching statistical graphs is even more limited. González and Pinto (2008)'s research is the sole study that also assessed teachers' pedagogical content knowledge for teaching statistical graphs. They found the teachers were lacking in their knowledge of content and teaching, as well as knowledge of content and students related to teaching statistical graphs. The teachers did not know about components of teaching students about statistical graphs (e.g., distinctions between reading data from a graph and reading beyond the data) nor common learning difficulties students have when learning to make and interpret statistical graphs.
Collectively, these studies highlight that teachers' knowledge for teaching statistical graphs is inadequate, likely due to insuf-ficient learning opportunities in teachers' education (González, Espinel, and Ainley 2011). This implies that research regarding how to develop teachers' knowledge for teaching statistical graphs is needed and important for the field.

Proportional Reasoning
The analysis of bivariate categorical data relies largely on sound proportional reasoning, for the data are typically summarized as frequencies in a two-way table and the total number of subjects usually varies across categories. Proportional reasoning is a complex topic; the development of proportional reasoning skills by students does not develop naturally (Sowder et al. 1998) and takes a prolonged effort over time (Behr et al. 1992). Interpreting the frequency information in two-way tables is often a challenge, particularly for those who have not fully developed their proportional reasoning skills (Watson, Callingham, and Donne 2008). Many persons use additive rather than multiplicative thinking in situations of proportion and have difficulty distinguishing between situations of proportion and situations of nonproportion (Hilton et al. 2016). Batanero et al. (1996) and Konold et al. (1997) found that lack of proportional reasoning resulted in secondary students often drawing incorrect conclusions about association of bivariate categorical data. In fact, at times students would use a correct numerical strategy for assessing association of bivariate categorical data (e.g., comparing relative frequencies across categories), but then come to an incorrect conclusion due to deficiencies in their proportional reasoning (e.g., decide there is an association because there are more subjects in one of the categories). Students in the Batanero et al. (1996) study had a success rate of approximately 30% when solving tasks involving analyzing bivariate categorical data for relationships, indicating the difficulty of such tasks. Understanding the importance of proportional reasoning to analyzing bivariate categorical data and how students learn proportional reasoning, including difficulties in that learning process, are central to teachers' knowledge of content and students and knowledge of content and teaching for this topic.

Research Questions
The present study had two research questions. They were: 1. What is the graphical competence of mathematics teachers for bivariate categorical data? 2. Did the use of the curriculum materials improve mathematics teachers' knowledge for teaching graphing of bivariate categorical data?

Curriculum Materials
As previously described, the curriculum materials utilized in this study were written by two of the authors to develop teachers' knowledge for teaching analysis of bivariate categorical data. Collectively, the authors had decades of statistics teaching experience at the high school and university levels on which to base the design and content of the materials as well as knowledge  and creation of research in statistics education related to teacher knowledge and teacher education. These curriculum materials represent their first effort to translate these experiences into a product that is usable in teacher education. Figure 2 presents the material's table of contents as an overview of the materials. In the manner of other activity-oriented curriculum materials (e.g., Rossman and Chance 2011), the format of the materials was a workbook filled with activities for the teachers to complete and reflect upon. The materials were aligned with the CCSS-M, addressing the knowledge for teaching all the content standards concerning bivariate categorical data (8.SP.4 and HSS-ID.B.5). In addition, they incorporated the Guidelines for Assessment and Instruction in Statistics Education (GAISE) report's (Franklin et al. 2007) statistical investigatory process. (Note: Since the curriculum materials were written in 2013, they used the 2007 GAISE report rather than the 2015 version.) The curriculum addressed all phases of the process including discussion of statistical questions that generate bivariate categorical data, appropriate graphical analysis of this type of data, and the types of conclusions that can be drawn. Users of the curriculum also carried out the entire process themselves. An external review of the materials was done by a statistics education expert to ensure appropriate alignment with and coverage of CCSS-M, GAISE, and the theoretical framework.
The focus of this article is upon graphs of bivariate categorical data, addressed in Activities 4 and 7; thus, more details regarding these activities are provided. Graphing categorical bivariate data are the focus of Activity 4 in the curriculum materials. Comparative (or side-by-side) bar graphs displaying frequencies (counts), side-by-side bar graphs displaying relative frequencies, segmented bar graphs, mosaic (or ribbon) charts, and pie charts comprise the graphs addressed. The authors' position is that knowledge for teaching graphing bivariate categorical data includes knowledge of all these graph types, as they are graphs students make independently (e.g., Casey, Hudson, and Ridley (2018) found that students often make comparative bar charts displaying frequencies) and/or are seen in other courses or media sources (e.g., pie charts). Figure 3 presents a portion of the materials that addressed comparative bar graphs.
The curriculum materials focus on the creation and interpretation of previously named graphs, the relationships between them, and their advantages and disadvantages-all in the context of teaching the topic. For example, Question 4-b (included in Figure 3) prompted the teachers to craft a response to a student who thinks a comparative bar chart displaying frequencies shows evidence of an association when in fact it does not. This is a frequent student error based on a common misconception to use frequencies rather than relative frequencies when analyzing two-way tables (Batanero et al. 1996), and one which the teachers learned about in the previous activity (Activity 3). Throughout the curriculum materials and in the directions for discussions the instructor would lead, a central idea was the use of relative frequencies. An emphasis was placed on the importance of using relative frequencies when graphing bivariate categorical data in order to place each group on the same scale (i.e., a percentage) and produce graphs that best communicate the information. The materials also addressed interpreting graphs to informally assess whether the variables are associated; the use of a segmented bar graph was encouraged for this purpose as it best facilitates comparison of the groups.
Later in the materials, Activity 7 engaged the teachers in analyzing and making connections between K-12 curriculum standards relevant to bivariate categorical data. A portion of this activity that is relevant to graphing bivariate categorical data is provided in Figure 4.
The instructors are directed to lead a discussion regarding teachers' responses to Question 7-b that addresses the use of graphs for helping students analyze and understand association of categorical data. This discussion brings this issue to the forefront and helps teachers understand the importance of graphing as part of the exploratory data analysis process and for supporting student understanding.

Participants
Fifty-nine teachers participated in the study during the 2014-15 academic year. All were secondary mathematics education majors enrolled at one of four large post-secondary schools in the United States. School A, located in the Midwest, had 23 preservice, undergraduate participants. School B, located in the West, had 18 preservice, undergraduate teachers participate. School D, located in the Midwest, provided nine preservice, undergraduate participants. Schools A, B, and D used the curriculum materials in an introductory statistics course required in their preservice teacher education programs. At School C, the participants were nine inservice teachers enrolled in a graduate program to earn their master's degree in mathematics education. They had on average 2.9 years of teaching experience and most of them were concurrently teaching secondary mathematics. At the time of the study, they were in a graduate course addressing teaching mathematics with technology. The appendix presents the title, course description, and prerequisite course (if any) for the course at each site where the materials were implemented. It also provides background information on the instructor (i.e., highest degree and job title). At one of the schools, a coauthor of the curriculum materials was the instructor. The participants' background in statistics prior to the study is unknown, but it is known that none of the courses where the study was conducted had a college-level statistics course as a prerequisite. Across all schools, the instructors implemented the curriculum materials (see Figure 2 for the table of contents) over 2 weeks. Prior to implementation, the instructors received two versions of the curriculum materials: a student version and a teacher version with instructional comments and answers to the prompts. They were directed to use these as written, without any revisions. They also received specific directions regarding how to implement the materials, including how much time to spend on each activity, what to assign for homework, and discussions to lead after students had worked through portions of the materials. The hope was that by supplying these collective resources the instructors would implement the materials similarly at their various sites. Relevant to the results presented in this article, each instructor taught Activity 4 regarding graphing of bivariate categorical data during one class period (ranging from 75 to 90 min). Each instructor reported that the activity was taught as directed by the curriculum materials.

Study Design
A basic interpretive qualitative study (Merriam 2002) was conducted to investigate teachers' knowledge for teaching graphing bivariate categorical data. It had a pre-post study design to learn whether the use of the curriculum materials improved teachers' relevant knowledge. All participants completed an assessment just prior to (pre) and immediately following (post) their use of the curriculum materials. This study focused on the data from a task on the assessment that addressed graphing, shown in Figure 5.
The problem initially presented in the task was included in the first free response problem from the 2009 AP Statistics Exam (College Board 2009). The first two parts of the task, which ask for a range of correct plausible graphs for this problem (part (a)) and determination of the best graph for representing association (part (b)), address subject matter knowledge for teaching graphing of bivariate categorical data. In particular, they address common content knowledge and specialized content knowledge. There is not one, definitive "correct" graph students could make when solving this problem (e.g., could condition on either gender or job experience), thus, it is important that teachers be able to generate and consider all the plausible correct graphs students could produce. While there may not be one "correct" answer to this problem, some graphs do a better job of representing the data in a way that enables analysis of association than others; part (b) addresses this.
Parts (c) and (d) of the task engage teachers in two high leverage teaching practices (Garcia and Shaughnessy 2015). The teachers analyze a student's work then provide feedback to further develop his or her thinking. These parts of the task assess their subject matter knowledge for teaching the graphing of bivariate categorical data, along with their knowledge of content and students (e.g., it is common for students to use frequencies rather than relative frequencies when analyzing bivariate categorical data) and knowledge of content and teaching (e.g., it is important to help students understand the need for relative frequencies, which can be tied to teaching about proportional reasoning) related to graphing bivariate categorical data. The presented student graph was selected because it has two primary deficiencies: use of frequencies instead of relative frequencies and lack of a label on the vertical axis.
The pre-and post-assessments were identical. They were assigned as a take-home assignment by each instructor with the stipulation that the participants could not collaborate with classmates nor receive help from the instructor or online sources. Following administration of the assessments, each instructor replaced the participant names with identification codes to anonymize the data and the completed assessments were gathered for analysis.

Data Analysis
As this was a basic interpretive qualitative study, the data were inductively analyzed to identify patterns and themes (Merriam 2002). The first and second authors, with input from the evaluator of the research project, developed the data analysis process then the first and second authors carried it out. A coding scheme for the graphs the teachers made in parts (a) and (b) of the assessment task data was developed based on an initial review of the data. The coding scheme comprehensively described all the graphs the teachers produced for these prompts. Coding noted the following properties for each of the graphs made by the teachers: graph type, the variable conditioned on, and whether the vertical axis displayed frequency or relative frequency (omitted for pie charts). If a teacher made a graph that did not include all categories of the conditioning variable (e.g., conditioned on job experience and omitted the category of 'never had a parttime job'), it was noted as incomplete. Graphs that presented the marginal totals only were noted as well. Following creation of the coding scheme, the first and second authors independently applied it to code the graphs created by the participants.
Analysis of the data from the remaining parts of the assessment began with open coding (Strauss and Corbin 1990), an appropriate initial step to attach codes to freeform written text as was the case in these parts of the task. The constant comparative method (Glaser and Strauss 1967) was utilized to minimize the number of codes. In part (b), the teachers were asked to justify their choice of the preferred graph. Open coding was used by the first author to develop a coding scheme for the justifications. Next, both the first and second authors coded the responses to part (b) according to this scheme.
Analysis of the data for part (c), where the teacher provided comments regarding the quality of the student's graph, began with each analyst dividing the responses into statements of approval and statements of deficiencies. Then, each analyst noted what the statement approved of or noted as a deficiency. This was done through open coding except for using two preestablished codes regarding known deficiencies of the graph: missing a label on the vertical axis, and the vertical axis displays frequencies rather than relative frequencies.
Each analyst independently assigned three codes to each teacher's response to part (d). The first code documented if any deficiencies with the student's graph were addressed in the response; the same codes were used for this coding as were used for part (c). The second code noted if and how the crafted response developed the student understanding further. An open coding process (Strauss and Corbin 1990) was used by the first author to develop the possible classifications for the second code. Lastly, the third code noted the level at which the response attended to content and pedagogy, according to the rubric presented in Figure 6. The rubric was designed in a like fashion to one used productively by Groth (2012) to analyze preservice teachers' responses to writing prompts similar to part (d) of this assessment item. The rubric assessed how complete the response was in its attention to both content and pedagogy, and whether the pedagogical approach of the response was more traditional (level 2) or aligned with reform-oriented teaching recommendations (Garfield and Ben-Zvi 2009) (level 3). The first and second author independently coded the responses to part (d), assigning three codes to each response.
After both analysts had independently coded the data, their codes were compared and any discrepancies resolved through conversation. Finally, the coded data were analyzed by the author team for themes to describe the teachers' knowledge for teaching graphing of bivariate categorical data in three ways: before intervention (based on the data from the pre-assessment), after intervention (based on the data from the post-assessment), and any change from using the curriculum materials (analyzing the paired data from pre-to post-assessment by teacher).

Limitations
The study had multiple limitations. The participants in the study were selected from teachers in participating instructors' courses at four schools in the United States; thus, they were not a random sample of mathematics teachers, which could limit the generalizability of the results.
Efforts were made to support consistency in the implementation of the curriculum materials at the four sites, as described in Section 3.2. However, the implementation of the materials was certainly not identical as it was done by different instructors with varying time length of class sessions (75-90 min) and would naturally vary based on students' responses to prompts in the materials (e.g., Question 4-b as shown in Figure 3) and ensuing discussions. Thus, these differences in implementation should be taken into consideration when interpreting the results.
In the absence of a control group, conclusions can only be drawn regarding the changes in teachers' knowledge for teaching graphing of bivariate categorical data following use of the implemented curriculum materials (as stated by the second research question). A future study should be replicated with a control group in place to compare teachers' change in knowledge for those who use the curriculum materials to those who receive customary instruction in an introductory statistics course required for mathematics teachers, particularly since this may have been the participants' first time receiving instruction on these topics. This would allow for broader conclusions regarding the effectiveness of the curricular materials.

Constructed Graphs
First, we consider types and attributes of the graphs the teachers constructed in part (a) of the assessment task to represent the full range of correct, plausible graphs. Table 1 lists the percentages of use of the different graph types and attributes on both the pre-and post-assessments. Recall that the data ( Figure 5) has unequal group sizes for both gender and job experience, thus relative frequencies (percentages) should be utilized and the use of frequencies (counts) is not appropriate. Also, the segmented bar graph was the graphical display emphasized in Table 1. Percentages of use of the graph types and attributes of the graphs constructed by the 59 teachers on the pre-and post-assessments. the curriculum materials for representing bivariate categorical data due to its facilitation of comparison of the groups, so inclusion of that graph was desirable. Since the prompt called for the creation of the full range of correct plausible graphs, it was also preferable for teachers to make graphs that conditioned on job as well as graphs that conditioned on gender. In Table 1 and subsequent tables and figures, desirable features are indicated with a (+) and undesirable features are indicated with a (−) to assist the interpretation of the results. Examining their responses on the pre-assessment, 85% of the 59 teachers constructed a side-by-side bar graph and smaller percentages used a segmented bar graph (31%) or a pie chart (36%). It was unlikely (14%) for teachers to graph relative frequencies, preferring frequencies instead (83% created graphs showing frequencies). When exploring the relationship between job and gender, one has the choice of conditioning on job or gender; a higher percentage (81%) constructed at least one graph conditioning on job than those that constructed any graph that conditioned on gender (49%). Nearly half (42%) of the teachers' responses to part (a) included graphs that conditioned on job as well as graphs that conditioned on gender. Some of the teachers drew graphs that were incomplete (15%) (such as not including the full dataset) or incorrect (12%) (a line graph), but the percentages were relatively small. Table 1 also presents the percentages of use of different graph types and attributes on the post-assessment. Figure 7 graphically displays the change in percentage use from pre-to postassessment. Among all the graphs constructed, there was a considerable increase in the percentage of teachers who constructed segmented bar charts (increase of 33% from 31% use to 64% use) and displayed relative frequencies in their graphs (increase of 40% from 14% use to 54% use), showing that their graphical competence for bivariate categorical data improved. They were also more likely to present at least one graph that conditioned on job on the post-assessment (63% compared with 49%).
Since the participating teachers came from four different schools, one may be interested in seeing if the general patterns were consistent across the schools. Figure 8 displays the percentages of teachers using different graph types and attributes for both the pre-and post-assessments for each school.
Although there is variation in the percentages across schools, the trend across all schools is a marked increase in the creation of segmented bar graphs and graphs that use relative frequencies from the pre-assessment to the post-assessment. Except for School C, all sites showed a decrease or minimal change (one participant) in undesirable features of incomplete graphs and line graphs. The inservice teachers at School C also followed different patterns from the rest in their increase of the creation of graphs that used frequencies and decline in the creation of graphs that conditioned on gender. It should be noted that the sample size at School C was nine teachers, so a change of one teacher is an 11% change.
The results of the paired analysis are now discussed where changes in the individual teachers' responses from pre-to postassessment are shown. Each teacher was classified by his or her use of different graph types or attributes on part (a) of the pre-and post-assessment. Table 2 displays the percentages of teachers who switched their responses (either no to yes or yes to no) or stayed (either remained yes or remained no). For    example, a switch from no to yes means the teacher did not make that graph on the pre-assessment but did make it on the postassessment.
If one looks at the two "switch" columns of Table 2, one notices a substantial percentage of teachers who switched to using segmented bar graphs (37%), relative frequencies (46%), and conditioning on gender (24%). However, there also were notable percentages of teachers who never created a segmented bar graph (32% under the No → No column) or never graphed relative frequencies (41% under the No → No column).

Preferred Graph
After the teachers constructed plausible graphs of the data in part (a), they were asked in part (b) to identify the preferred graph to analyze the situation. Table 3 displays the percentages of attributes of this "best" graph on the pre-and post-assessments.
On the pre-assessment, 75% chose a side-by-side bar graph and likewise a majority (73%) preferred a graph conditioning on job. It was unlikely to see a best graph that used relative frequencies, segmented bars, or conditioned on gender. Comparing the pre-assessment and post-assessment responses, one observes a shift toward a preference of a segmented bar graph (36% compared with 7%); however, most of the teachers still prefer the side by side bar graph (54%) to the segmented bar graph. There is an observable increase in the choice of relative frequencies (46% compared with 10%) on the preferred graph on the post-assessment accompanied by a decrease in preference for frequencies (71% on pre-assessment compared to 44% on post-assessment). Teachers' preference for a graph conditioned on gender also showed change (41% compared with 12%). Figure 9 displays the percentages of teachers using different graph types and attributes on the preferred graph on the preand post-assessments, both overall and by school site. This figure shows a consistent tendency across schools for the teachers to be more likely to use relative frequencies, segmented bar graphs, and condition on gender on their preferred graph post-instruction. In contrast, teachers at all schools (with the exception of School C) were less likely to use frequencies and side-by-side bar graphs in their preferred graph on the postassessment. Consistent with the findings regarding School C's teachers in Section 4.1, these teachers increased their use of frequency in their preferred graph from pre-(four teachers) to post-assessment (five teachers). This was not the case at any of the other school sites.
Part (b) of the item also asked the teacher why he or she preferred the selected graph. The most common justifications named on both the pre-and post-assessment were the ease of comparison of the groups and readability. On the postassessment, teachers were much more likely to say the graph they preferred helped analyze the data for association. These results were consistent across all four school sites. Table 4 displays the percentages of teachers who switched or remained for characteristics of the preferred graph chosen on part (b) of the assessment. Of those teachers who switched their response, they changed to prefer segmented bar graphs (29%), relative frequencies (37%), and conditioning to gender (32%). Also, they decreased their use of side-by-side bar graphs (39%), frequencies (37%), and conditioning on job (25%). For those teachers who did not change their response, the most common result was not making a segmented bar graph on the pre-or post-assessment (64%). Fifty-three percent of the teachers did not use relative frequencies on either assessment.
To further analyze the results regarding the attributes involving frequencies and relative frequencies, Table 5 displays the calculation of the percentage of use of frequencies and relative frequencies by the teachers who constructed side-by-side bar graphs and segmented bar graphs on part (a) of the assessment. Recall that the teachers could construct multiple graphs in response to the prompt, so the total number of teachers producing the named graph differs across each instance and the total number of teachers does not total 59.
After working through the curriculum, the teachers decreased their use of frequencies in both types of bar graphs and increased their use of relative frequencies. These results Figure 9. Percentages of use of the graph types and attributes of the preferred graph on the pre-and post-assessments, overall and by school. Table 4. Percentages of changes of graph types and attributes of the preferred graph constructed by the teachers on part (b) of the pre-and post-assessment. support the claim that the teachers' graphical competence improved following use of the curriculum, particularly their use of relative frequencies when graphically analyzing bivariate categorical data with unequal sized groups in each category. Additional insights emerged following analysis of the relationship between the teachers' constructed graphs and their choice of preferred graph, and on their use of conditioning. On the post-assessment, of the 32 teachers who used relative frequencies in at least one of their constructed graphs for part (a), 27 (84%) chose a preferred graph that used relative frequencies. This indicated that most teachers who made correct graphs that utilized relative frequencies thought it was desirable to use relative frequencies instead of frequencies in the "best" graph. Similarly, of the 38 teachers who constructed a segmented bar graph on part (a) of the post-assessment, 21 (55%) chose a segmented bar graph as their preferred graph. This indicates some desirability to use a segmented bar graph as the "best" graph. Finally, of the 27 teachers who conditioned only one way (either by gender or by job) among their constructed graphs on the preassessment, 12 (44%) of these teachers conditioned both ways on the post-assessment. This indicated that the teachers were learning that correct graphs can be constructed that condition either way.

Analyzing and Responding to Student Work
The assessment task asked the teachers to analyze a studentmade graph (part (c)) and provide feedback to the student to further develop his or her thinking (part (d)). Recall that the student's graph ( Figure 5, part (c)) had two primary deficiencies: use of frequencies instead of relative frequencies (which is inappropriate for this data with unequal group sizes); and lack Table 6. Frequency and relative frequency of the positive comments made by the teachers in part (c) about the presented student's graph on the pre-and postassessment.

Pre-assessment
Post-assessment  of a label on the vertical axis. The results regarding teachers' responses to these parts are presented in aggregate only, as disaggregation by school resulted in too few responses in each category for trends to be recognized. The findings regarding the analysis of student work (part (c)) are presented first, separated into comments that were complimentary of the student's work and comments that noted deficiencies. Table 6 summarizes the positive comments the teachers made about the student's graph. Some of the comments have been grouped into themes of "clarity" (including comments such as organized, clear, title, key, easy to read, and concise), "data" (including comments that all of the data were included and one could reconstruct the data from the graph), and "association" (comments regarding the ability to determine association or compare the groups).
On the pre-assessment, the teachers made a larger number of positive or laudatory comments than on the post-assessment (75 compared with 55). Comparing the percentages of comment types (among all teachers), teachers were more likely to make positive comments about the clarity of the graph and the use of labels on the pre-assessment. Table 7 presents frequencies and relative frequencies of the comments that were critical (i.e., noted deficiencies) regarding the student's graph. Generally, the teachers were more critical (47 comments compared with 31 comments) of the student's graph on the post-assessment, with more of the teachers noting the graphs' deficiencies. The teachers were more likely to mention that the graph did not use relative frequencies on the postassessment (32% compared with 7%). Also, many of the teachers noticed the lack of a label on the vertical axis of the graph on both the pre-assessment (32%) and post-assessment (34%).
We observed an association between the preferred graph type from part (b) and the critical comments made by teachers in part (c), graphically illustrated by the segmented bar graphs in Figure 10. Of the 27 teachers whose preferred graph displayed relative frequencies, 15 (56%) noted the lack of relative frequencies in the graph the student constructed. In contrast, of the 32 teachers whose preferred graph did not display relative frequencies, only 4 (13%) noted the issue of relative frequencies in the student graph. Similarly, there was an association between the use of relative frequencies on any graph in part (a) and a mention of relative frequencies as a weakness of the studentconstructed graph. Of the 27 teachers who did not construct a relative frequency graph, only 8% mentioned relative frequencies as an issue; in contrast, of the 32 teachers who made a relative frequency graph, 53% mentioned relative frequency as something the student-constructed graph needed. Therefore, there is evidence that if a teacher constructs and/or prefers a graph which uses relative frequencies, he or she will be more likely to note the lack of relative frequencies when analyzing students' graphs. Now we shift our attention to the responses the teachers crafted in part (d) for the student whose graph was given in part (c). The analysis showed that there was a change in attention to the deficiencies of label and relative frequency in the student's graph. Twenty-five percent of the teachers indicated an issue with the labels on the pre-assessment, compared with 10% of the teachers on the post-assessment. In contrast, 12% of the teachers on the pre-assessment indicated an issue with relative frequencies compared with 49% of the teachers on the post-assessment. This pattern of response is consistent with the findings from part (c). Since many of the teachers did not note issues with the lack of labels or relative frequencies on the student's graph on the pre-assessment, the teachers' responses often developed the student's thinking by teaching him or her how to make other types of graphs. On the post-assessment, the majority (59%) of the teacher responses directly addressed the issues of labels and relative frequency in the student-constructed graph. Other points attended to in the teacher responses on the postassessment were teaching the student how to interpret graphs and comparing the graph the student created to other graphs with respect to their effectiveness for analyzing association.
The results of the analysis of the teachers' responses to part (d) for attention to content and pedagogy is summarized in Table 8. On the pre-assessment, half of the responses received a "0" code, meaning they made incorrect claims about the content (e.g., claimed the student's graph was without error) or did not attend to content or pedagogy. This declined to 29% on the postassessment. Fifty-six percent of their responses received codes of "2" or "3" on the post-assessment, compared to 40% on the preassessment. Collectively, this shows that the teachers were more likely to craft responses to student work that attended to both content and pedagogy following their work with the curriculum materials.

Discussion of the Results
The goals of this study were to describe the graphical competence of mathematics teachers regarding bivariate categorical data and the impact the curriculum materials had on the teachers' knowledge for teaching graphing of such data. The results showed that prior to instruction, mathematics teachers' Figure 10. Association between preferred graph displaying relative frequencies and criticism of student graph for lack of relative frequencies (Graph 1) and Association between use of relative frequencies in graph and criticism of student graph for lack of relative frequencies (Graph 2). graphical competence for bivariate categorical data (part of their subject matter knowledge for teaching) was weak, with a preference for making side-by-side bar graphs displaying frequencies. This is an incorrect graph for the bivariate categorical data they were given ( Figure 5) because the groups-regardless of which way one conditioned-were of unequal size. This finding is not surprising given the afore-mentioned general findings regarding the difficulties of developing proportional reasoning skills and recognizing proportional situations (Section 2.3) along with the lack of graphical competence by teachers (Section 2.2), but it is the first time that it has been documented for this type of data. Following their use of the curriculum materials, the teachers' graphical competence showed improvement. Points of note are that overall (i) their attention to the importance of relative frequency-when constructing graphs, choosing a preferred graph, and analyzing a student's graph-improved; (ii) they raised their awareness that one can construct a correct graph when conditioning on either variable; and (iii) that a segmented bar graph is a correct and often preferred graphical display for bivariate categorical data. An anomaly regarding point (i) is an increase from preto post-assessment of one teacher (11% increase) from the nine inservice teachers at School C who used frequencies and preferred frequencies in the graphs. Of all the sites, teachers at School C had the highest percentage of use of relative frequencies on their constructed graphs and near the lowest percentage of use of frequencies on their constructed graphs prior to instruction (Figure 8). Their use of relative frequencies on constructed graphs increased on the post-assessment, which was promising, but the addition of one teacher who used frequencies was a move in the wrong direction. Similar trends were seen regarding their preferred graph (Figure 9). They also differed from the other sites in their increase by one or two teachers in the making of incomplete or inappropriate graphs (see Figure 8) for part (a) of the task. It is unclear why the results for the teachers at School C differed from the other sites in these ways. It could be related to a difference in experiences these teachers had when using the materials in class, or it could be that as inservice teachers taking graduate courses they did not have as much time as the undergraduate students at the other sites to work on the materials or reflect on their work with the materials when completing the take-home post-assessment.
There is still much room for improvement in the participants' graphical competence. Forty-one percent of the teachers never made a graph that utilized relative frequencies and 53% never selected a graph that used relative frequency for their preferred graph. This falls well short of the graphical competence we would like teachers to have considering they are responsible for instructing students on this topic. Thus, the results showed that while the teachers made positive gains in their graphical competence, perhaps extended time and/or different activities involving continued work on this topic are needed for teachers to become graphically competent with bivariate categorical data. Parts (c) and (d) of the assessment task assessed the teachers' ability to analyze and respond to student work; this addresses their subject matter knowledge for teaching along with knowledge of content and students and knowledge of content and teaching. Prior to instruction, the teachers were generally laudatory of the student's graph despite its deficiencies of no label on the vertical axis (noted by 32% of the teachers) and use of frequencies rather than relative frequencies (noted by 7% of the teachers). After the instructional unit, the teachers were much more attuned to these deficiencies, with 59% of the teachers' responses addressing both in their response to the student. This observed increase shows that their knowledge for teaching graphing of bivariate categorical data improved over the course of the unit; however, over a third of the teachers still did not note that the student's graph is lacking a label and incorrectly uses frequencies. As with their graphical competence, there is still room for improvement in this area. Lastly, the teachers' proposed student responses were more attentive to both content and pedagogy following instruction and more likely to note at least one error in the graph (instead of saying it was correct). As this part of the assessment is relevant to the high-leverage teaching practice of providing oral and written feedback to students (Garcia and Shaughnessy 2015), it is promising that the materials helped teachers improve in this important practice.

Implications for the Field
Due to the increased emphasis on statistical content in the CCSS-M (2010) and other current curriculum standards, mathematics teachers face high expectations for teaching statistics (Franklin et al. 2015). A coordinated, research-based effort is needed to improve the preparation of preservice teachers to teach statistics, including addressing their knowledge for teaching statistics (CBMS 2012a;Franklin et al. 2015). The pre-assessment data showed that teachers' knowledge for teaching graphing of bivariate categorical data is lacking and needs attention in teacher education. This study also demonstrated that the curriculum materials written to address knowledge for teaching graphing bivariate categorical data generally improved such knowledge; hence, they can be used as a beneficial resource. Also, methods used in the curriculum materials, including tying content work to K-12 curriculum standards and developing teachers' abilities to analyze and respond to student work, can inform future curriculum efforts to improve knowledge for teaching other statistical topics.

Future Directions
Ideas regarding future research studies and teacher education efforts emerged from this work. Regarding ideas for future research, the ability of the curriculum materials to improve teachers' attitudes and beliefs toward statistics as well as the teaching and learning of statistics is important to ascertain, since attitudes and beliefs are influential to teachers' actions (Estrada, Batanero, and Lancaster 2011;Pierce and Chick 2011). Work could also be done to more broadly study the use of the materials with inservice teachers, including studying if any changes are needed to adapt the materials to this group. More research is also needed to understand how students learn and conceive of bivariate data to inform the pedagogical content knowledge categories of the knowledge for teaching bivariate data. Of note, no learning trajectory currently exists that depicts the progression students likely move through as they develop their understanding of bivariate data. On a related note, it is still an open question whether it is advantageous for students to learn about bivariate quantitative data prior to or following learning about bivariate categorical data. Alternatively, perhaps the topics should be learned simultaneously, with common aspects (e.g., graphs, predictions) being addressed with both types of data. Answers to questions like these would contribute to the knowledge base needed by teachers to be effective at teaching statistical analysis of bivariate data.