Do government schools improve learning for poor students? Evidence from rural Pakistan

ABSTRACT Pakistan’s Punjab province has witnessed numerous education reforms in recent years. Many of these reforms have been aimed at improving the well-documented low levels of learning by focusing on improving teaching quality. The rhetoric suggests that government schools, particularly those in rural areas with a more disadvantaged pupil base, are especially ineffective at imparting learning. This paper seeks to investigate whether children in rural Punjab are learning literacy and numeracy over the course of a year, and if so, are some pupils progressing more than others. Using recently collected data, it finds that children in our sample are making progress. Variation in progress is found to be greater within schools rather than across them. The competence and qualifications of a teacher also makes a significant difference to a child’s academic progress. The paper further finds differential progress for rich and poor students within schools, suggesting an important role for education policy to put in place targeted support towards those from disadvantaged backgrounds to ensure improvements in their learning keep pace with their peers.


Introduction
Pakistan's education sector has been the focus of global attention over recent decades. Punjab province has been a testing ground for many domestic and donor-funded initiatives ranging from traditional input-based reforms, such as improving school infrastructure, to more radical strategies to strengthen the teaching cadre. Despite a spate of reforms, the policy perception is that teaching quality in the province and in the country remains poor (Government of Pakistan, 2013).
Evidence tends to support this view, with learning outcomes identified as being persistently low, particularly in government schools (ASER [Annual Status of Education report], various years). These data further highlight stark inequalities in relation to location, gender, and socio-economic status (Alcott & Rose, 2015;Aslam, 2012). There is also some evidence that low average achievement is related to poor teaching quality, measured by teacher competence, subject-knowledge, efficacy of training, recruitment, deployment, motivation and absenteeism (Andrabi et al., 2015;Dundar et al., 2014).
Globally, robust empirical research has stressed the importance of improving teaching quality in improving pupil outcomes (Glewwe & Kremer, 2006). However, the extent to which inequalities in achievement between poor and rich children can be explained by differential access to high quality schools and teaching is less understood. Some studies have quantified the impact of teachers on pupil achievement in Pakistan, or explored the importance of particular teacher characteristics in determining pupil outcomes in primary (typically children aged 5-9 years) or middle (aged 10-14 years) schools (Aslam & Kingdon, 2011;Rawal et al., 2013;De Talance, 2017;Bau & Das, 2017). These studies consistently point to the importance of teachers in improving student learning in Punjab, with teacher content knowledge and the 'process' of teaching, rather than the observed resumé characteristics of teachers being important determinants of pupil outcomes.
This paper contributes to this literature by providing new evidence on the extent to which there is a socio-economic gap in the achievement among children in rural Punjab, and whether this is related to students' access to good schools with high quality teaching. Whilst Pakistan has a large private sector, government schools continue to cater to the most socially disadvantaged populations (Alcott & Rose, 2015). Given the emphasis in this paper on socio-economic disadvantage, this study focuses on the achievement of pupils in government schools. It provides empirical evidence using recent data on the achievement gap between rich and poor students in 30 rural villages in Punjab to answer three research questions: (1) Are pupils learning over the course of a year, and what is the variation in this learning both across schools and within schools? (2) To what extent do academic achievement gains vary according to the socioeconomic status (SES) of the student, and is this because of the school they attend and the teacher they are taught by (e.g. do poorer students attend poorer quality schools)? (3) Which particular school and teacher characteristics, if any, are correlated with greater pupil achievement gains, particularly for low SES students?
The paper is set out as follows. The next section reviews relevant literature, with a particular focus on how teacher quality can be measured and the empirical evidence relating to this. This is followed by a description of the primary data collected and the models specified for the analysis. The subsequent section presents key empirical findings and the final section concludes.

Review of literature
Amongst the key policy levers to improve learning are reforms that the state can implement in schools and with teachers. Literature from a range of different contexts points to teachers as the most crucial institutional input into a child's educational experience (Aaronson et al., 2003;Araujo et al., 2016;Britton & Vignoles, 2017;Dundar et al., 2014;Glewwe & Kremer, 2006;Hanushek, 2011;Hanushek & Rivkin, 2006;Nonoyama-Tarumi et al., 2015;Rivkin et al., 2005;Rockoff, 2004). This does not, however, imply that schools do not matter. Clearly the institutional structures that support good teaching are important. However, many studies show that variation in pupil achievement is more highly correlated with measures of teacher effectiveness than with measures of school effectiveness. This also implies that, within schools, teaching quality can be quite variable. In addition, reforms outside of the school could also be influential in some settings, such as raising the literacy levels of the mothers, and changing attitudes towards parental involvement in their children's learning. Our focus in this paper is school and teacher factors, but we also incorporate family circumstances into our model to allow for this important influence on children's development.
Studies of school and teacher effectiveness that have been conducted in Pakistan also confirm the importance of teacher quality for student achievement (Azam & Kingdon, 2015;Bau & Das, 2017;. Bau and Das (2017), using data from rural Punjab, find that moving a student from a teacher in the 5 th percentile of the teacher quality distribution to the 95 th percentile would lead to a 0.64 standard deviation increase in test scores. Studies from neighbouring India have found similar results (Azam & Kingdon, 2015). Most empirical studies attempting to credibly estimate 'effective teaching' globally have struggled to identify the characteristics of effective teachers. Most have found that standard resumé characteristics, such as qualifications and experience, do not correlate highly with teacher effectiveness (Hanushek & Rivkin, 2006;Dundar et al., 2014). Previous literature has suggested this may also be the case in Pakistan (Aslam & Kingdon, 2011;Bau & Das, 2017). Evidence from Pakistan has also shown that poor teacher subject knowledge, as measured by a standardised achievement test, is associated with low levels of pupil achievement (Aslam & Kingdon, 2011). Teacher attitudes, the social distance between teacher and student, and classroom practices also appear to matter more for student achievement than teacher characteristics (Aslam & Kingdon, 2011;Rawal et al., 2013;Rawal & Kingdon, 2010). A large research base globally, and increasingly in the South Asia region, has also found that teacher pay and the nature of contracts impacts on teaching quality (e.g. Alcázar et al., 2006;Bau & Das, 2017;Chaudhury et al., 2006;Eide et al., 2004;Goyal & Pandey, 2013;Hanushek & Woessmann, 2011;Kingdon & Teal, 2007;Muralidharan & Sundararaman, 2011. For example, evidence from Pakistan by Bau and Das (2017) and De Talancé (2017) finds a positive relationship between the nature of a teacher's contract, teacher quality and value added in pupil achievement. 1 Existing evidence identifies that the quality or teaching and, to a lesser extent, of the school, matters for pupil achievement. A key question is whether differences in both school and teacher quality explain the large socio-economic gaps in pupil achievement that we see in most countries. Literature is more scant in this regard. Studies largely from developed countries have shown that poorer students tend to attend schools with less experienced or less well qualified teachers (Hanushek et al., 2004;Lankford et al., 2002;Guarino et al., 2006). However, Torres (2018), using data from Chile, found a high proportion of effective teachers in schools with a very high proportion of low SES pupils, but equally that there is considerable variability in teaching quality within low SES schools. Such analysis is not currently available in the Pakistani context. This paper seeks to fill this gap by providing recent evidence from Pakistan on the extent to which poor children access lower quality schools and potentially experience less effective teaching.
Methodologically, identifying teacher and school effects is difficult because pupils are not randomly allocated to teachers and schools. Higher or lower achieving students may be systematically allocated to particular schools and teachers, confounding the school or teacher effect with the impact of the student's initial achievement. Various strategies to overcome this selection bias have been used in different contexts, including instrumental variables, panel data and randomised and quasi experiments (e.g. Kingdon & Teal, 2010;Hanushek et al., 2005;Lavy, 2002;Bau &Das, 2017 andGlewwe & for an overview). In some contexts, the estimated teacher effects from non-randomised designs have been found to be similar to those from randomised designs for models that condition for prior student achievement, student characteristics and teacher variables (Burgess, 2015). While our research does not adopt a quasi-experimental design and so is not causal, the availability of rich data enables us to condition for factors that influence pupil allocation to teachers and schools. In particular, the availability of data on a variety of controls including the child's innate ability, as measured by adapted Ravens Progressive Matrices administered to each child in the sample, allows a proxy for a commonly unobserved variable likely to bias existing estimates. 2 Such data are rare and allow us to be more confident that we are identifying the additional impact from schools and teachers on pupils' outcomes, over and above any impact from the child's ability.

Data
Primary data was collected from 30 villages in the Punjab province in Pakistan for this study. This province offers an appealing context for our research. The largest and most populous of the four provinces in the country, it has been home to wide-ranging education reforms, 3 many of which have directly targeted reforming teacher quality.
The existing study was based on a sample of three purposively selected rural districts in Central Punjab, with 10 randomly selected villages from within each district yielding a total sample of 30 villages. Within each village, 35 households and one government primary school were randomly selected (or two government schools if they were single sex). At the school level, the research undertook a survey of all children in grades 3 to 5 (approximately 8-12 year olds) in the government school yielding a sample of 50 schools. The survey was conducted during the year 2016/17: data were collected from children in these three grades at the beginning and towards the end of the academic year (10 months apart). Having data on learning outcomes at the beginning and end of the year allows the estimation of 'value-added' models which help overcome some of the methodological limitations associated with assessing school quality based on 'levels' of performance. Value-added models isolate the contribution schools make in improving learning outcomes over time. The decision to sample children in grades 3-5 was made to ensure not only that children already had some experience of primary schooling but also to collect more reliable evidence based on children's self-report. Children were assessed in mathematics and Urdu, the national language of Pakistan and the language of teaching in government primary schools in the province. The assessment instruments were adapted from the Young Lives India cognitive numeracy and literacy tests. 4 The resulting assessments are continuous measures with a maximum score of 22 for literacy and a maximum score of 28 for numeracy. Questions in the test were adapted slightly between the beginning and end of the school year, to ensure they were as comparable as possible while avoiding the possibility of memorising the responses.
The final school sample consists of 1683 pupils for whom assessment data are available at the beginning and the end of the year, as well as the necessary data on other variables that are used in the specified model (see Tables A1 and A2 for descriptive statistics). Attrition was low during the course of the study: 90% of the sample of children who were assessed at the beginning of the year were also assessed at the end of the year. It is important to note that there was a significant difference in learning levels in the baseline between those who dropped out and the sample who stayed in school. The mean baseline numeracy (literacy) score for those who remained in the sample was 14.6, (15) whilst the mean for those who dropped out was 13.2 (13.9).
Rich information was also collected from children on key individual and family background variables such as number of siblings, language spoken at home, household assets, whether the mother works and their reporting of maternal and paternal literacy. Drawing on approaches used with Demographic and Health Survey (DHS) data (Rutstein & Johnson, 2004), an index of household assets was created as a proxy for the SES of the child using Principal Component Analysis. 5 Similar to DHS, assets include whether their house has a table, chair, radio, stove, mobile phone, colour television, bicycle, motorbike, car, fridge, the material with which their house is constructed and the nature of their water supply. Pictures were used to facilitate children's responses. 6 This study faced a challenge common to other studies, namely that of identifying an effective approach to measuring SES from a school sample. However, several robustness checks were undertaken using matched school and household samples and we are confident that the measure used is captures the SES of the child's household, albeit with the measurement error that is always inherent in such measures. 7 Using the corresponding household data (on a sample of 1414 children aged 8-12 years), we also note that nearly one in ten children are not enrolled in school and these children differ from those who are observed in school (e.g. non-enrolled were from more disadvantaged backgrounds). As a consequence, we are mindful that the research is focused on more advantaged students who have access to schooling but equally that it also excluded the more privileged who attended private schools.
Detailed surveys were administered to all teachers teaching grades 3, 4 and 5 within the sampled schools. Teachers' responses on their self-reported individual and background characteristics were gathered. The instrument drew on existing instruments e.g. SchoolTells, Young Lives and 'Learning While You Teach' 8 that have been used in the South Asian context, capturing teachers' expectations and beliefs, aspects of the school environment etc. In total, 190 teachers were surveyed at the start of the academic year (approximately 4 per school), with 75% of the same teachers identified at the end of the year. Their data were subsequently linked to the pupils that they teach in the sample. 9 Teacher attrition from the sample was largely driven by teachers being transferred from the school, as teacher absenteeism was low. The final sample of teachers used in the analysis is reduced as only those teaching Urdu and mathematics in grades 3, 4 and 5 (depending on the model specification) were retained, and any student with missing data on key variables was also excluded from the analysis, along with their teacher. The sample broadly constitutes about 100 teachers on whom all the necessary data are available.
Another particular feature of this research involved the measurement of teacher competency in literacy and numeracy, specifically their knowledge of the subject they teach. To test teachers directly is potentially sensitive and can affect teacher engagement with the study. To overcome this, teachers' literacy and numeracy skill levels were determined by asking teachers to mark a pupil's responses to the assessments that were administered. Each teacher was provided with one student test for the subject they were teaching. If a teacher taught both mathematics and Urdu, then they were asked to mark one of each. 10

Descriptive statistics: learning gains
Over the course of the academic year, pupils in the sample experienced learning gains during the academic year for numeracy and literacy. The mean gain in literacy scores was 2.3 from a test with a maximum score of 22, and for numeracy scores it was 2.7 from a test with a maximum score of 28. 11 Whilst it is difficult to determine whether these gains should be considered large or small, an approximate 10% gain in test scores over one academic year is certainly a measurable gain. Table 1 shows mean numeracy and literacy scores for different groups of students at the beginning and end of the school year (baseline and endline). It indicates that on average girls in school have higher test scores at both points in time compared to boys, which may in part be due to girls from the most disadvantaged backgrounds being more likely to be out of school. Table 1 also indicates that, whilst students with illiterate fathers and mothers have lower test scores, there are relatively modest differences in test scores by SES quartile. This may seem counterintuitive given the existing literature on the significant socio-economic inequalities in academic achievement in Pakistan discussed earlier. However, we have focused on a relatively poor set of rural villages, and within each village we only surveyed children who were in the government school. Whilst there is certainly heterogeneity in socio-economic circumstances as measured by our child-reported asset index (see Figure 1 and Appendix A for descriptive statistics), it is striking that there are only modest differences in test scores by socio-economic background (Table 1).
Another question that this paper seeks to address is the extent to which there is variation in learning (by SES or other demographic factors) across or within such rural government schools in Pakistan. To explore this, a simple school fixed effects model 12 without covariates is estimated and shows that 17% (9%) of the variation in endline test scores in literacy (numeracy) is across schools. Hence much of the variation that is observed in test scores is within, rather than across, schools. It is not the case that all the children in high-quality schools are learning substantially and those in low-quality schools learning very little, rather pupils in the same school are achieving a wide variety of test score outcomes. 13 Variation in pupil outcomes across schools may not necessarily indicate differences in school quality. It is also potentially indicative of sorting of pupils across schools, with higher achieving students (generally high SES students) more likely to attend some schools than others. In Pakistan, such sorting is likely to be associated either with the relative poverty of the geographical location of the school and/or the presence of private schools in the vicinity of the selected government school. 14 Certainly, some schools are more advantaged than others in terms of their pupil intake. This issue is addressed later in the paper by specifying a model estimating the value added by schools and teachers, i.e. changes in pupil achievement between the beginning and end of the year rather than absolute levels of achievement. This, and the use of a rich set of controls, should be more indicative of school and teacher quality and less related to pupil sorting.

Modelling approach
This study uses a value added model, whereby the child's achievement h i , in school s, at time t, is given by h ist and is determined by their prior achievement in that subject in the This graph shows the proportion of the sample for each value of the asset index. The index is constructed from an array of different questions about the material circumstances of the household (see text of paper) and is created using factor analysis. By design, it has a mean value of zero and a standard deviation of 1.
Source: TEACh data set, school sample.
previous periodh istÀ1 , their family background and demographic characteristics as measured at the beginning of the period X istÀ1 , and likewise on their school and teacher characteristics, also measured at the beginning of the period Z istÀ1 . Separate equations are estimated for literacy and numeracy achievement. As there are multiple pupils per school, it is important to recognise the clustered and hierarchical nature of the data. A common modelling approach in education is a multi-level (random effects) model. However, a Hausmann test rejected a random effects specification in this instance. 15 A school fixed effects model is estimated which imposes fewer assumptions and specifically does not require the assumption that the school effects are uncorrelated with the other explanatory variables (an assumption required for the random effects model). In the fixed effects model, mean differences across schools are estimated by a time invariant fixed effect S s (see Clarke et al., 2013 for a discussion of these model specification issues). 16 The model includes teacher characteristics as correlates. However, there may be unobserved variation in teacher quality, as distinct from school quality, that would ideally be incorporated into the modelling. In this study, however, there are relatively few teachers in grades 3-5 per school (in our analysis sample, around 1.9 teachers in these three grades per school). This is primarily due to multi-grade teaching, with 54% of teachers reporting teaching in multi-grade classes. Hence, as is the case in previous studies in this context (Bau & Das, 2017), identification of teacher effects is problematic where it is not possible to observe multiple teachers teaching each grade in the school. Given this, a school fixed effects model (1) is estimated that incorporates teacher characteristics as described above. This model is intended to determine the extent to which pupil achievement gains are correlated with the school attended. It allows for mean differences in pupil value added across different schools, but in schools with just one teacher it essentially measures the teacher value added which cannot be identified separately from the school effect. For comparison purposes and to assess the robustness of our findings, a teacher fixed effects model is also estimated which generates an estimate of the test score value added by each teacher (TVA) in the sample (see Chetty et al., 2014). We then determine which teacher characteristics are correlated with our estimates of TVA (in a similar manner to Bau & Das, 2017). 17

Model specification
Using equation (1), the relationship between household SES and pupil achievement gains (in literacy and numeracy respectively) is estimated in the first instanceas shown in Tables 2 and 3, column 2 (column 1 provides a standard OLS regression for comparison purposes). The model presented in column 2 of both tables includes prior student achievement in a valueadded formulation which conditions on the test score at the beginning of the year and therefore shows the association between the pupil's SES and their learning gain over the year. Column 3 in both tables includes additional variables that capture other aspects of the child's socio-economic circumstances and demographic factors. 18 Column 4 includes school and  ) teacher characteristics. Column 5 shows results that include teacher literacy and numeracy scores: this reduces the sample size and hence is shown as an additional model. The additional controls include a range of indicators related to child and family background as suggested by the theoretical and empirical literature. Parental literacy is included as a measure of parental human capital. Additional controls used in prior literature include: the grade in which the child currently studies, their gender, age and home language. The Ravens score proxies for innate ability. The numbers of siblings that the child has, as well as whether the mother works, are also included as controls.
As a measure of parental support to the child's learning, an indicator is also included responding to the question: 'Does someone in your home help you with your schoolwork?'. 19 Of key interest is the role of school, and particularly teacher characteristics, in determining pupil achievement gains. The model in Tables 2 and 3 in column 4 includes quantifiable characteristics of the school environment and the teacher, again selected on the basis of evidence from the existing literature. Specifically, the gender of the teacher, the teacher's academic qualification level, their teaching experience, and whether they are teaching in a multi grade classroom, are added as control variables. In keeping with education literature, the size of the class is included as an independent variable. In column 5, the literacy or numeracy skill of the teacher is added to identify to what extent the teacher's own knowledge is associated with pupil outcomes in the same subject. 20 This paper also explores the extent to which school and teacher characteristics might influence the achievement of low or high SES pupils differently. For this, separate models are estimated for low and high SES students, defined as the bottom and top quartiles of the household asset index respectively. Results are presented in Tables 4 and 5. As sample sizes are more limited, the standard errors are larger in these models, making it harder to detect relationships.

Results
As demonstrated above, pupils in rural Punjab are showing progress in numeracy and literacy on average over the course of the academic year. Where there are differences in progress rates, however, these appear to be more apparent within rather than across Source: TEACh data set, school sample. SE in parentheses. *** p < 0.01, ** p < 0.05, * p < 0.1 The base case in the final model is a student in year 3, female, speaking Urdu at home, whose parent does not help them with homework, with an illiterate father and an illiterate mother, whose mother does not work, who is taught by a female teacher, whose teacher has a below degree level education, whose teacher has more than 3 years work experience and who is in a single grade classroom. Source: TEACh data set, school sample. SE in parentheses. *** p < 0.01, ** p < 0.05, * p < 0.1 The base case in the final model is a student in year 3, female, speaking Urdu at home, whose parent does not help them with homework, with an illiterate father and an illiterate mother, whose mother does not work, who is taught by a female teacher, whose teacher has a below degree level education, whose teacher has more than 3 years work experience and who is in a single grade classroom. Source: TEACh data set, school sample. SE in parentheses. *** p < 0.01, ** p < 0.05, * p < 0.1 The base case in the final model is a student in year 3, female, speaking Urdu at home, whose parent does not help them with homework, with an illiterate father and an illiterate mother, whose mother does not work, who is taught by a female teacher, whose teacher has a below degree level education, whose teacher has more than 3 years work experience and who is in a single grade classroom. Source: TEACh data set, school sample. SE in parentheses. *** p < 0.01, ** p < 0.05, * p < 0.1 The base case in the final model is a student in year 3, female, speaking Urdu at home, whose parent does not help them with homework, with an illiterate father and an illiterate mother, whose mother does not work, who is taught by a female teacher, whose teacher has a below degree level education, whose teacher has more than 3 years work experience and who is in a single grade classroom.
schools. Whether the trends observed in the simple descriptive statistics are robust is further explored in this section, using more advanced statistical techniques. Tables 2 and 3 present the results of the determinants of the endline numeracy and literacy scores with column 1 in each table presenting the ordinary least squares (OLS) estimates, column 2 presenting fixed-effects results with some controls and each additional column in the table adding further controls into the specification. Column 1, which presents the OLS specifications controlling for SES and parental literacy, suggests that pupil achievement in numeracy is not correlated with students' SES once the model controls for baseline scores. 21 However, higher SES students do have higher levels of numeracy. Both the OLS specification (column 1) and the fixed effect model (column 2) also suggest that initial baseline numeracy is a crucial determinant of pupils' endline numeracy scores. This highlights the fact that prior achievement is a key driver of subsequent achievement, and its inclusion renders the SES variable insignificant. This implies that whilst high SES students have higher levels of initial numeracy they do not make more progress during the year than low SES students. Other socio-economic indicators, such as parental literacy, are also statistically insignificant once controls for prior achievement are added. In other words, our measures of SES do predict students' initial levels of achievement but do not predict their progress in numeracy. Given the importance of prior achievement in determining pupil outcomes later in life, how early life disadvantage due to SES manifests itself, and how this accumulates over the years of education, remains an important policy consideration. Whilst these results do not show large differences across socio-economic groups in the progress a child makes over the course of a year, the fact that their prior achievement (which is in turn determined by their SES) does determine their progress, means that a continued focus on improving learning for the poorest children is needed.
Column 3 of Table 2 includes various other demographic factors in the model. Once a wider set of demographic factors have been included, the grade of the student impacts on the progress they make, with higher grades making more progress. Controlling for grade, older children in their year make less progress. This is consistent with some weaker learners repeating grades. 22 As mentioned earlier, this research is particularly interested in which school and teacher characteristics are correlated with greater pupil achievement gains, particularly for low SES students. Some of these aspects are explored in column 4 in Tables 2 and 3. In Table 2, most of the school and teacher characteristics are statistically insignificant in the model, though the school fixed effects remain highly significant and the Intra Class Correlation (ICC) is larger in this conditional model (16.7%) implying that around 17% of the variation that we observe in pupils' test score gains over the period is attributable to the school that they attend. The overall impact of the school attended on pupils' learning is therefore important, even if it is not possible to identify the specific characteristics of schools and teachers that are correlated with gains in achievement. However, in relation to teachers, it is notable that children who were taught by a more academically qualified teacher (with a Masters degree) appear to have greater gains in numeracy scores. Additionally, as shown in the final column (5), the teacher's own numeracy score is also positive and statistically significant. This indicates that pupils make more progress if they are taught by teachers with higher levels of numeracy themselves. Whilst prior research has shown that teacher's own competence in the subject matters in determining their pupils' outcomes, this research has additionally shown that teacher competence also matters in determining pupil progress over the course of a year, at least for numeracy. Table 3 shows the results for literacy. In this instance, SES remains a significant predictor, even in the value-added specification that controls for baseline literacy scores. This finding holds in the OLS regression (column 1) and in the school fixed effects model (column 2). Whilst prior achievement is still highly predictive of future achievement, when it comes to literacy, SES continues to have an ongoing role in explaining student progress. Students from poorer households make less progress than their more advantaged counterparts. This is consistent with previous literature which has found a greater influence from family background and home environment on pupils' literacy than numeracy, potentially because language development is more influenced by the nature of the language used in the home environment than the development of other skills such as mathematics. The school attended is also highly predictive of pupil achievement, suggesting that attending a high-quality school makes a difference to the progression a child makes in literacy. In the fixed effects models, the school effects are statistically significant, and in the full model (column 3) the ICC is higher than in the equivalent numeracy model, at 22.2%. This implies that nearly a quarter of the variation in pupils' progress in literacy is attributable to the particular school that they attended.
Similar to the numeracy findings, children in higher grades make more progress. In literacy, students with higher innate ability, as measured by the Ravens Progressive Matrices, also make more progress (column 3 in Table 3). In the final columns (columns 4 and 5) the results suggest that children who are taught by more academically qualified teachers (those with a Bachelors or Masters degree) make more progress in literacy, and this finding is consistent with the numeracy results.
Controlling for teachers' skill levels in literacy (column 4), counterintuitively, students with more literate teachers also had lower test scores on average. One possible explanation could be that, if new teachers have higher literacy levels and if these newer teachers are allocated to schools in more difficult circumstances and where pupils learn less, it may appear that literate teachers are associated with less pupil progress. We cannot explore this potential explanation in our model. However, we have data on the nature of the teacher's contract: 15% of teachers reporting teaching literacy in our sample identified as being on a temporary contract, which is associated with newer teachers in their probationary period. Including a variable controlling for an early career teacher in the model suggests that newer teachers are more academically qualified, which may reflect attempts by policy-makers to improve the quality of new teachers. Tables 4 and 5 show results using the same models separately for children in the low (bottom) SES quartile and high (top) SES quartile. Due to smaller sample sizes, many of the coefficients become statistically insignificant. Interestingly, however, in Table 4 prior achievement in numeracy is a stronger predictor of future achievement for richer students. In other words, an initially high-achieving rich child is likely to have higher achievement in mathematics at endline than a similar poor child who starts the year with relatively high levels of achievement. Since the richer child will make more progress, the gap between rich and poor will widen. Hence, despite SES not being associated with progress in numeracy for the whole sample, there is some evidence that poorer students are less likely to reap the rewards of having initially higher levels of achievement as compared to richer students. Turning to literacy in Table 5, there is less of a difference in the role of prior achievement for low and high SES students. For both groups, prior achievement in literacy is a strong predictor of future achievement, suggesting that literacy requires a strong base to build on and that students (whether high or low SES) who have low initial literacy are likely to struggle. 23

Discussion and conclusions
This paper explored whether students in a sample of schools in rural Punjab are learning, and the extent to which learning varies across and within schools. Using a rich dataset from 2016/2017, and robust statistical techniques, we show that, on average, pupils in the sample did indeed experience learning gains in both literacy and numeracy. This finding counters the discourse that pupils are not learning in rural government schools in the province. In this sample, there was considerable variation in the baseline achievement levels within schools, with less variation across schools. This finding suggests that differences in academic progress cannot simply be attributed to some pupils accessing better schools. Substantial variation in progress was also observed within schools. This indicates that within the very same school, different pupils are learning at very different rates and that their prior achievement is the key factor determining their achievement level at the end of the year. From a policy perspective, this is a crucial indication that differences in academic achievement manifest themselves early on and continue to cumulatively impact on a child's learning on an ongoing basis.
The second question addressed is the extent to which learning inequalities can be explained by pupils' socio-economic background, or other demographic characteristics. The modelling approach used attempted to identify the factors that predict pupils' learning gains during the course of the year. It was found that whilst pupils' level of achievement does vary significantly according to the SES of the pupil, this is not the case for progress in numeracy. While children starting with higher levels of numeracy at the beginning of the year achieve far higher levels of numeracy by the end of the year than those starting with lower skill levels, progress during the year was not significantly related to SES or other demographic characteristics. By contrast, for literacy, the socioeconomic background of the pupil did affect their progress during the year. Irrespective of what level of achievement the pupil starts the year with, those from poorer households make less progress. Hence some of the inequalities that have been observed in achievement levels in literacy are attributable to the ongoing negative impact of socioeconomic background. Some care is needed since it may be the influence of home that is being observed in the data rather than any failing on the part of schools per se, although it could be argued that it is a failing of schools not to close the socio-economic literacy gap irrespective of its causes. Hence, from a policy perspective, this finding is important, highlighting the need for schools (a conducive policy lever to improve pupil outcomes) to provide pupils with opportunities that they may not otherwise have due to the socio-economic disadvantages they face.
We do not discount the importance of interventions focused on families and the home learning environment but our evidence does shed light on the importance of teaching within schools. Specifically, this paper investigated the importance of school and teacher characteristics in explaining any differences in pupil achievement. Both school and teacher fixed effects were highly significant in our models. This indicates that which school the pupil attends and which teacher they have both make a difference to their academic progress.
The attempt to determine specifically which particular school and teacher characteristics explain why some pupils make more progress than others provides rather mixed evidence, though some distinct findings did emerge. For numeracy, and to an even greater extent literacy, having a more qualified teacher was associated with greater pupil progress. Previous literature has generally found that teacher academic qualification levels do not necessarily predict pupil value-added. Given the focus of the Government of Pakistan in prioritising hiring more qualified teachers, this paper's finding that pupils taught by more qualified teachers make more progress is encouraging and supportive of these efforts. Future research could usefully investigate whether it is the content of specific postgraduate qualifications in the Pakistan system post reforms that may explain this greater teacher effectiveness in this more recent data: this paper has simply observed that teachers who have higher levels of academic qualifications are more effective. Such teachers may be more able in other respects that we do not observe, and we are not able to fully investigate the dynamics and causal nature of this relationship.
Importantly, this paper has also found that being taught by a teacher with higher levels of numeracy themselves is associated with greater pupil achievement gains. With teacher competence in the subject being a policy amenable tool and having been shown to be related to increasing pupil outcomes encourages investment in this type of policy intervention. In relation to teacher recruitment, this finding highlights how one might identify teachers of mathematics who are likely to be more effective based on their own competence in numeracy. In relation to teacher training, this research reiterates the importance of high-quality training programmes that focus on improving teacher competence in mathematics. For literacy, however, the result was reversed with teachers who had higher literacy levels being significantly less effective. It is not clear why this might be the case. However, it is noted that in our data less experienced teachers have much higher levels of literacy than long-standing teachers. Although we control for teacher experience in our model, we cannot discount the possibility that some less experienced teachers with higher levels of literacy are being allocated to schools where pupils have lower levels of prior achievement and are likely to learn less.
Overall, this paper highlights that government schools in rural Punjab are indeed improving students' literacy and numeracy skills, an encouraging finding given the recent spate of education reforms in the province. Access to good schools was also found to be important for low and high SES students as it explains a substantial amount of the variation that is observed in pupils' learning. However, variation in progress is greater within schools than across them. Therefore, unpacking what happens within the school, how the differences manifest themselves, and what can be done to mitigate them, remains an important research and policy consideration. Furthermore, evidence has been found that there is differential progress for rich and poor students within schools, suggesting that the attainment gap is in danger of widening unless corrective measures are put in place. 24 Importantly, in line with previous research, this paper has also shown that teachers are key for children's learning. Whilst the paper has highlighted some aspects of teacher characteristics that can make a difference, further research that can identify what is needed to enable teachers to accelerate progress for the most disadvantaged students would be a valuable next step. Notes 1. The policy environment in Pakistan has since changed. Our data reveal only 15% of the teachers in rural Punjab identify as 'contract' teachers, and these are teachers on a threeyear probation. 2. See Aslam and Kingdon (2011) for previous use of the Ravens test in the Pakistani context. 3. https://www.economist.com/briefing/2018/01/04/pakistan-is-home-to-the-most-freneticeducation-reforms-in-the-world. 4. Information on the Young Lives assessments are available on their website: https://www. younglives.org.uk/content/our-research-methods. See also Iyer and Moore (2017). 5. We retained the first component in the Principal Component Analysis, the Kaiser-Meyer-Olkin measure of sampling adequacy was 0.71. 6. The survey can be found here: http://www.educ.cam.ac.uk/centres/real/downloads/TEACh_ Child%20background_in%20school_Field%20version_Pakistan_CC.pdf. 7. Further details on the robustness checks are available from the authors on request. 8. For details of Learning While You Teach (LWYT) see: http://ideaspak.org/people/item/276learn-while-you-teach. 9. Head teachers who also teach in grades 3-5 are also included in the analysis. 10. Recognising potential sensitivities, the purpose of collecting the data was explained to teachers, who could choose to opt out. 11. We standardised test scores for ease of interpretation in our regression analyses. 12. A fixed effects model is equivalent to estimating a model allowing for mean differences in pupil achievement by school. 13. This is interesting as literature from developing contexts has pointed to schools varying far more between themselves than in more industrialised contexts (see Scheerens, 2001, for a review of the evidence). 14. The relative poverty of the geographical area and availability of private schools are also likely to be closely associated with each other. 15. A Hausmann test rejected the null hypothesis that the coefficients from a random and fixed effects model were the same with a chi squared statistic of 50.39, significant at the 1% level. 16. Models are estimated using STATA version 15.1, fixed effect command. 17. This model is estimated as a robustness check and further details on specification and modelling available from authors on request. 18. The specification used for column 3 is also used to estimate our teacher fixed effects model as shown by equation (2). 19. Responses include always, sometimes, never. 20. The correlates in columns 4 and 5 are also used in our model of TVA (as shown by equation 3). 21. Note that if the baseline test score measure is excluded, SES is correlated with pupil achievement as expected. 22. Other demographic factors are insignificant. 23. As a robustness check, teacher fixed effects models are estimated to generate predictions of Teacher Value Added. Results are available on request. The TVA scores for each teacher were then regressed against teacher characteristics and the results are reported in Appendix Tables A3 and A4 and the results are broadly consistent with the school fixed effects model. 24. This finding is similar to recent evidence from Young Lives data in neighbouring India, albeit for students in grade 9, which has similarly shown that attainment gaps can widen over time (Rolleston & Moore, 2018).  Source: TEACh data set, school sample. SE in parentheses. *** p < 0.01, ** p < 0.05, * p < 0.1