Many children left behind? Textbooks and test scores in Kenya

Paul Glewwe, Michael Kremer, Sylvie Moulin , 20 August 2008

a

A

Education is often seen as critical to economic development. In recent years, more and more children have been going to school. The challenge moving forward is arguably to improve school quality.

While there is ample evidence that reducing costs can increase access to schooling, it is much less clear how to improve school quality. Some argue that the key issue is funding, while others, pointing toward the low correlations between spending and student outcomes, argue that reforming education systems is more important. However, even those who are typically sceptical about the impact of funding typically believe that provision of such a basic input as textbooks is important to learning, particularly in very poor schools where few students have textbooks. Indeed some argue that non-teacher inputs, such as textbooks systematically receive too little investment relative to spending on teacher salaries.

Recent research suggests that distortions in education systems may limit the impact of even basic inputs such as textbooks. We find that a textbook distribution program conducted in Kenya by International Child Support Fund Africa (ICS), a Dutch NGO, raised scores for students with strong initial academic background, but not for the typical student. We argue that this is likely due to distortions in the educational system in Kenya. It is worth noting that non-experimental analyses of previous textbook programs in the area suggest that textbooks have a positive effect on average test scores.

Textbooks and test scores

ICS provided assistance to 100 schools in western Kenya over a four-year period. Schools were listed alphabetically within geographic divisions, and these lists were combined into a single list by the alphabetical order of the division names. Then, every fourth school on this list starting from the first school was assigned to the group that would receive textbooks in the first year of the program. Every fourth school starting from the second, third, and fourth on the list received assistance in the second, third, and fourth years of the program, respectively. This procedure implies that the schools that receive the textbooks in the first year should not differ in expectation from the schools that received inputs later. After one school year, the difference in the average test score (averaged over English, math and science) between the textbook schools and the schools that did not receive textbooks was not statistically significant. In fact, we can reject the hypothesis that the textbook provision raised average student test scores by 0.13 or more standard deviations, about five percentile points. Moreover, under some alternate specifications that compare grades and subjects that received and did not receive textbooks, we can reject effects as small as 0.07 standard deviations of test scores. The program also failed to reduce grade repetition, dropout rates, and student absence.

The textbooks, however, did improve scores for the strongest students - specifically, it raised scores by a statistically significant 0.22 standard deviations for students whose pre-test scores were in the top quintile. Moreover, students in eighth grade in the textbook schools were more likely to enter secondary school than eighth grade students in the comparison schools, a finding consistent with textbooks’ being most helpful to initially high-achieving students since only academically strong students go on to secondary school.

There is reason to believe that Kenya’s curriculum and the instructional materials prepared around that curriculum may be well suited towards students in the top of the distribution and less well suited to students further down in the distribution. For example, the language of instruction in the Kenyan education system, after the first few years of school, is English, most students’ third language. In such a situation, it is not surprising that many students may have had trouble using the textbooks. Indeed, the median students in lower grades had difficulty even reading the textbooks. In grade 3, only 16% of the median students in program schools could read the grade 3 English textbook, and only 28% of their grade 4 counterparts could read the grade 4 English textbook. Difficulty reading the textbooks was less pronounced in higher grades (the figures are 67% for grade 5 and over 90% for grades 6-8), partly because many weaker students drop out before reaching the higher grades.

Such a mismatch between curricula and education materials and the needs of the majority of the population may be a common feature of centralised educational systems in contexts where there is great heterogeneity in students’ educational preparation due to rapid expansion of education and where elites have disproportionate power in setting the curriculum. Many developing countries’ education systems have changed little since they were implemented in colonial times for the purpose of producing a small bureaucratic elite.

Educational expansion and structure

The recent expansion of education in these countries has created an influx of students with different levels of preparation. Kenyan students are extremely heterogeneous in their family background, preparation for schooling, and economic circumstances. Middle-class children in Nairobi and other cities grow up with constant exposure to English, good nutrition, and electricity, while the children of subsistence farmers hear very little English until they go to school, have poor health and nutrition, have no electricity, often miss school, and have teachers who are often absent. Although the gross enrolment rate in primary school rose to 90% by 1980 (Deolalikar, 1999) from 47 %in 1960, the curriculum remains better suited to children with educated parents. In the United States, the expansion of secondary education over the first half of the twentieth century was followed by a transformation of the curriculum (e.g., dropping Latin) and the incorporation of vocational education (Ravitch, 1983). Developing countries that are undergoing similar rapid expansions of education may need to undertake similar transformations of the curriculum.

Once Kenya had made the decision to have a uniform, centralised curriculum, presumably in part for nation-building reasons, no curriculum could perfectly suit all of its students. But why did it select a curriculum that served only the top quartile, rather than a broader group? For most of its post-independence history, the elites of Kenya have held disproportionate political power, and they tend to prefer an educational system targeted towards their children. Even parents of average students may favour a curriculum designed for stronger students in order to secure more desirable peers in their children’s schools, because a curriculum best suited for the typical student may cause elites to switch from government schools to private schools. Teachers also have incentives to demand advanced textbooks and devote less effort to pupils who cannot read them; primary schools are judged by students’ scores on the KCPE exam (Kenya Certificate of Primary Education), a national exam administered at the end of grade 8, and students who drop out of school before taking the KCPE exam – about half of all students – are excluded from calculations of school performance.

Possible solutions

Evidence is accumulating from other evaluations on strategies to help less prepared students who fall behind, many of which randomly assign schools to treatment and control groups. One option is remedial education for children who cannot keep up with the official curriculum. Banerjee, Cole, Duflo, and Linden (2007) examined a remedial education programme in India and found it to be very effective. As in Kenya, India’s curriculum appears to be too difficult and inappropriate for many students (Pratham, 2005). The programme offered remedial education to children who reach grade three or four without mastering basic skills; these children left the classroom to receive tutoring for two hours per day. A randomised evaluation of the program shows that test scores in treatment schools increased by 0.14 standard deviations in the first year and by 0.28 standard deviations after two years, and the weakest students benefited the most.

Another option would be to allow different schools or different programs within schools to teach the curriculum at different speeds, as in Singapore. In the same region of Kenya, another recent study found that tracking students by their initial achievement increased test scores for all students – both the initially high achievement and the initially low achievement students – by 0.23 to 0.27 standard deviations (Dupas, Duflo and Kremer, 2007).

In general, more evidence from these types of studies is needed in order to design interventions that can effectively raise student achievement in developing countries. Future research should also examine reforms that reorient curricula or offer vocational education.

Conclusion

Providing textbooks to students who lack them seems to be an obvious way to improve educational performance. Textbook provision is almost universally accepted as an effective education policy, even by those who doubt the effectiveness of increased school spending. Yet our results show that providing textbooks to rural schools in Kenya did not increase average test scores, although it did increase the scores of students with high initial achievement. The latter finding suggests that the official textbooks are ill-suited for the typical student and may reflect more fundamental problems with centralised educational systems, heterogeneous student populations, and entrenched elite power. Remedial education and suitably designed achievement tracking may be promising ways to address these larger problems.

References

Banerjee, Abhijit, Shawn Cole, Esther Duflo and Leigh Linden (2007), “Remedying Education: Evidence from Two Randomized Experiments in India,” Quarterly Journal of Economics, 122 (3), pp. 1235-1264.

Deolalikar, Anil B. (1999) “Primary and Secondary Education in Kenya: A Sector Review.”

Dupas, Pascaline, Esther Duflo and Michael Kremer. 2007. "Peer Effects, Pupil-Teacher Ratios, and Teacher Incentives”.

Glewwe, Paul; Kremer, Michael; Moulin, Sylvie (2008) “Many Children Left Behind?: Textbooks and Test Scores in Kenya”. American Economic Journal: Applied Economics, Forthcoming.

Pratham Organization (2005), “Annual Status of Education Report,” (Pratham Resource Center, Mumbai).

Ravitch, Diane (1983) “The Troubled Crusade: American Education, 1945-1980,” New York : Basic Books

Topics: Education
Tags: education, Kenya

Director, Center for International Food and Agricultural Policy, Stanford University

Gates Professor of Developing Societies in the Department of Economics, Harvard University

Curriculum specialist and language teacher at the Rabat American School, Morocco