Do remedial mathematics courses help economics students?

Johan Lagerlöf, Andrew Seltzer, 9 August 2008

a

A

University-level economics makes extensive use of basic mathematics. As many economics professors can testify, this makes the subject difficult for less technically able students. Moreover, expanding participation in higher education (as has been targeted by the UK government) may further worsen the problem. The students who enter higher education due to the expansion are likely to be, on average, less technically able than current students, and thus it is likely that the number of students who struggle with their economics studies due to lack of basic math knowledge will increase over time.

A possible solution to the problem

What can be done to tackle the problem? One possible approach is to offer remedial math courses to students early in their university studies. Supporters of remedial education have argued that it aids less technically able students, who are often from disadvantaged backgrounds, and facilitates their integration into university studies.

In the United States, the use of remedial education at the university level has been extensive. It has also been controversial. In recent years opponents have successfully argued that tax money should not be used in universities to teach high school courses, and many states have trimmed remedial programmes or eliminated them altogether.

A key question when assessing the possible virtues of remedial math courses is whether they actually work. That is, does taking remedial math improve students’ performance in their economics courses relative to how they would have done without it?

Methodology

Measuring the relationship between remediation and student performance is not straightforward.

In most universities, students are assigned to remedial courses based upon their secondary school curriculum. Thus a comparison between remedial and non-remedial students is likely to reflect differences in secondary education as well as the effects of remediation. Alternatively, suppose that a group of students from similar backgrounds were given the choice of whether or not to take a remedial math course. A simple comparison between the performances of students who took the remedial course with those who did not would not provide a valid estimate of the effects of remediation because we do not know the reasons behind the students’ choice of whether or not to take the remedial course. For example, the non-remedial students may have known that they were relatively strong in math and therefore did not need to take it. Alternatively the students who chose to take the remedial course may have on average been more motivated than those who did not. In both cases, the estimated effect of remediation will be biased. In the former case a simple comparison will underestimate the true effects of remediation, and in the latter case it will overestimate the true effects.

Ideally, to deal with these methodological issues, one would like to be able to run a controlled experiment in which a group of students is randomly allocated into a compulsory remedial course, while a similar group is not allowed to take the course. Of course, this experiment is impractical and would raise ethical concerns. Nevertheless, recent studies by Pozo and Stull (2006), Bettinger and Long (2008), and us (Lagerlöf and Seltzer 2008) approximate this ideal controlled experiment, examining the effects of random differences in the probability of being assigned to study remedial mathematics on student performance in economics courses.

The Pozo-Stull and Bettinger-Long studies

The study by Pozo and Stull (2006) examines a controlled experiment in a Principles of Macroeconomics course at Western Michigan University. Students taking the course had access to an on-line tutorial and test, but some of them (randomly selected) were given a small grade incentive to actually do the tutorial and perform well in the test. Pozo and Stull find that the grade incentive had a positive and statistically significant impact on the grade for the course’s midterm exam but a smaller and statistically insignificant effect on the grade for the final exam.

Bettinger and Long (2008) examine the effects of remedial courses in math in the Ohio public university system. In order to control for possible selection problems, Battinger and Long use the estimated probability of receiving remediation as an instrument for having actually received remedial education. They find that remediation had a positive and statistically significant effect on several long-term performance measures. Students with a higher probability of being exposed to remediation were, other things being equal, less likely to drop out of college and more likely to transfer to a higher-level college and to complete a bachelor’s degree. However, their methodological approach leaves some questions about the interpretation of their findings. It might be that remediation had a limited effect in increasing students’ mathematical abilities, but it helped match students to appropriate degree subjects.

A natural experiment in Britain

In Lagerlöf and Seltzer (2008) we examine the effects of remedial math instruction in a British university. The British context is distinct from that of the United States because students study a single subject while at university and cannot switch between programmes without restarting from the beginning. The greater focus means that undergraduate economics courses in the United Kingdom are typically much more mathematical than those in the United States.

Our study makes use of a natural experiment created by the implementation of a remedial mathematics course by the Economics Department of Royal Holloway, University of London.

Prior to 1999, the department did not offer remedial math to any students, but in 1999 a remedial mathematics course lasting seven weeks became compulsory for students who had not studied mathematics beyond age 16 or had performed poorly in secondary-school mathematics. Importantly for our study, the remedial course was not made available to other students. We compare the performance of the cohorts admitted prior to 1999 to that of the 1999 cohort in a variety of compulsory economics courses using a difference-in-difference approach. The methodology allows us to identify the effects of remedial mathematics by comparing the change in performance of the math-deficient students (who were required to take remedial courses after 1999) to the change in performance of students with a stronger math background (who did not take the remedial course prior or subsequent to 1999).

In line with a large body of existing literature, we find:

  • the level of and performance in mathematics courses taken prior to university have strong predictive power on student performance in a range of economics courses.
  • overall performance across all secondary-school subjects has strong predictive power for university performance.

However, contrary to the existing literature, we find:

  • much weaker evidence that taking remedial mathematics has an effect on student performance.
  • no evidence that remedial math has an effect on students’ performance in Principles of Economics or Quantitative Methods, the two core courses in the first year of the economics degree.
  • no evidence of longer-term effects, as performance in the second and third year of the programme is unrelated to exposure to the remedial course.

Surprisingly, the only statistically significant results that we find for the effectiveness of remedial mathematics are for Economics Workshop, a nonmathematical first-year course assessed by two essays, and for the overall average across first-year subjects.

Conclusions

Pozo and Stull (2006) provide convincing evidence that a Western Michigan University remedial programme had a positive short-term effect, but they do not find much evidence that the effect lasted even to the end of a one-semester course. Bettinger and Long (2008) find that increased access to remediation improved a number of short-term and long-term outcomes, although their results may simply reflect the fact that students with access to remediation may have also had access to other resources or that remediation only helped students by matching them to appropriate degree subjects, rather than improving their math skills. Our own results are perhaps more pessimistic still. In Lagerlöf and Seltzer (2008), we find that exposure to a fairly extensive remedial course did not have a statistically significant effect on student performance in any mathematically oriented university-level economics subject.

Overall, it seems fair to conclude that there is currently no evidence that remedial mathematics actually has an effect on performance in university-level economics courses that is (i) long-term and which (ii) can be conclusively attributed to remediation bringing about improved math skills.

References

Bettinger, Eric P. and Bridget Terry Long (2008), “Addressing the Needs for Under-Prepared Students in Higher Education: Does College Remediation Work?” Journal of Human Resources, forthcoming.

Lagerlöf, Johan N. M. and Andrew J. Seltzer (2008), “The Effects of Remedial Mathematics on the Learning of Economics: Evidence from a Natural Experiment”. Journal of Economic Education, forthcoming. Also available as CEPR DP No. 6895.

Pozo, Susan and Charles A. Stull (2006), “Requiring a Math Skills Unit: Results of a Randomized Experiment”. American Economic Review, Papers and Proceedings, Vol. 96(2), pp. 437-441.

Topics: Education
Tags: Economics, mathematics, remedial course

Associate Professor of Economics at the University of Copenhagen and a CEPR Research Affiliate
Professor of Economics and Economic History at Royal Holloway, University of London