Publication lags and young economists’ research output

John P Conley, Ali Sina Önder, Mario Crucini, Robert Driskill, 24 October 2011

a

A

Ellison (2002) documents that the time an economics paper typically spends at a journal between submission and publication has more than doubled over the last 30 or so years, from about 8 to 16 months. As Ellison notes, this has important implications:

The change in the publication process affects the economics profession in a number of ways: it affects the timeliness of journals, the readability and completeness of papers, the evaluation of junior faculty, and so forth.

While all of this is true, the stakes are probably highest when it comes to the evaluation of junior faculty. Slower turnaround times for papers (added to lower acceptance rates at top journals and increases in average page count of published manuscripts) would seem to make it a mathematical certainty that equally capable and hardworking junior faculty will end up with shorter CVs at the end of six years today than they would have in the past, under a quicker and more accepting publishing regime.

In Conley, Crucini, Driskill and Onder (2011), we try to get a sense of the possible magnitude of this effect considering a simple model of research production with either one-period and two-period lags between submission and publication. We assume that individuals begin professional life with a stock of three manuscripts, and write one new manuscript every year. Each year, individuals submit all of their unpublished manuscripts not currently under consideration to a journal which we assume to have a 20% acceptance rate. We find individuals can expect to have 4.52 accepted papers after six years if the delay is one period, but only 2.58 accepted papers if the delay is two periods (a 43% drop in CV length). Clearly the “Ellison effect” has the potential to be quite significant.

If institutions fail to internalise this new reality, then fewer junior faculty will receive tenure than in the past. Of course, at an individual level, the cost of not gaining tenure is large. It should be noted that the costs are large for the profession in general as well. Failure to promote qualified scholars leads to more frequent and costly searches for new junior faculty, the exit of qualified scholars who would otherwise enrich the stock of economic research, and the discouragement of talented undergraduate and graduate students from attempting to make a career in research economics.

It is possible, of course, that young scholars might realise all this and compensate for the new more difficult publishing environment by working harder. Although this might make academic economics a less attractive career, it might also make the CVs of new PhDs more comparable to those of earlier cohorts.

We therefore investigate the effect of these changes in the publishing environment on successive cohorts of new PhDs from an empirical standpoint. We combine data from various sources to reconstruct the JEL-listed journal publication records of the 14,271 graduates of US and Canadian PhD-granting economics departments from 1986 to 2000.

We focus on the approximately one half of graduates who publish at least one paper within six year of finishing their PhD. For descriptive purposes, we group these “publishing” or “research-active” graduates into five cohorts, each pooling three consecutive years of PhD graduates, e.g. the 1987 cohort consists of graduates of either 1986, 1987 or 1988.

To begin with, we find that the productivity life-cycle of different cohorts is very consistent. Figure 1 shows the average annual number of AER-equivalent pages published by research active graduates of the top thirty US and Canadian economics departments. All cohorts show a peak in annual productivity from the 4th to 6th year after graduation, when they reach a publication rate of 1.5 to 2 AER-equivalent pages per year. In subsequent years, annual productivity slowly declines to around 60% of the peak. Clearly, the tenure clock has significant influence on scholarly productivity. The pattern is similar for non-top 30 graduates.

Figure 1. AER-equivalent number of annual pages per publishing graduates

We find a pattern of very skewed productivity which is also consistent across cohorts. Table 1 shows part of an “intellectual Lorenz curve” constructed from our data. We see that the most productive 1% of research-active PhDs produce about 12% to 14% of all quality-weighted publications regardless of cohort, while the top 10% produce about 57% to 58%, and the top 20% produce about 78% to 79%.

Table 1. Intellectual Lorenz curve


 

1987

1990

1993

1996

1999

Top 1%

11.9%

13.2%

14.1%

12.7%

12.9%

Top 10%

56.6%

58%

57.5%

58.1%

58.2%

Top 20%

78.1%

78.4%

78.1%

78.7%

79%

Source: own calculations based on Conley, Crucini, Driskill and Onder (2011)

Our central question is the effect of publication slowdown on the relative productivity of different cohorts. To this end, we considered the number of AER-equivalent pages published at the end of the sixth year (the approximate time that tenure decisions are made). By this measure, among graduates of the top thirty programs, older cohorts are on average more productive than middle cohorts, and middle cohorts are on average more productive than the youngest cohorts. However, there is no such pattern of declining productivity for the non-top 30 departments using this productivity measure. Thus, there is only weak evidence of the Ellison effect.

When we look instead at the number of AER-equivalent publications rather the number of pages published at the end of six years, a much more dramatic and clearer picture emerges. By this measure, among graduates of the top 30 programmes, the oldest cohort is 48% more productive than the middle cohorts, and 68% more productive than the youngest. The middle cohorts, in turn, are 12% more productive than the youngest cohorts. For non-top 30 departments, the oldest cohort is 19% more productive than the middle, and 58% more productive than the youngest, while the middle cohorts are 33% more productive than the youngest cohorts. These numbers are both large and statistically significant. Since tenure decisions are more likely to be made on the basis of the number of lines on a CV than on a more abstract count of published pages, we think that this is the more relevant measure, and the implications for the tenure process are important.

To give a sense of the magnitude of this shift, Table 2 shows the average number of AER-equivalent publications produced by the end of the sixth year for PhDs ranked at the 99th, 90th, and 80th percentiles in their cohorts. This table shows both the extreme skewness of productivity and the significant drop-off of the publication rates of younger generations of new economists.

Table 2. AER-equivalent papers by productivity percentile and cohort


Percentile

1987

1990

1993

1996

1999
99th
3.87

3.06

3.23

2.45

2.48

90th

1.34

0.98

0.85

0.76

0.73

80th

0.62

0.43

0.44

0.37

0.37

Source: own calculations based on Conley, Crucini, Driskill and Onder (2011)

We find that the institution from which students receive their PhDs has a significant impact on both the quality and quantity of their published research. Publishing graduates of top 30 departments produce more than three times as many AER-equivalent pages and papers than do their counterparts from non-top 30 departments. The average quality of each published paper and page is about three times better for graduates of the top programs compared to the non-top programmes, and this holds for all cohorts. However, we do not see much change in the quality of the average publication over time for either top or non-top programmes.

Finally, these data allow us to investigate the relative performance of economics graduate programmes in terms of the research output of their PhDs. This allows us to construct a new type of metric for ranking departments as an alternative to the more traditional methods which focus on the publications of faculty members. We find that MIT, Princeton, Harvard and Rochester do best by this quality measure and, more generally, that the rankings of other departments does not entirely agree with more traditional measures that use faculty output.

These data show that the economics profession is extremely hierarchical, both in the sense that top scholars vastly out-publish even average ones, and that top programmes produce graduates who are significantly better than non-top programmes. Our most important conclusion, however, is that there has been a significant slowdown in the publication rates of junior faculty over recent years, and this is likely due to a more difficult publishing environment than a drop in the quality of new PhDs. This suggests that our profession should be careful when evaluating people for tenure and promotions. The rules of the game have changed and members of more recent cohorts who may be just as talented and hard-working as their predecessors almost certainly will have shorter CVs in comparison.

References

Conley, J.P., M.J. Crucini, R.A. Driskill, and A.S. Onder (2011) “Incentives and the Effects of Publication Lags on Life Cycle Research Productivity in Economics,” NBER Working Paper 17043.
Ellison, G. (2002) “The Slowdown of the Economics Publishing Process,” Journal of Political Economy, 110 (5): 947-993.
Kalaitzidakis, P., T.P. Mamuneas, and T. Stengos (2003) “Rankings of Academic Journals and Institutions in Economics,” Journal of the European Economic Association, 1 (6): 1346-1366.

Topics: Education, Productivity and Innovation
Tags: economic journals, Economics research, tenure

John P Conley
Professor of Economics, Department of Economics, Vanderbilt University
Mario Crucini
Associate Professor of Economics, Department of Economics, Vanderbilt University
Robert Driskill
Professor of Economics, Department of Economics, Vanderbilt University
Ali Sina Önder
Assistant Professor of Economics, Uppsala University; Researcher, Uppsala Center for Fiscal Studies

Most Read