The Journal Impact Factor (JIF; average number of citations to a paper in a journal in the first two years since it was published) is such a poor predictor of an individual paper’s citation count that quoting it is a sign of statistical illiteracy, yet cursory judgements about the quality of scientific papers are routinely based upon of the JIF of the journal in which they are published. At one end of the spectrum, this means that a paper in Nature or Science is automatically assumed to be so important that a single one on a CV can be a passport to job interviews, speaking invitations, jobs and promotions. At the other, papers in discipline-specific journals can be passed over, assumed to be unimportant, despite being more useful to the group of people that actually read them.
Volcanology journals are an example of the latter; it’s a small field and, with just a few volcanologists to cite each other’s work, the JIFs of the specialist journals are quite low. Nevertheless, they are clearly very important to the world of volcanology. I wanted to find out what the most important journals in volcanology are, and if there was any correlation with the JIF.
To do this, I looked at the reference list of a recent review article, How volcanoes work: A 25 year perspective, written by Kathy Cashman and Steve Sparks. Both authors have distinguished careers and extremely wide-ranging interests so can be trusted to give a reliable overview of the subject. In the paper, they “focus particularly on the physical processes that modulate magma accumulation in the upper crust, transport magma to the surface, and control eruptive activity“, which is actually a huge scope. Volcanologists working with data from the depths of the mantle and lower crust or from orbit on satellites may feel a bit neglected, but most of physical volcanology is represented.
The paper cites a whopping 364 references. I extracted the journal or book title from each, and with a few lines of Python code, got counts of which were cited most. The number of counts is a proxy for the importance of the journal in the field of volcanology.
The most important journals in volcanology
The most cited journals by Cashman and Sparks are the Journal of Volcanology and Geothermal Research and Bulletin of Volcanology. Both are respected journals, well-read by volcanologists, so it is no surprise that they are the most important journals in the field. JGR and EPSL are next. Between them, these top four journals represent just under half (175/364) of all papers cited in the review.
Nature and Geology feature in 5th and 6th, showing that some important volcanology papers are published there, but they certainly do not dominate in the way that their JIFs would suggest.
Comparison between JIF and citations in Cashman and Sparks (2013)
Comparing the number of citations in Cashman and Sparks (CCS) with the JIF (2 year citation data from SCImago; top 20 journals only) shows an overall lack of correlation. The data can be divided into three groups:
- Journals with JIF > 5: These ‘big hitters’ show a positive correlation between JIF and CCS, but, with the possible exception of Nature, they are not the most important journals to volcanologists.
- Journals with CCS > 10: The four journals shown to be the most influential in volcanology all have a modest JIF that does not reflect their significance within the subject. With Nature as an exception once more, the most important journals in volcanology actually show a slightly inverse correlation with JIF.
- Journals with JIF < 5 and CCS < 10: These show no correlation between JIF and CCS.
The data show that the most important journals in the field of volcanology, based on citations in a comprehensive review of the advances of the past 25 years, are Journal of Volcanology and Geothermal Research and Bulletin of Volcanology. These are discipline-specific journals with low JIFs. Consequently, the JIF of these journals is of especially little value in assessing the quality of the work in their articles or the importance of their contribution to the field.
EDIT 2014-03-12 21:30. Of course the CCS is just another journal-based metric, so it still can’t tell you anything about an individual article or scientist.
If you don’t believe that journal-based metrics, such as Journal Impact Factors, should be used as a surrogate measure of the quality of individual research articles, or to assess an individual scientist’s contributions, or in hiring, promotion or funding decisions, then you can join over 10,000 others in signing the San Francisco Declaration on Research Assessment.
Cashman KV, Sparks RSJ (2013) How volcanoes work: A 25 year perspective. Geological Society of America Bulletin. doi: 10.1130/B30720.1
Results in Full:
Journal of Volcanology and Geothermal Research,57
Bulletin of Volcanology,47
Journal of Geophysical Research,40
Earth and Planetary Science Letters,31
Journal of Petrology,12
Geochemistry Geophysics Geosystems,11
Geological Society of America Bulletin,10
U.S. Geological Survey Professional Paper,10
Geophysical Research Letters,9
Geological Society of London Memoir,7
Contributions to Mineralogy and Petrology,6
Reviews in Mineralogy and Geochemistry,6
Philosophical Transactions of the Royal Society of London,4
Journal of Human Evolution,3
Geological Society of London Special Publication,2
The Journal of Geology,2
Journal of the Geological Society of London,2
International Journal of Remote Sensing,2
Physics and Chemistry of Minerals,2
Annual Review of Fluid Mechanics,2
Geological Society of America Special Paper,2
Journal of the Geological Society,1
Fire and Mud—Eruptions and Lahars of Mount Pinatubo,1
Geophysical Journal of the Royal Astronomical Society,1
Geological Society of London,1
The Geochemical Society,1
Journal of Colloid and Interface Science,1
Krakatau 1883: The Volcanic Eruption and its Effects,1
Geophysical Journal International,1
Pure and Applied Geophysics,1
Lava Flows and Domes,1
Disaster Resilience: An Integrated Approach,1
Volcano Hazard and Exposure in Track II Countries and Risk Mitigation Measures—GFDRR Volcano Risk Study,1
Assessment of Risk and Uncertainty for Natural Hazards,1
Encyclopedia of Complexity and Systems Science,1
Eos (Transactions, American Geophysical Union),1
Chemie der Erde–Geochemistry,1
Journal of Fluid Mechanics,1
American Journal of Physics,1
Computers & Geosciences,1
Living Under the Shadow: Cultural Impacts of Volcanic Eruptions,1
Paricutín: The Volcano Born in a Mexican Cornfield,1
Journal of Geology,1
Physics of the Earth and Planetary Interiors,1
Encyclopedia of Volcanoes,1
Progress in Physical Geography,1
Archives of Environmental Health: An International Journal,1
Timescales of Magmatic Processes,1
Reviews of Geophysics,1
Applications of Percolation Theory,1