The Journal Impact Factor (JIF; average number of citations to a paper in a journal in the first two years since it was published) is such a poor predictor of an individual paper’s citation count that quoting it is a sign of statistical illiteracy, yet cursory judgements about the quality of scientific papers are routinely based upon of the JIF of the journal in which they are published. At one end of the spectrum, this means that a paper in Nature or Science is automatically assumed to be so important that a single one on a CV can be a passport to job interviews, speaking invitations, jobs and promotions. At the other, papers in discipline-specific journals can be passed over, assumed to be unimportant, despite being more useful to the group of people that actually read them.
Volcanology journals are an example of the latter; it’s a small field and, with just a few volcanologists to cite each other’s work, the JIFs of the specialist journals are quite low. Nevertheless, they are clearly very important to the world of volcanology. I wanted to find out what the most important journals in volcanology are, and if there was any correlation with the JIF.
To do this, I looked at the reference list of a recent review article, How volcanoes work: A 25 year perspective, written by Kathy Cashman and Steve Sparks. Both authors have distinguished careers and extremely wide-ranging interests so can be trusted to give a reliable overview of the subject. In the paper, they “focus particularly on the physical processes that modulate magma accumulation in the upper crust, transport magma to the surface, and control eruptive activity“, which is actually a huge scope. Volcanologists working with data from the depths of the mantle and lower crust or from orbit on satellites may feel a bit neglected, but most of physical volcanology is represented.
The paper cites a whopping 364 references. I extracted the journal or book title from each, and with a few lines of Python code, got counts of which were cited most. The number of counts is a proxy for the importance of the journal in the field of volcanology.
The most important journals in volcanology
The most cited journals by Cashman and Sparks are the Journal of Volcanology and Geothermal Research and Bulletin of Volcanology. Both are respected journals, well-read by volcanologists, so it is no surprise that they are the most important journals in the field. JGR and EPSL are next. Between them, these top four journals represent just under half (175/364) of all papers cited in the review.
Nature and Geology feature in 5th and 6th, showing that some important volcanology papers are published there, but they certainly do not dominate in the way that their JIFs would suggest.
Comparison between JIF and citations in Cashman and Sparks (2013)
Comparing the number of citations in Cashman and Sparks (CCS) with the JIF (2 year citation data from SCImago; top 20 journals only) shows an overall lack of correlation. The data can be divided into three groups:
- Journals with JIF > 5: These ‘big hitters’ show a positive correlation between JIF and CCS, but, with the possible exception of Nature, they are not the most important journals to volcanologists.
- Journals with CCS > 10: The four journals shown to be the most influential in volcanology all have a modest JIF that does not reflect their significance within the subject. With Nature as an exception once more, the most important journals in volcanology actually show a slightly inverse correlation with JIF.
- Journals with JIF < 5 and CCS < 10: These show no correlation between JIF and CCS.
Conclusion
The data show that the most important journals in the field of volcanology, based on citations in a comprehensive review of the advances of the past 25 years, are Journal of Volcanology and Geothermal Research and Bulletin of Volcanology. These are discipline-specific journals with low JIFs. Consequently, the JIF of these journals is of especially little value in assessing the quality of the work in their articles or the importance of their contribution to the field.
EDIT 2014-03-12 21:30. Of course the CCS is just another journal-based metric, so it still can’t tell you anything about an individual article or scientist.
If you don’t believe that journal-based metrics, such as Journal Impact Factors, should be used as a surrogate measure of the quality of individual research articles, or to assess an individual scientist’s contributions, or in hiring, promotion or funding decisions, then you can join over 10,000 others in signing the San Francisco Declaration on Research Assessment.
Reference:
Cashman KV, Sparks RSJ (2013) How volcanoes work: A 25 year perspective. Geological Society of America Bulletin. doi: 10.1130/B30720.1
Results in Full:
Journal of Volcanology and Geothermal Research,57
Bulletin of Volcanology,47
Journal of Geophysical Research,40
Earth and Planetary Science Letters,31
Nature,18
Geology,17
Journal of Petrology,12
Geochemistry Geophysics Geosystems,11
Geological Society of America Bulletin,10
U.S. Geological Survey Professional Paper,10
Geophysical Research Letters,9
Science,8
Geological Society of London Memoir,7
Contributions to Mineralogy and Petrology,6
Reviews in Mineralogy and Geochemistry,6
Nature Geoscience,5
Philosophical Transactions of the Royal Society of London,4
Geophysical Monograph,4
Journal of Human Evolution,3
Earth-Science Reviews,2
Geological Society of London Special Publication,2
The Journal of Geology,2
Journal of the Geological Society of London,2
Mathematical,2
International Journal of Remote Sensing,2
Physics and Chemistry of Minerals,2
Annual Review of Fluid Mechanics,2
Geological Society of America Special Paper,2
Journal of the Geological Society,1
Fire and Mud—Eruptions and Lahars of Mount Pinatubo,1
Oceanography,1
Geological Magazine,1
Geophysical Journal of the Royal Astronomical Society,1
Tectonophysics,1
Geological Society of London,1
Religion,1
The Geochemical Society,1
Journal of Colloid and Interface Science,1
Marine Geology,1
Krakatau 1883: The Volcanic Eruption and its Effects,1
Geophysical Journal International,1
GeoJournal,1
Pure and Applied Geophysics,1
Lava Flows and Domes,1
Disaster Resilience: An Integrated Approach,1
Volcano Hazard and Exposure in Track II Countries and Risk Mitigation Measures—GFDRR Volcano Risk Study,1
Assessment of Risk and Uncertainty for Natural Hazards,1
Encyclopedia of Complexity and Systems Science,1
Precambrian Research,1
Eos (Transactions, American Geophysical Union),1
Chemie der Erde–Geochemistry,1
IEEE Transactions,1
Journal of Fluid Mechanics,1
American Journal of Physics,1
Chemical Geology,1
Computers & Geosciences,1
Living Under the Shadow: Cultural Impacts of Volcanic Eruptions,1
Paricutín: The Volcano Born in a Mexican Cornfield,1
Journal of Geology,1
Physics of the Earth and Planetary Interiors,1
Encyclopedia of Volcanoes,1
Volcanic Plumes,1
Progress in Physical Geography,1
Washington,1
Archives of Environmental Health: An International Journal,1
Timescales of Magmatic Processes,1
Scientific Drilling,1
Reviews of Geophysics,1
Applications of Percolation Theory,1
Nice analysis and point well made. I wonder how the distribution has changed over time however? Could you sub-sample your data and look at what journals were important when? Say is the past 5 years, 10 years and 25 years? It would be interesting to see the distribution.
Hi Janine, I think that the distribution will certainly change with time, if only because the journals have changed. For example, Nature Geoscience didn’t exist 5 (or 10?) years ago, so it has picked up lots of citations in its short life. I think that Bull Volc has been running for longer than JVGR, too. For this analysis I quickly stripped out all the information apart from the journal/book titles, so I don’t have the data readily available to analyse things by year. But the information is all in the paper if you wanted to have a go.
Hi John,
A great post, as ever!
Something that you don’t mention is the number of volcano-themed articles published in each journal. Commonly a month goes by without a single volcanology paper being published in Nature. Meanwhile the JVGR and BV editors require a large number of volcano-related articles to be published in each issue, otherwise the journals would cease to exist. Therefore, if each paper had an equal probability of being referenced by Cashman and Sparks, regardless of the journal it appeared in, we would naturally expect the journals that publish most articles to crop up most often in their citation list.
It would be interesting to see this ratio:
The number of times articles in a specific journal are referenced by C&S / The number of volcanology articles published in that journal.
This ratio would be akin to a volcanic influence rating (VIR), as would indicate the relative likelihood of articles published in specific journals being considered to be influential by the volcanology community. In all likelihood the VIR will correlate with the impact factor of the journals in question, although the bio/health sciences inflate the impact factors of multidisciplinary journals.
So, if you are a volcanologist with some interesting and publishable new findings do you choose to publish them in JVGR or BV, because these have the largest number of articles cited by C&S, or aim to publish in the journal with the highest VIR? Of course, I fully support the ethos of the San Francisco Declaration on Research Assessment and think that the best paper I have written to date is in BV, not in a glossy weekly!
Hi Hugh,
Good points. The people to whom the C&S index, like the JIF, is most useful are librarians. If you have limited money for your volcanology library, then you buy JVGR, Bull Volc and JGR first. And as scientific publishing moves towards open access, it won’t even matter much to them.
There are many problems with this simple analysis. The changes with time mentioned above, the fact that JIF covers 2 years but C&S covers 25, and the only-a-few-volcano-articles-in-multidisciplinary-journals issue that you mention and that was also flagged up on Twitter. Your VIR would address this, if the data were available. I think that JGR would receive a boost from this, too. To do a comparison with impact factor, you would also need to calculate the impact factors of just the volcano articles in the multidisciplinary journals.
In a recent editorial in American Mineralogist, Keith Putirka makes a case for submitting your interesting and publishable new findings to specialist society journals. He argues that specialist journals give more space to publish detailed work that is more useful in the long term, that they are more likely to pick your paper as a highlight, and that they target their promotions specifically at the people who are most likely to read and cite the papers. As evidence, he says that American Mineralogist’s top cited paper has 3,019 citations, compared to 1,659 for EPSL’s. Meanwhile there are no papers in the mineralogy, petrology or geochemistry section of Science with more than 500 citations. These are obviously the very top papers, and have had 30 years to rack the citations up, but I don’t know what happens to majority, or more recently.
I think that the big question is how would you make it clear to someone scanning your CV that your Bull Volc paper was the best one?
Later,
John
Pingback: Four years of volcan01010: Highlights of 2014 | Volcan01010
Read this article a few times, and about one year later, I want say some thing. I do not know the ratio between amount of each single discipline published their paper in STAR Journal like Nature or Science, and the total paper amount in that discipline. If this ratio is positive correlation with so-called IF, IF is fair for every discipline! However, I do not expect this happen.
Another one is, the frequencies of a given journal appears in every single paper (at least in general, or random sampling of some high level papers) in this discipline. Maybe more than 70% Volcanology papers citing BV or JVGR. This is just an extent of you method.
Anyway, BV and JVGR are definite most important journals in Volcanology.
Supplementary
In first ratio, It must contains the worker in that discipline.
…..I mean, The amount of workers….
Hi Zhengquan,
This point is similar to Hugh Tuffen’s point above. I think that if you divide by the number of volcanology papers in Science or Nature then their scores will increase dramatically because they publish so few volcanology articles.
I once asked an Editor at Science why this was. He said that they liked articles that looked at the big picture and made assertions about how the whole world worked. He said that volcanology articles, which often focus on a single volcano or eruption, tend to read more like case studies. I think that is true, but I obviously also think that case studies are very useful.
Even if normalised scores were higher, I think that it is hard to argue that journals that actively choose to publish very few volcano articles are important to the field of volcanology.
John
Hi, Im currently working on a project that consists on designing a bag for a volcanologists that´s going on a filed study to Iceland…. im wondering what would be the essential things that a volcanologist has to take with him/her, including instruments and clothing and what would the ideal bag for this kind of professional who´s going to Iceland..
Sounds like an interesting list. Two of my other blog posts might be useful for you as they describe the clothing and field equipment that I use in Iceland:
Volcano suit / What to wear in Iceland
Fieldwork Guide for Robots (and Humans)
Hope that helps,
John