It has been over a year now since Beall’s list of predatory Open Access publishers closed down. In the meantime, copies of this list and a new database that black- or whitelists scholarly journals has emerged. Together with TU Delft, we tested Cabell’s initiative. Do we actually need such a database?

The case for Open Access in The Netherlands is clear: our government wants 60 per cent Open Access in 2018 and 100 per cent in 2020, funders such as NWO and ERC have requirements and universities have policies in place as well. In most fields, authors are spoiled with choices for publishing Open Access because of deals our universities and libraries made with publishers such as the American Chemical Society, Oxford University Press, Springer, Taylor & Francis and Wiley (check the journal browser for possibilities).

However, some authors and librarians fear they might inadvertently publish or pay for Open Access in pirate journals. These journals, usually pretending to be high profile and with a quick time to print, simply take the money and might even publish your child’s homework if you pay. It is not always clear what status a journal has, as new journals may enter the market or fake journals might try to look as much as their established sisters. This is where the former Beall's List came in handy: it gave an overview, controversial but helpful, of publishers and journals that lacked certain quality criteria according to librarian Jeffrey Beall. Since the list closed, several copies and alternatives filled in the gap.

In this review we - Jorden Esser, Rutger de Jong, Just de Leeuwe, Dirk-Jan Ligtenbelt and Kyle Siler - take a look at Cabell's Blacklist and Whitelist, which institutions can subscribe to. The focus in the review is not so much on the website interface, which is simple one-by-one browsing of titles (searchable by amongst others publisher and country for the Blacklist and by review type, acceptance rate and subject for the Whitelist), but on the criteria used for building the database and on the question if it can meet the needs of Dutch researchers.

Transparency

At the time of trial there were some 6.800 journals on the Blacklist and 11.000 on the Whitelist, which means coverage is far from complete. To put this into perspective: the Directory of Open Access Journals contains 11.000+ journals and in Leiden we offer over 40.000 journals to our users.

The 64 criteria that Cabell’s offers for checking the journals against, is unfortunately the only thing that is truly transparent in this database. It is unclear how journals are added or removed from the index and, as the database itself is subscription-based, so publishers may not know they have been listed and thus can’t appeal.

For a further look we numbered the criteria and discussed them with each other (see table and numbered criteria). The table shows that there are quite a lot of criteria where we question the methodology of setting the criterion. For example, how does Cabell’s know how many emails a publisher sends out? What is a controversial publication? Other criteria fall into the trap of field specificity. Is 4 weeks really too fast for publishing? It totally depends upon the type of article you publish. Is gender bias a problem? In psychology there are more women and in mathematics more men. It is a fact of life. For some criteria we think high quality journal (publishers) will also fail. How many editorial offices does a Springer have?

Luckily we can see on what grounds a publication has been blacklisted and the number of violations is displayed in large letters next to a journal entry. Cabell’s says it weighs the importance of these violations to make the selection for the Blacklist. Because of all of our foot notes on the criteria, the weighing is a process that should be clear, but isn’t.

Is it worth paying several thousands of dollars a year to have a set of these criteria, while you can check most of them yourselves? While looking at some fake journals it is hard to believe anyone falls for them. Even though journals are listed by themselves, not because they belong to a certain publisher, we could not find any journal from a major publisher on the Blacklist. Controversial publications from Europe, such as some Frontiers journals, could neither be found on the Blacklist nor on the Whitelist.

Comparison to The False Academy

Eriksson and Helgesson (Eriksson & Helgesson, 2016) made another popular list of criteria to help author’s discriminate between proper and predatory publishers: The False Academy. Of the 25 criteria on this list, 8 (almost a third) do not appear on Cabell’s list. For example is the publisher part of a publisher’s association and easy to check things such as advertisements on the publisher’s website. The most important criteria are to be found on both lists though.

Criteria on Cabell’s list missing on ‘The false academy’-list, are for a large part the ones harder to check or the less decisive criteria. Read our list with checkmarks of items appearing on both lists in this overview.

Our advice on the Blacklist

At the moment we do not think publishing at predatory publishers is a large problem in The Netherlands yet, partially due to our deals with large publishers. Unfortunately we were unable to check all of our publications against the database as an API for doing so is missing and each journal has to be looked up separately (which would be a pain with the thousands of publications our universities put out each year).

If, as an author, you are in doubt about publishing in a journal outside of our library deals, I would suggest simply checking the following criteria first, freely translated from the think-check-submit website:

• Do I – or my direct peers - know and read this journal or other journals published by this publisher?
• Is it easy to find and contact the publisher?
• Is the journal peer-reviewed and is the peer-review process easy to find?
• Are the articles indexed in important community indexes, such as PubMed or Web of Science? (Don’t check the journal website, but check in the index)
• Is the journal in the Directory of Open Access Journals?
• Are the article processing costs clear and only to be paid upon acceptance?
• Do you recognize the editorial board?

Whitelist

At first the Blacklist seemed to be the most important part of Beall’s List. However when we received access to the Whitelist as well, this seemed to give some good additional information on for example the time to publish, the peer review process and the acceptance rate, how well they are doing on the social impact. All helpful information for authors trying to find the right journal.

It would be great if we could add such information to for example the journal list tool, as chances are slim our authors will find the database otherwise as it is not within the same set as the blacklist journals. This is another lack of transparency: the blacklist criteria have not been applied. Personally I have hardly ever seen a journal that would pass all checkmarks; editorial boards are usually outdated for example, as you learn when looking for editors from your own university.

We did some quick checks for important publishers with which we do have an Open Access deal to get an idea of the coverage. Not all journals of these publishers are on the whitelist yet (but neither are they on the blacklist). Unfortunately the limited coverage and lack of ways to integrate with other systems mean this database is too expensive for us to justify.

Overall conclusion

Lists such as Beall's List and Cabell's Blacklist and Whitelist may be beneficial as they provide a lot of information on journals that might save the scholar time when trying to get published. However, criteria for blacklisting are controversial at least, and the process of weighing them is not transparent. The same lack of transparency can be found in the process of adding a journal or getting it delisted.

If Cabell's was fully open and transparent, preferrably also with an API to integrate it in other sources, I think it would really be worth investing in. But the lack of transparency is a real deal breaker to us, For now, I think it better to point towards our lists of 'big deal' journals and using a limited set of criteria, such as those from the 'False academy', if the journal is a new one and/or not on the list.

Participants

• Jorden Esser, TU Delft Library
• Rutger de Jong, Leiden University Libraries
• Just de Leeuwe, TU Delft Library
• Dirk-Jan Ligtenbelt, TU Delft Library
• Kyle Siler, UU

Image credits:

Illustration: Just de Leeuwe