Invisible & Exciting: What’s New in Zotero 4.0.19

Zotero just released version 4.0.18 and 4.0.19, with some major, mainly under the hood improvements. These changes aren’t obvious at first sight, but are a major improvement. There are three major improvements, as well as a couple of minor ones.

Better Metadata from PDFs

Especially for new users of Zotero, the Retrieve Metadata function can be crucial: It allows you to simply drag&drop a couple of PDFs into Zotero and then with a simple right-click add find their metadata online. Up to now, this had two major problems.

      1. The search relies principally on google scholar and users would frequently get locked out as suspected robots after retrieving data for more than 20–25 items, making the feature almost unusable to import large collections of PDFs
      2. The data coming from google scholar is not the best. While it has gotten dramatically better over time, it still doesn’t include DOIs, abstracts, and has gaps and errors at times.

The new Zotero version mostly solves the first of these. Lock-outs are now rarer, and where they happen, users get prompted with the captcha google scholar uses to identify human users. So retrieve all you want! However, the second issue is still relevant, so once you have Zotero up and running, you should use the function sparingly for adding new items. In almost all cases the URL-bar icon is your tool of choice.

Faster Indexing

Zotero doesn’t just let you search through your items’ metadata, but also through the full text of your attachments as long as they are in plain text, html, or PDF format. For that purpose, Zotero indexes the full text of those attachments. In the past this was slow and adding several 100 page PDFs could freeze Zotero/Firefox for minutes. The speed of this has now improved dramatically: “War an Peace in 1.5secs” as Dan Stillman puts it.

It has always been possible to add links in Zotero notes, however, until now you couldn’t actually follow them. This has now been fixed. This is, of course, useful by itself, but let me highlight two non-obvious use cases.

  1. As various users point out in this thread many other organizational tools like Evernote or DevonThink have their own linking protocol (just like Zotero’s zotero://) that allows you to go straight to an item/note. Now you can get to an item in DevonThink directly from a Zotero note.
  2. The indispensable Zotfile has long extracted PDF annotations into notes. Since version 3.1, Zotfile also includes a link to the page containing the annotation with every annotation. Now, with the links functioning, you can click on an annotation and have the PDF open on the right page.


Some Smaller Improvements

    • Zotero recommends storing titles in sentence case. Right-clicking on an item’s title, allows you to convert that title to sentence case. The new Zotero version contains various improvements of this feature, including better handling of beginning punctuation (like quotation marks or the Spanish ¡ and ¿) as well as capitalization after punctuation within the title. Now the first letter after colon, question mark, and exclamation mark are capitalized, in line with the APA’s requirements for sentence case in titles. Unfortunately, other styles, in particular the Citing Medicine guide do not capitalize after colons, so that we will roll back that part in the next release. This thread explains the logic behind that in more detail.
    • The reference test pane — Zotero’s built in tool that facilitates modifying citation styles—now has a save button. It is no longer necessary to take the detour via a text editor to save CSL styles.
    • Both PMID and PMCID are now properly parsed from the extra field (each should start on a new line) so that citation styles using the PMID and PMCID variables, like the NIH/NLM grant style now work correctly.

Free Software at work

Another remarkable aspect of this release is that it is largely community driven: All major improvements are based on patches submitted by different volunteers: Emiliano Heyns (who I’ve mentioned several times before) submitted the patch for faster indexing; Aurimas Vinckevicius, who together with me does a lot of work on Zotero translators and support (among many other contributions) contributed the retrieve metadata improvements; Joscha, the developer of Zotfile, fixed the links in notes; the improvements to sentence case conversion, the reference test pane, and PMCID parsing came from me.

Zotero is an increasingly mature free software project with a broad base of contributors. That doesn’t mean, of course, that Zotero’s developers are idle: For one, each suggested patch—”pull request” in the language of git— gets reviewed very thoroughly, often with a lot of back and forth. Beyond that, Dan adds a constant stream of small fixes and improvements to both Zotero and the sync server, and, perhaps most importantly, has been working on the complete re-write of the sync infrastructure that will be the cornerstone of Zotero 4.1


12 thoughts on “Invisible & Exciting: What’s New in Zotero 4.0.19

  1. Pingback: Twitter Open Access Report – 25 Mar 2014 |

  2. joetoe

    You write: “Now you can get to an item in DevonThink directly from a Zotero note.” Is there an easy way to index the Zotero (Zotfile) notes (with pdf annotations) in DevonThink? Currently I am indexing Zotero’s pdf storage folder in DT but it doesn’t seem to be catching the Zotfile annotation notes.

    1. adam3smith Post author

      not 100% sure, but I don’t think so. Zotero notes are stored right into the Zotero database, so I don’t see how Devonthink – short of directly accessing the sqlite – would be able to index them.

  3. joetoe

    I just discovered that DT found the annotation but gives it the .skim suffix. I found it with a search for the pdf file name it seems to find text but it doesn’t display the note just a blank icon. I would be happy to upload the note somewhere if interested.

    1. joetoe

      Sorry no.. that was the annotation extraction that Skim produced not Zotfile. (However I can’t read the .skim file in DT either.)

  4. joetoe

    One last follow-up… Do you know of a way to batch export Zotero notes (perhaps as reports) for eventual indexing by DT in txt or rtf format?

    1. adam3smith Post author

      reports could work nicely – how about using the report customizer (featured in an earlier post here) – and having it just show title and note?

  5. joetoe

    I missed the link to your earlier post. Is it this one: FYI I have never used reports (just read about them now) – is it possible to easily generate on report per ref/pdf and store it in the pdf folder of the Zotero’s storage folder for DT to index? This could be rewritten (over) as new annotations and reports are generated.

    1. adam3smith Post author

      yup, that’s the one – but no, one report per item isn’t easy to automate with reports. I don’t have a good idea to automate that which doesn’t involve scripting.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s