Announcing RTF/ODF Scan for Zotero

May 12, 2013: Updated to reflect changes in the 1.0.9 update of the plugin. Changes are backwards compatible, i.e. everything that worked before still works now. See in particular the section on hidden preferences below

Support for Scrivener and google docs is one of the most frequently requested Zotero features. It is with great excitement that Frank Bennett and I announce today RTF/ODF Scan for Zotero, a Zotero plugin that extends Zotero support to any word processor capable of saving/exporting ODF (Open Document Format — .odt). You can find a brief, but comprehensive set of instructions on the project webpage. Here some additional comment and pictures.

The Short version

The plugin will install a “Scannable Cite” translator for Zotero. With that translator you can create citation markers to be inserted in your document (in Scrivener, google docs, etc.). Save/Export that document as an ODF file and convert the markers to Zotero citations using the “RTF/ODF Scan” feature from Zotero’s gear menu. Open the converted document in LibreOffice and set a citation style using Zotero’s LibreOffice plugin. Done!

In a Nutshell

What you need:

  • A word processor that should ideally be able to save/export as ODF (.odt)
  • Zotero for Firefox and/or Standalone on all computers you want to use
  • Install the RTF/ODF Scan for Zotero plugin from the project webpage by clicking on “Download Add-on”
  • You will need LibreOffice and the Zotero LibreOffice plugin installed on at least one computer. You will only need this for the final editing step.

Creating Citation Markers

default-output

Select the Scannable Cite translator as “Default Output”

After installing the plugin (no re-start required), set the Default output format for quick copy in Zotero’s Export preferences to “Scannable Cite.”  (The export format is installed by the add-on). You can now either drag&drop to insert citation markers into a document  or use  the ctrl+alt+c (cmd+shift+c on Mac) to copy them to the clipoboard and then paste them to your document.

A citation marker will look like this:

{ | Smith, (2012) | | |zu:2433:WQVBH98K}

It has five sections for — in that order — prefix, a summary of the cited item, a locator (e.g. a page number), a suffix, and a unique identifier. E.g. the marker

{cf. | Smith, (2012) |ch. 3 |, for the main argument |zu:2433:WQVBH98K}

drag-and-drop

Drag & drop citation markers into google doc

Would be converted into  (Smith, 2012, Chapter 3, for the main argument) when APA style is selected. You can use simple mark-up to use italics and bold print in suffix and affix: a single asterisk for *italics* two  asterisks for **bold**. Citation markers directly adjacent to each other will be turned into a single citation, as in (Smith 1776, 2012). Dragging/copying multiple items from Zotero at once will automatically create such adjacent markers. Locators require a specific set of abbreviations (e.g. p. or pp. for pages). For a complete list see the add-on’s webpage. A sample google docs document with citation markers is here.

Saving and Converting Documents

export-odt-scrivener

ODF export in Scrivener

export-odf-gs

ODF Export in Google Docs

Once you’re done writing, export your document as ODF. Many word processors have that option, including google docs and Scrivener. If your word processor does not export ODF, you can save in a different format (RTF, .doc), open the file with LibreOffice and save as ODF.

select-odf-scanThen go to Zotero and click on the RTF/ODF Scan option in the gears/action menu. Select “ODF (to citations)” as File format, the .odt file you saved as “Input File” and a location for the (converted) “Output File.” Click “Next” to perform the conversion.

odf-scan-window

Selecting a Citation Style

lo-basic-docOpen the converted document — by default it will have the same filename as the input file with (citations) appended — in LibreOffice. The citations will show up without any particular formatting and normally be shaded in dark grey. Click “Set Document Preferences” in Zotero’s LibreOffice plugin and select the citation style you want to use (you can change it later).  All citations will now appear formatted correctly. You can insert a bibliography by clicking “Insert Bibliography” from the same toolbar. The document is now indistinguishable from a document written entirely with the LibreOffice plugin, so you can opt to edit or continue writing it in LIbreOffice and citations will update according to changes in your Zotero database as well as the citation style.

RTF/ODF Scan as a Collaboration Tool

RTF/ODF Scan will work with group libraries, so it is ideally suited for collaborative writing. The only caveat is that, unlike Zotero 3.0+, it will not “store references” in a document, so all people collaborating on a paper need to cite items from a shared library, typically a group.

Reverse Conversion

RTF/ODF Scan can also convert LibreOffice citations in the Reference Mark format back into citation markers. For that purpose choose “ODF (markers)” as the file input format. This may come in handy if you have authored a document in LIbreOffice so far, but want to switch to google docs or Scrivener, you can convert your citations to markers. Another use is the ability to merge adjacent citations as requested by several Zotero users (this is Endnote’s default behavior, so many users are accustomed to it). RTF/ODF scan will do that simply by converting the same document back and forth.

Hidden Preferences

You can modify markers with two hidden preferences. See here for general instructions on accessing hidden preferences. Both of these preferences can be toggled back and forth using right-click –> Toggle in the about:config settings of Firefox/Zotero.

extensions.zotero.translators.ODFScan.includeTitle
will include the title of the item in the marker (default: false, i.e. only author, (date))

extensions.zotero.translators.ODFScan.useZoteroSelect
will use zotero://select links instead of simple item ids. These have the disadvantage of being longer, but can be used to access Zotero items directly (e.g. by pasting them in Firefox’s URL bar). This preference affects both the translator and the markers produced by reverse conversion. (default: false, i.e. citation markers as displayed above). Because of the way library IDs are constructed in Zotero, the group IDs used in the default markers and those in zotero://select differ. Trying to adjust them manually will break ODF scan

Getting Help

Please post questions and problems to the plugin’s thread on the Zotero forums.

{See | Smith, (2012) |p. 45 | for an example |zu:2433:WQVBH98K}

46 thoughts on “Announcing RTF/ODF Scan for Zotero

  1. Hans

    That’s interesting! Particularly, because Thomson Reuters holds a patent in the US for exactly this function: http://www.google.com/patents/US8082241 – at least that’s what I think the patent is about.

    Do you agree? If so, what do you think: Will Thomson enforce the patent – or do they fear a second “Zotero Desaster”?

    Reply
    1. adam3smith Post author

      Interesting. We weren’t aware of this, but no, what we’re doing isn’t covered by the patent. We’re using unique identifiers – not unformatted citations – to link a citation marker to the Zotero database. That’s what BibTeX has been doing since 1985 so I’m pretty sure it can’t be patented.

      The T&R patent is closer to Zotero’s RTF scan which has been around since the beginning and they’ve never complained about that. It still seems slightly different as the patent involves real time (“background”) scanning of the document rather than a separate scan procedure. (In general I’m not convinced the patent would be enforceable – something along those lines seems common in reference software predating 2001 – but IANAL).

      Reply
      1. Hans

        The patent doesn’t define what an unformatted citation is and therefore it might also be an unique identifier. And that BibTeX has been doing it since 1985 doesn’t mean that the USTPO noticed that.
        They filed the patent in June 2002, publication date was Decemeber 2011, so you are right, they probably would have had enough time to complain. And probably they they don’t think it is enforceable, too.
        BTW: Real time scanning is only an additional claim of the patent.

      2. adam3smith Post author

        right, but getting a patent approved by USPTO doesn’t mean something is legally patentable. Anyway – the answer to your question is that no, I’m not worried. I don’t actually think a reasonable reading of the patent describes what we’re doing and even if it did I’m pretty sure that Endnote would go after Zotero for RTF-Scan, Papers/Springer for their word processor implementation (also based on scanning a citekey), Biblioscape etc. etc. first.
        How did you handle that at Citavi? Did you get a T&R license?

  2. Hans

    Well, we’ve decided to offer Citavi 4 without a RTF/ODT scan and only with an Add-In for Word in the US. We also think that the patent is not enforceable in the end (see BibTeX), but we are not sure whether Thomson Reuters knows that, too (they even state the patent in the new start screen). So, we try to concentrate on developing Citavi and not on patent issues – at least as far as possible as these comments show.

    Reply
    1. adam3smith Post author

      makes sense. That’s why software patents need to die.
      For Frank and me (who are entirely independent of Zotero) the worst that can happen is likely a cease and desist – and as I say, I doubt it. If Zotero itself were to get involved, they have the advantage of having GMUs legal office for representation, so a lawsuit that’s likely without merit is much less of a threat to them than to a small company like yours (or individuals like us).

      Reply
      1. Tim

        Just wanted to point out that if they tried to file and BibTex would invalidate their patent, there are typically in-USPTO reviews that can be instituted that may be used to stay prosecution and invalidate a patent if it should not have been issued due to unnoticed prior art.

      2. adam3smith Post author

        thanks; as I say, we’re not really worried about this. Citavi is in a somewhat different position, since they’re a small company that can’t afford a lawsuit. In the very unlikely case that TR would bring suit, Frank&I would likely just fold. It’s not worth it. But they’d not come after us anyway, we’re way too small. If anything, they’d go after Zotero, which if it comes to that has access to GMU lawyers, which have given TR a bloody nose once already.

  3. Pingback: Announcing RTF/ODF Scan for Zotero | Everything Scrivener

  4. Pingback: Literaturverwaltung kompakt 4/2013 | Literaturverwaltung

  5. smawhort

    Many many thanks for developing this plugin! I will start writing my dissertation in the fall, and I was afraid that the Scrivener/Zotero link would be too clunky for such a complex project (it has worked just fine for my shorter papers). I have been scanning the forums for several months hoping for something like this. I am thrilled that you’ve developed such an elegant and user-friendly solution.

    Reply
  6. David Smith

    Another vote of thanks from a scrivener user, and a cheeky question… Is there any reason, since .docx files are now also XML and open, that this plugin couldn’t be fairly easily adapted to work with them too?

    Reply
    1. adam3smith Post author

      Most likely that would be possible – though it depends on how Fields are included in docx files – but neither of us uses or even owns Word (nor, typically, an operating system running Word), so we’re not very motivated to undertake this (I assume that’s the case for Frank as well, haven’t actually talked to him about this). If someone is, though, we’re happy to patch it in. The main work would be to figure out the regex for scanning. The other things that need to be adjusted are the structure of the packaged XML and the Fields to replace the markers with, but those should be quick. Since you can rely on all the basic infrastructure in place for the scan, it’d be a lot less work indeed. We’re happy to help out with specific questions.

      Reply
      1. smawhort

        I don’t mind the simple task of going back and forth between Libre office and Word now that I can easily go back and forth between Scrivener and Zotero! However, I would be grateful for a Word extension of the plugin as I often need to use Word when I collaborate with co-authors. (I don’t have the technical skills to contribute to a solution myself.)

      2. David Smith

        I’ll have a look through the code (which looks pretty well documented!) and see what I can manage. It seems like it should be something which even somebody with basic knowledge of the command line should be able to do, if they’re willing to spend the time! I’m writing papers this week, I’ll keep in touch.

  7. adam3smith Post author

    Just to be clear – there are various ways of making this work in Word already: you can just keep the citation markers and only do the final conversion in LO, or you can convert to reference marks in LibreOffice and then convert to Bookmarks (under Set Document Prefs in the Zotero LO plugin) which are compatible with Word and LibreOffice.
    In other words, you can make this work for you in most settings with a couple of extra steps. I’m very much in favor of avoiding extra steps, so I’d be happy for docx support to be added and be happy to advise, but as I said it would be an inordinate amount of extra work for Frank or me because we’re not set-up to work or even test with Word so it would have to be done by someone else.

    Reply
    1. smawhort

      Thank you for letting me know about these options! They should work perfectly well for me, and eliminating these few extra steps does not seem necessary, especially given the time investment.

      Reply
  8. uma

    Hi I am using scrivener for writing out my dissertation..but I would need to convert all of my work into .docx/.doc for my DAC’s review. So after doing the odf scan can I still do that and retain all the references? I use zotero and scrivener on Windows..please help…thanks!

    Reply
    1. adam3smith Post author

      Generally yes: Once you’ve run ODF Scan, you would need to convert change the Zotero citations from Reference Marks to Bookmarks (in the Document Preferences of the LibreOffice plugin) and you can then convert the document to docx and keep the references live. Two warnings about that: 1. Bookmarks are more fragile than Reference Marks, which may be a particular problem if your DAC will edit in the document. 2. In my experience, converting back and forth between docx and odt isn’t flawless, so you may run into formatting problems on the way. My recommendation would be to test this out with a smaller chunk before committing to it for a whole dissertation.

      Reply
  9. Faye

    Hello. Please can you help? I have installed LibreOffice and the Zotero LibreOffice Plugin Scannable Cite does not appear under the Defalut Output Format options. Can you think of any reasons for this?
    Thank you!

    Reply
  10. Pingback: Zotero et Scrivener | Zotero francophone

  11. Liz Covart

    Thank you for this great post. I finally understand how to use Zotero with Scrievener.

    Reply
  12. Pingback: Wednesday Link Roundup #49: History Funds, Fun, & Writing Tips - Elizabeth M. Covart | Elizabeth M. Covart

  13. Pingback: Zotero + Google Docs – une combinaison possible ! | franziska.fr

  14. Pingback: Utiliser Zotero avec Google Docs, Scrivener, Pages… | franziska.fr

  15. Pingback: Zotero + Google Docs – une combinaison possible ! | La boîte à outils des historiens

  16. Pingback: Utiliser Zotero avec Google Docs, Scrivener, Pages… | La boîte à outils des historiens

  17. Pingback: Zotero + Google Docs - une combinaison possible ! | La boîte à outils des historiens

  18. Pingback: Scrivener + Zotero | Outside the Fishbowl

  19. Pingback: Utiliser Zotero avec Google Docs, Scrivener, Pages... - La boîte à outils des historiens

  20. Pingback: Referencing and working with bibliographies in Markdown: the recommended method - History to the Public

  21. Pingback: Scrivener, un outil d'écriture créative - La boîte à outils des historiens

  22. Felix Mueller-Sarnowski

    Thank you for this great tool! Might it be possible to make the plugin compatible with the pandoc citation format (http://pandoc.org/demo/example19/Extension-citations.html)? It’s i.e. Blah blah [see @doe99, pp. 33-35; also @smith04, ch. 1]. Thanks to the better-biblatex-plugin it is already possible to use zotero citations with pandoc. But it would be just prefect, if one could create a “real” zotero document from the markdown source (instead of plain text).

    Reply
    1. adam3smith Post author

      not really, unfortunately. The reason we didn’t go with Pandoc markdown (which already existed and was used when we came up with this) is mostly that we rely on the unique Zotero identifier to generate the LibreOffice Reference Mark. The way this works, the actual scan doesn’t even require us to query the Zotero database.
      Working with Pandoc markdown would require us to look up the unique ID from the identifier as part of the scan — surely possible, but a significant change in the code that I don’t think is in the card for either Frank or me any time soon. Just as for docx scan, though, we’d take patches, of course.

      Reply
  23. Smik Miksen

    Great – would it be possible to make this work for Microsoft Word as well? (Since Zotero’s ‘formatted’ citation management is unbearably slow in large documents)

    Reply
  24. Jurgen Strydom

    Would it be possible to call this from the command line? I’m writing all my documentation in markdown, and would like to build a python script that automatically builds my large documents.

    Reply
    1. adam3smith Post author

      The Better BibTex CWYW picker allows creating the syntax from the Zotero picker, permitting a keyboard-only workflow. That said, if I were to write in MD, I’d just go with pandoc markdown citations.

      Reply
  25. Vincent

    Hi, I’m using the Zotero-ODF Scan plugin to write my thesis. Do you plan to release a Zotero 5 compatible version of this wonderful plugin?

    Reply
  26. Pingback: The PhD starter kit – The NeuroBlog

  27. Chris Fetterly

    Hey guys, awesome addon. Any experience converting betterbibtek citation keys within a google doc to a scannable key? Ie. I have [@author1999] within my google doc and would like to convert it to a scannable key. Any way to do that in bulk instead of doing find/replace?
    Cheers
    Chris

    Reply
    1. adam3smith Post author

      Sorry, never replied to this, but the answer is no. The citation markers have to be generated from Zotero directly to have the unique item ID included. You can’t generate them from citation keys, or at least not reasonably easily.

      Reply
  28. Pingback: Working with references in Markdown: a better way | Simon D. Coll

  29. Pingback: Scrivener, un outil d’écriture créative – La boîte à outils des historien·ne·s – Macchiatto Café

Leave a reply to adam3smith Cancel reply