Linking text and visual content

By Grant Jacobs 30/10/2009

There are several projects with a general concept of tight linking between the text and visual elements in a document for presenting scientific information.

It can often be easier to illustrate a concept with an image. Technically at least, it’s a little trickier, for moving or three-dimensional objects.

Easily the most successful is video, witness the YouTube and it’s ilk. One problem with video is that really it wants simplified verbal presentation: if it gets to wordy, long-winded or technical, it loses the audience. Viewers would probably say they’d be better reading a book. The other is that you can’t interact with the objects in image in the way you might explore a 3-D object.

iSee model

For a larger copy of this figure, visit the PLoS One site.

Enter a hybrid solution, text with links to trigger visual elements which let you zoom in, out and rotate 3-D images, or have them played as an animation. Previous solutions have really caught on beyond a niche application. Maybe it’s time has come?

The PLoS One article I reviewed was a mixed experience for me. The overall aims are fine, but not for the first time a paper my field has elements of an extended advertisement. The lead author works for the company developing the product. To be fair, it is an “overview” paper, which are short overviews of a particular application, but nonetheless I was a little uncomfortable with the lack of discussion of alternative approaches.

Their aims can be summarised in the figure above, they propose that the material be presented in three forms (see figure above): standalone with a special browser, operating via a web browser plug-in and working within Microsoft Office with the key point being the active linking of text and graphics.

Another example of this concept is Proteopedia, which explains molecular data using links altering the contents of a 3-D image of the molecule, that can be moved and examined by users:


Click to try it out. It may take some time to load as a Java applet is downloaded. Drag the mouse to rotate models; control-click to bring up a menu of options; hold down the alt key and move mouse up or down to zoom in and out, respectively. The green text alters what is displayed in the images.

Attempting to portray 3-D objects such as proteins and animations in scientific journals via 3-D viewers is not new. I recall David & Jane Richardson’s Kinemage from the early 1990s that was used by the Protein Science journal. While tailored to working with proteins it could–and was–used for other data.

MAGE, the program that displayed Kinemages was one of the first molecular visualisation programs for personal computers, along with RasMol. (This was the days of 33Mhz or 66MHz Intel 486 CPUs and Windows 3.1, or Macintosh LCIIs.) My recollection is that both these programs could be driven by scripts to animate a visualisation. Proteopedia uses a more up-to-date version of the MAGE software as part of its tools. More recently a revised version this work, KiNG is able to work within web pages.

Another approach is to include 3-D images into PDF files, which has been discussed elsewhere, using Adobe Acrobat 3D and, in this case, molecular graphics output. (Note PDF examples on this page.)

Although some might argue it’s forgivable for an “overview” article, I would like to have seen Raush et al cite this earlier work and the alternatives. Leaving out Kinemage seems particularly regrettable as it is in effect a (if not the) pioneer of what Raush et al aim to acheive. It reminds me of my recent article Forgetting older science and I worry that this may be another example of this.

(Disclosure: A few weeks ago I wrote about linking text and different types of visual information as part of a science communication proposal.)


1. A New Method for Publishing Three-Dimensional Content

Raush et al., PLoS ONE 4(10): e7394. (2009) doi:10.1371/journal.pone.0007394 (Open Access)

2. Proteopedia – a scientific ‘wiki’ bridging the rift between three-dimensional structure and function of biomacromolecules
Hodis et al., Genome Biology 9:R121 (2008) doi:10.1186/gb-2008-9-8-r121 (Open Access)
Visit the Proteopedia wiki website to see this in action

3. The kinemage: a tool for scientific communication.

Richardson DC, Richardson JS. Protein Sci. Jan;1(1):3-9. (1992) (Open Access)

4. KING (Kinemage, Next Generation): A versatile interactive molecular and scientific visualization program.
Chen VB, Davis IW, Richardson DC.
Protein Sci. Sep 18;18(11):2403-2409. (2009) [Epub ahead of print]
DOI:10.1002/pro.250 (Registration required for full article; abstract free.)
KiNG (the software) is freely available for download. The latest release is August 29th, 2009.

5. Creating interactive 3-D PDF documents with VMD and Acrobat3D
Theorhetical and Computational Biophysics Group,
University Illinois at Urbana-Champiagn (Web page created 24 Feb 2007)