Skip to end of metadata
Go to start of metadata

Image annotation tools and ontologies

Shared Canvas a shared canvas (for collaborative image annotation) based on OAC ( ontology.
Collaboration of Stanford and LANL, with funding by Mellon

  • sample implementation
    Under heavy development, still rough around the edges and slow: JS processing in browser, does not seem to use Deep Zoom server while the images are 1-2 Mb.
  • sample files, can be used for understanding and as engineering examples.
    In adition to OAC, uses ore:Aggregation (which is also heavily used in Europeana EDM).
    The files for all demos are available (/impl/demoN/res)
  • SharedCanvas- Data Model (Mar 2011)

Stanford DMStech

Data Model and Architecture

Rough description of Data Model and Architecture.

  • data model uses OAC, OAI-ORE for aggregation (eg all pages of a manuscript), a Manifest to describe all related pages
  • architecture uses load-balanced Djatoka image servers

Digital Mappaemundi

Digital Mappaemundi is about collaboration on digitized Manuscripts. I think the most significant applications in image annotation have been developed for Manuscripts.

Please watch this video: it's 14 min, but you can watch only the first half.

  • I think it's very interesting and instructive, both in terms of use cases and UI. 
  • Nothing to do with semtech, but it's built to solve real researcher needs.

Link between image region and concept

Show all image regions denoting a certain concept:

Application to genealogy


Text-Image Linking Environment
The site sounds good, but the demo doesn’t seem to work

Seadragon Ajax is open source
Seadragon Ajax (sometimes called Seajax) is in fact open source under a very liberal New BSD license !!!
As part of the Ajax Control Toolkit (ACT)

Of the many deep zoom interfaces to have emerged over the last few years for viewing high-resolution images on the web, Microsoft’s Seadragon is both one of the most beautiful and easiest to use. Unlike many of its competitors, the software seamlessly transitions between zoom levels without noticeably replacing the tiles, an effect that is arguably both more aesthetically pleasing and more functional than a page refresh as the user need not relocate their position each time a new zoom level loads. Additionally, Microsoft has provided a *well-developed code library for annotating images and binding regions* of the image to particular actions when clicked.
Cutting the large source image into a Deep Zoom Image (or DZI) tile set... By far the easiest way to do this is through Microsoft’s Deep Zoom Composer, a Windows-only, free, but proprietary piece of desktop software. However, there are also a set of third party tools, some open source, which can also generate the tiles on Macintosh and Linux machines.

Djatoka image server
link the open source version of Microsoft’s Seadragon to Los Alamos National Laboratory’s Jpeg2000 server, Djatoka

(I don't know how Djatoka compares to IIP Server)

YUMA Universal Media Annotator

Multimedia Annotation Service (YUMA Framework) (EuropeanaConnect D5.6.1).pdf
Also called EuropeanaConnect ThoughtLab Media Annotation Prototype, YUMA is a powerful media annotator.
It was reviewed in our Offer, sec EuropeanaConnect Map Annotation.
I first looked at it last summer, and was extremely impressed. We've looked at a number of tools, and your tool seems to come on top on several counts:

  • not only image, but also map, audio and video annotation
  • deep zoom support
  • OAC export
  • annotation server (maybe we'll store them ourselves, but it is a nice option)

YUMA videos

The demos at are broken:

YUMA full features

Features that are yet to come in yuma.min.js (some of them implemented in YUMA full):

  • OpenID support
  • Data interoperability and Linked Data publishing based on standard formats (OAC RDF)
  • Tagging
  • Semantic Tagging based on controlled vocabularies (or we can customize this to work against a thesaurus)
  • Semantic tagging using DBpedia/wikipedia (hope to customize this to work against LOD and internal RDF data)
  • Sharing and embedding of annotations
  • History/version track for annotations
  • Support for HTML5 video
  • Customizable search and data export
  • Annotate images, maps, audio, video
  • Shape drawing tools on images, maps and video
  • Special support for high-resolution map images
    • Tile-based rendering for faster delivery
    • Geo-referencing
    • Semantic tag suggestions based on geographic location
    • Place search functionality
    • Overlay of present-day country borders, coast outlines, etc

YUMA development status

We'd like to use the full version, and so I have some questions:

  • is the full version open source?
  • what is its status: is it in a serviceable state?
  • is yuma.min.js a complete rewrite of the full version?
  • what's the language/implementation platform of the full version?
  • if the technical plan is to rewrite the full version into yuma.js, what is the timeline for this?
  • if open source, we could contribute some development to speed up the timeline: are you open to cooperation?
  • how hard it would be for us to ramp up on yuma development, and how much effort it would take?

Simon Rainer answers

  • the "full version" is Open Source as well, and the code is available on Europeana's SVN repository at ( BUT - to be frank - the code is not really in a serviceable state; and it hasn't been maintained at all since the end of the EuropeanaConnect project. So I'd really advise against using it. That's also the reason we took the old demos offline after the end of the project.
  • This, in fact, was the reason for why we started yuma.min.js: the original prototype had only been developed to the stage of a proof-of-concept. While many of the features were demo-able, they were yet incomplete and not suitable for full production use (which would have been addressed in a follow-up project which, unfortunately, never happend); usability was less then optimal (based on decisions that had been taken years ago...); the architecture (and authentication model in particular) was tightly coupled to the Europeana architecture. All in all, we came to the decision that in order to make this truly (re)usable, it would take almost a complete rewrite.
  • And that's what yuma.min.js essentially is: a complete rewrite of the core framework, architecture, UI, with salvaged and cherry-picked code from the old version where it made sense. Since we focused on doing one feature at a time (but complete rather than just demo-able), there are still features missing. But the plan is to have all of them in the new version eventually!
  • YUMA is written with the Google Web Toolkit (GWT - essentially a Java-to-Javascript compiler). That's still a "historic" decision from the Europeana days which we kept to minimize the migration effort. I've since come to like GWT, but been thinking about porting to pure JavaScript since the potential developer codebase would be much larger, apparently.
  • Unfortunately, there is no fixed timeline for rewrite to yuma.js, as there is currently no funded project involving yuma. We are currently working on an EU proposal which would give us funding; plus some smaller potential showcases which might provide some budget for enhancements. And I might also get some free budget to play with next year... But in the short term, it's a spare time effort & I can't give any guarantees. (Except for the fact that OpenID support is close!)
  • "we could contribute some development to speed up the timeline: are you open to cooperation?"
    Absolutely! Would be great!
  • "how hard it would be for us to ramp up on yuma development"
    Documentation is a bit sparse... But the code is really not that complex. There a some architectural issues which would need some rethinking in order to make the code for the UI compoents a bit more decoupled, but apart from that it should be straightforward to read & I'm absolutely willing to help.
  • "important missing features are: Semantic Tagging
    The main reason we don't have Semantic Tagging in .min.js yet is really the UI components... I.e. I'd say in order to get this to work, 45% is UI, 45% is creating a solid "Tag server" solution for feeding the thesaurus into the client UI. The rest is a bit of wiring in the client code, but the data model and the server still support everything the old version did. We're definitely planning to do this. But as I said - at the moment we're two people doing this as a hobby, so it's hard to specify a timeframe...
  • "Map annotation"
    Can I ask what features you're looking for in particular? For georeferencing, we're also talking to the people from on cross-integration between yuma and georeferencer.

David Haskiya answers

(Product Developer for Europeana)

  • I’m responding as Vanessa no longer works at Europeana and the ThoughtLab section is now my responsibility.
  • I’m sorry that the Annotation demo isn’t working (we’ll try to fix that!), but very happy that you’d be interested to re-use it in ResearchSpace.
  • The code is indeed open. Like all Europeana source code it is or other open license compatible with EUPL.
  • I’ll let Rainier answer as to the specifics of YUMA, but I believe the idea is to develop YUMA to have all of the functionality of the Annotation demo yet be more light-weight and simpler to integrate.


yuma.min.js aims to be a leaner, lighter and more portable user interface for the YUMA annotation system, still missing various features.
It's implemented as a 'Google Maps JavaScript API'-like library: add annotation features to your existing HTML pages with just a few lines of JavaScript!
The present version of yuma.min.js provides annotation for

  • images (any JPG, PNG, etc. included in your HTML page)
  • maps displayed in the OpenLayers Web map viewer
  • Deep Zoom images embedded with the Seadragon AJAX JavaScript viewer
  • audio files (unfinished, HTML5-capable browsers only)
  • Free-text annotation
  • Runs in your browser


  • yuma.min.js is implemented using GWT, same as YUMA.
    (Not just JQuery as I previously thought)
  • The reason for choosing GWT was productivity: the speed advantage we gain from using a statically-typed language with good tool support simply outweighed the initial 'setup penalty' that GWT suffers over traditional JavaScript. The fact that GWT automagically creates highly cross-browser compatible JavaScript is nice, too.
  • It is built with Gradle. Use "gradle gwtCompile" to build the project. Your optimized, minified JavaScript will be in the /build/gwt/yuma.min folder.
  • yuma.min.js is a purely client-side application. It does not make any use of GWT's server-side parts or the RPC mechanism.

yuma.min.js demos

  • Image annotation with replies
    Image annotation, with commenting enabled and server-side storage using the yuma4j annotation server
  • Using OpenLayers for maps
    Annotate maps and high resolution zoomable images on OpenLayers. Formats supported by OpenLayers include Zoomify, WMS, TMS, Google
  • Using Seadragon AJAX for deep zoom
    annotation of Deep Zoom images using the Seadragon AJAX viewer. The image below is hosted on the (free) Deep Zoom hosting service.

Annotation server (yuma4j)

RDF and OAC export

  • Linked Data interface: annotations are exposed as RDF resources
  • Exports to JSON, RDF, N3, TTL, OAC (beta): see below, can use for examples (from video)
  • I guess this works only with the YUMA server, since annotations are initially stored in a RDBMS



[ (formerly
Web portal for hosting digitized, high-resolution, historic maps. Allows users to Annotate and Georeference, thus contributing their knowledge about historic maps.

  • currently being developed as part of OAC: created by Cornell, builds upon YUMA
  • created with Ruby on Rails. Open source.
  • uses Zoomify (commercial) for tiled images
  • All annotations are represented in the OA data model and become Web resources that can be referred to


Last year Rainier also developed a back-end for semantic tagging using Dbpedia as the backend and ontology. This may also be of interest to you. lt goes under the working name DBpedix and the source code is available here:

Nuxeo DAM Image Annotation

(Originally from Images - view, annotate, deep zoom, references to by Jana)
Nuxeo DAM offers deem zoom + image annotation


  • Annotate image, picture and office format assets directly from the browser
  • Let users comment and discuss digital content collaboratively
  • Comment on fine details with zoom in/zoom out on graphic items
  • Navigate from annotation to annotation for quick comment review
  • GUI uses Ajax for tiling and implements annotation at different levels
  • Annotating documents by the same mechanism as images

Other useful DAM image functions:

  • Automated extraction of media metadata supporting IPTC and EXIF standards
  • Bulk tagging of media assets metadata
  • Metadata: coverage, source, region, subject, campaign, and custom metadata
  • define and configure a content model for the specific application needs
  • define metadata for media assets
  • controlled vocabularies for content description and discovery



  • Nuxeo DAM is a separate product but may be installed together with Nuxeo DMS and share same repository (docs and images)
  • Nuxeo DAM relies on ImageMagick binaries for tiling/deep zoom
    • much like the way OpenOffice is used to handle MS Office documents
    • Jana: Not sure this scales good enough. Vlado: ImageMagick is a very good library and set of tools, but it depends how they use it
    • Jana: I think they might be doing tiling on the fly which would be slow: but we need to check this


  • Only rectangular shapes are supported
  • User cannot zoom out beyond original size (Jana, what does this mean?)
  • Zoom in/zoom out via mouse wheel is not supported
  • Moving view area via dragging is not supported
  • When annotation region is not entirely into view area, it is not shown

Bottomline: Has a lot of what we need

  • but at quite basic level and with somewhat clumsy GUI
  • the solution may not scale well enough for many users and very large pictures.
  • TODO: Need to check how easy it is to extend/modify

OKFN Annotator
Similar to NYT annotator. Works on text only

PhotoStuff is an early image annotation tool (latest version May 2006).
It's created in the MINDSWAP project at Maryland, which was led by Jim Hendler and has spawned some of the important developments in semantic web (eg Clark & Parsia).
It's a Java standalone application


Core Ontology on Multimedia


But Complex


SEMLIB is an FP7-SME-2010 project that started in Jan 2011, has 2 year duration (Feb 2012 is at mid-point), and budget 920 kEUR.
Announced on OAC mlist by Christian Morbidioni (Net7, Italy).
We've partnered (with 8 others) in a proposal called WSR4Europeana (web science research for Europeana)

Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.