Image annotation tools and ontologies
- Shared Canvas
- Digital Mappaemundi
- Seadragon Ajax is open source
- Djatoka image server
- YUMA Universal Media Annotator
- Nuxeo DAM Image Annotation
- OKFN Annotator
http://www.shared-canvas.org: a shared canvas (for collaborative image annotation) based on OAC (OpenAnnotation.org) ontology.
Collaboration of Stanford and LANL, with funding by Mellon
- sample implementation
Under heavy development, still rough around the edges and slow: JS processing in browser, does not seem to use Deep Zoom server while the images are 1-2 Mb.
- sample files, can be used for understanding and as engineering examples.
In adition to OAC, uses ore:Aggregation (which is also heavily used in Europeana EDM).
The files for all demos are available (/impl/demoN/res)
- SharedCanvas- Data Model (Mar 2011)
- demo1 (Morgan Library): quite impressive!
Allows polygon areas, I think this is important for paintings (although the RS Spec shows rectangles only).
In OAC Constraints, this maps to oac:SvgConstraint
- Uses SVG and Raphael (http://raphaeljs.com/)
- demo3 (Dirac's thesis): uses MathJax
- Interoperation for Digital Medieval Manuscripts
Rough description of Data Model and Architecture.
- data model uses OAC, OAI-ORE for aggregation (eg all pages of a manuscript), a Manifest to describe all related pages
- architecture uses load-balanced Djatoka image servers
Digital Mappaemundi is about collaboration on digitized Manuscripts. I think the most significant applications in image annotation have been developed for Manuscripts.
Please watch this video: it's 14 min, but you can watch only the first half.
- Nothing to do with semtech, but it's built to solve real researcher needs.
Link between image region and concept
Show all image regions denoting a certain concept:
Application to genealogy
Text-Image Linking Environmenthttp://mith.umd.edu/tile/
The site sounds good, but the demo doesn’t seem to work
Seadragon Ajax (sometimes called Seajax) is in fact open source under a very liberal New BSD license !!!
As part of the Ajax Control Toolkit (ACT)
Of the many deep zoom interfaces to have emerged over the last few years for viewing high-resolution images on the web, Microsoft’s Seadragon is both one of the most beautiful and easiest to use. Unlike many of its competitors, the software seamlessly transitions between zoom levels without noticeably replacing the tiles, an effect that is arguably both more aesthetically pleasing and more functional than a page refresh as the user need not relocate their position each time a new zoom level loads. Additionally, Microsoft has provided a *well-developed code library for annotating images and binding regions* of the image to particular actions when clicked.
Cutting the large source image into a Deep Zoom Image (or DZI) tile set... By far the easiest way to do this is through Microsoft’s Deep Zoom Composer, a Windows-only, free, but proprietary piece of desktop software. However, there are also a set of third party tools, some open source, which can also generate the tiles on Macintosh and Linux machines.
link the open source version of Microsoft’s Seadragon to Los Alamos National Laboratory’s Jpeg2000 server, Djatoka
(I don't know how Djatoka compares to IIP Server)
Multimedia Annotation Service (YUMA Framework) (EuropeanaConnect D5.6.1).pdf
Authors: firstname.lastname@example.org; email@example.com
Also called EuropeanaConnect ThoughtLab Media Annotation Prototype, YUMA is a powerful media annotator.
It was reviewed in our Offer, sec 126.96.36.199 EuropeanaConnect Map Annotation.
I first looked at it last summer, and was extremely impressed. We've looked at a number of tools, and your tool seems to come on top on several counts:
- not only image, but also map, audio and video annotation
- deep zoom support
- OAC export
- annotation server (maybe we'll store them ourselves, but it is a nice option)
- Videos of image, map, audio, video annotation
- Annotate ancient map and correlate to modern places: Comprehensive video
The demos at http://dme.ait.ac.at/annotation are broken:
- go to http://dme.ait.ac.at/annotation/sample-images.html
- click on any image: get "The requested resource (/yuma-suite/image) is not available."
Features that are yet to come in yuma.min.js (some of them implemented in YUMA full):
- OpenID support
- Data interoperability and Linked Data publishing based on standard formats (OAC RDF)
- Semantic Tagging based on controlled vocabularies (or we can customize this to work against a thesaurus)
- Semantic tagging using DBpedia/wikipedia (hope to customize this to work against LOD and internal RDF data)
- Sharing and embedding of annotations
- History/version track for annotations
- Support for HTML5 video
- Customizable search and data export
- Annotate images, maps, audio, video
- Shape drawing tools on images, maps and video
- Special support for high-resolution map images
- Tile-based rendering for faster delivery
- Semantic tag suggestions based on geographic location
- Place search functionality
- Overlay of present-day country borders, coast outlines, etc
We'd like to use the full version, and so I have some questions:
- is the full version open source?
- what is its status: is it in a serviceable state?
- is yuma.min.js a complete rewrite of the full version?
- what's the language/implementation platform of the full version?
- if the technical plan is to rewrite the full version into yuma.js, what is the timeline for this?
- if open source, we could contribute some development to speed up the timeline: are you open to cooperation?
- how hard it would be for us to ramp up on yuma development, and how much effort it would take?
- the "full version" is Open Source as well, and the code is available on Europeana's SVN repository at europeanalabs.eu (http://europeanalabs.eu/browser/contrib/ait/trunk/yuma-client-suite). BUT - to be frank - the code is not really in a serviceable state; and it hasn't been maintained at all since the end of the EuropeanaConnect project. So I'd really advise against using it. That's also the reason we took the old demos offline after the end of the project.
- This, in fact, was the reason for why we started yuma.min.js: the original prototype had only been developed to the stage of a proof-of-concept. While many of the features were demo-able, they were yet incomplete and not suitable for full production use (which would have been addressed in a follow-up project which, unfortunately, never happend); usability was less then optimal (based on decisions that had been taken years ago...); the architecture (and authentication model in particular) was tightly coupled to the Europeana architecture. All in all, we came to the decision that in order to make this truly (re)usable, it would take almost a complete rewrite.
- And that's what yuma.min.js essentially is: a complete rewrite of the core framework, architecture, UI, with salvaged and cherry-picked code from the old version where it made sense. Since we focused on doing one feature at a time (but complete rather than just demo-able), there are still features missing. But the plan is to have all of them in the new version eventually!
- Unfortunately, there is no fixed timeline for rewrite to yuma.js, as there is currently no funded project involving yuma. We are currently working on an EU proposal which would give us funding; plus some smaller potential showcases which might provide some budget for enhancements. And I might also get some free budget to play with next year... But in the short term, it's a spare time effort & I can't give any guarantees. (Except for the fact that OpenID support is close!)
- "we could contribute some development to speed up the timeline: are you open to cooperation?"
Absolutely! Would be great!
- "how hard it would be for us to ramp up on yuma development"
Documentation is a bit sparse... But the code is really not that complex. There a some architectural issues which would need some rethinking in order to make the code for the UI compoents a bit more decoupled, but apart from that it should be straightforward to read & I'm absolutely willing to help.
- "important missing features are: Semantic Tagging
The main reason we don't have Semantic Tagging in .min.js yet is really the UI components... I.e. I'd say in order to get this to work, 45% is UI, 45% is creating a solid "Tag server" solution for feeding the thesaurus into the client UI. The rest is a bit of wiring in the client code, but the data model and the server still support everything the old version did. We're definitely planning to do this. But as I said - at the moment we're two people doing this as a hobby, so it's hard to specify a timeframe...
- "Map annotation"
Can I ask what features you're looking for in particular? For georeferencing, we're also talking to the people from georeferencer.org on cross-integration between yuma and georeferencer.
(Product Developer for Europeana)
- I’m responding as Vanessa no longer works at Europeana and the ThoughtLab section is now my responsibility.
- I’m sorry that the Annotation demo isn’t working (we’ll try to fix that!), but very happy that you’d be interested to re-use it in ResearchSpace.
- The code is indeed open. Like all Europeana source code it is http://en.wikipedia.org/wiki/European_Union_Public_Licence or other open license compatible with EUPL.
- I’ll let Rainier answer as to the specifics of YUMA, but I believe the idea is to develop YUMA to have all of the functionality of the Annotation demo yet be more light-weight and simpler to integrate.
yuma.min.js aims to be a leaner, lighter and more portable user interface for the YUMA annotation system, still missing various features.
The present version of yuma.min.js provides annotation for
- images (any JPG, PNG, etc. included in your HTML page)
- maps displayed in the OpenLayers Web map viewer
- audio files (unfinished, HTML5-capable browsers only)
- Free-text annotation
- Runs in your browser
- yuma.min.js is implemented using GWT, same as YUMA.
(Not just JQuery as I previously thought)
- yuma.min.js is a purely client-side application. It does not make any use of GWT's server-side parts or the RPC mechanism.
- Image annotation with replies
Image annotation, with commenting enabled and server-side storage using the yuma4j annotation server
- Using OpenLayers for maps
Annotate maps and high resolution zoomable images on OpenLayers. Formats supported by OpenLayers include Zoomify, WMS, TMS, Google
- Using Seadragon AJAX for deep zoom
annotation of Deep Zoom images using the Seadragon AJAX viewer. The image below is hosted on the (free) zoom.it Deep Zoom hosting service.
- Stores annotations. Then they can be searched, viewed, put on a timeline
- Implementation: jboss.resteasy-jaxrs hibernate postgresql apache.wicket codehaus.jackson-mapper-asl jena rome:rome rome:opensearch openid4java hsqldb
- Built with Gradle
- Defines abstract AnnotationStore, default implementation is a HibernateAnnotationStore
- See Annotation fields
- yuma4j source
- yuma4j demo server
- Linked Data interface: annotations are exposed as RDF resources
- Exports to JSON, RDF, N3, TTL, OAC (beta): see below, can use for examples (from video)
- I guess this works only with the YUMA server, since annotations are initially stored in a RDBMS
[http://maphub.github.com/api/ (formerly https://github.com/yuma-annotation/maphub-portal)
Web portal for hosting digitized, high-resolution, historic maps. Allows users to Annotate and Georeference, thus contributing their knowledge about historic maps.
- currently being developed as part of OAC: created by Cornell, builds upon YUMA
- created with Ruby on Rails. Open source.
- uses Zoomify (commercial) for tiled images
- All annotations are represented in the OA data model and become Web resources that can be referred to
Last year Rainier also developed a back-end for semantic tagging using Dbpedia as the backend and ontology. This may also be of interest to you. lt goes under the working name DBpedix and the source code is available here:http://europeanalabs.eu/browser/contrib/ait/trunk/dbpedix
(Originally from Images - view, annotate, deep zoom, references to by Jana)
Nuxeo DAM offers deem zoom + image annotation
- Annotate image, picture and office format assets directly from the browser
- Let users comment and discuss digital content collaboratively
- Comment on fine details with zoom in/zoom out on graphic items
- Navigate from annotation to annotation for quick comment review
- GUI uses Ajax for tiling and implements annotation at different levels
- Annotating documents by the same mechanism as images
Other useful DAM image functions:
- Automated extraction of media metadata supporting IPTC and EXIF standards
- Bulk tagging of media assets metadata
- Metadata: coverage, source, region, subject, campaign, and custom metadata
- define and configure a content model for the specific application needs
- define metadata for media assets
- controlled vocabularies for content description and discovery
- Annotation user doc
- Commenting assets: this is like an annotation on the image as a whole
- Questions and answers
- Nuxeo DAM is a separate product but may be installed together with Nuxeo DMS and share same repository (docs and images)
- Nuxeo DAM relies on ImageMagick binaries for tiling/deep zoom
- much like the way OpenOffice is used to handle MS Office documents
- Jana: Not sure this scales good enough. Vlado: ImageMagick is a very good library and set of tools, but it depends how they use it
- Jana: I think they might be doing tiling on the fly which would be slow: but we need to check this
- Only rectangular shapes are supported
- User cannot zoom out beyond original size (Jana, what does this mean?)
- Zoom in/zoom out via mouse wheel is not supported
- Moving view area via dragging is not supported
- When annotation region is not entirely into view area, it is not shown
Bottomline: Has a lot of what we need
- but at quite basic level and with somewhat clumsy GUI
- the solution may not scale well enough for many users and very large pictures.
- TODO: Need to check how easy it is to extend/modify
Similar to NYT annotator. Works on text only
http://www.mindswap.org/2003/PhotoStuff is an early image annotation tool (latest version May 2006).
It's created in the MINDSWAP project at Maryland, which was led by Jim Hendler and has spawned some of the important developments in semantic web (eg Clark & Parsia).
It's a Java standalone application
Core Ontology on Multimedia
SEMLIB is an FP7-SME-2010 project that started in Jan 2011, has 2 year duration (Feb 2012 is at mid-point), and budget 920 kEUR.
Announced on OAC mlist by Christian Morbidioni (Net7, Italy).
We've partnered (with 8 others) in a proposal called WSR4Europeana (web science research for Europeana)
EXCELLENT VIDEO, watch it.
Has two parts:
- creating annotation. Terms are searched in thesauri or in LOD (in this case FreeBase).
I think that looking for terms outside the local repo is an EXCELLENT idea
- exploring annotations, using Exhibit
- creating annotation. Terms are searched in thesauri or in LOD (in this case FreeBase).
- posted some videos recently that reflects more the current state of development:http://www.youtube.com/watch?v=gVA_v152Qn0http://www.youtube.com/watch?v=z1hXr5K3kTM http://www.youtube.com/watch?v=Q5KTzEcRrPs
The demo is upposed to be live here, but is broken