Image Annotation Design

OAC Representation

We use the [Open Annotation Collaboration] (OAC) ontology to represent image annotations.

• the target is an oac:ConstrainedTarget that uses:
  • cnt:ContentAsText (Inline Constraint) to embed the SVG info in RDF
  • oac:SvgConstraint to represent the annotated region
  • The OAC spec says "should be a single SVG element, not an entire SVG canvas" so it seems an implicit <svg> tag is assumed around the SVG data:
• the body is an oac:Body and includes the same properties as for [Data Annotation]
  • if a reply is made, it is attached to the same oac:ConstrainedTarget and has a link rso:reply_to pointing to the original annotation

Example

We start with an example of the basic case, then treat some special requirements.

image-annotation-example.vsd

Special Requirements

Markers

A marker (point, pin) is a point carrying a particular symbol (shape).
OAC does not support markers, since an image annotation target is represented as an oac:SvgConstraint, which consists of an SVG shape: path, rect, circle, ellipse, line, polyline, polygon, g(roup) (eg used to create a region with a hole).

A ResearchSpace requirement is to allow markers as annotation targets

• A marker is characterized by:
  • Point (coordinates)
  • Symbol (marker type or shape)
    • Can be any geometry or image
    • Should not scale with image zoom (should remain constant in size)
    • RS3.3 will support a small circle and a pin (as on Google Maps)
• It could also be useful to support multi-markers consisting of several points, where the symbol is fixed

SVG does not have the concept of a standalone point/marker, but there are several alternatives:

1. Use-Symbol

Represent the point as use and the shape as a symbol

• "use" carries x,y,width,height
• "symbol" carries the shape geometry or image
• pro: supports constant size through explicit width,height
• cons: doesn't support multi-markers
• cons: requires <svg><defs> while it's unclear whether OAC supports <svg>
• if the symbol contains a single graphics element, then <symbol> can be skipped:

2. Polyline-Marker

Represent the point as polyline and the shape as marker

• Markers are styling elements (symbols) "attached to one or more vertices of ‘path’, ‘line’, ‘polyline’ and ‘polygon’ elements"
  • the marker carries markerWidth,markerHeight
  • use markerUnits="userSpaceOnUse" so the marker size is absolute (the default markerUnits="strokeWidth" scales the marker according to the path's stroke)
  • use the default orient="0" so the marker has absolute orientation (orient="auto" rotates the marker along the path's tangent)
• A single point can be represented as a singular (degenerate) polyline; a multi-marker as a polyline with no stroke
• pro: supports multi-markers
• cons: requires <svg><defs> while it's unclear whether OAC supports <svg>

3. Class "marker"

Use a normal graphics element (shape or image), signify it as a marker using class

• cons: to achieve constant size, requires a transform or manual size recalculation
• cons: does not support multi-markers
• pro: does not require <svg><defs>

This is the current decision, but it's unclear whether it is optimal

Image URLs

Since Image URLs are used to bind the annotation, they need to be stable. The following should be taken into account (see RS URLs and URIs):

• the possibility of per-project domains
• server migration from development to testing to production: research and finally publication
• need to support deep zoom images
• possible changes in image format or server technology
  Eg an image served by IIP and converted from TIF to JPG may have this URL (Rembrandt data#Image URLs): http://rembrandtdatabase.adlibsoft.com/IIPImageServer/IIPImageServer.exe?FIF=D:/rembrandtdatabase.adlibsoft.com/Images/mh0147_front_nl_2002.tif&HEI=300&CVT=JPEG

TODO TODO: not yet resolved

View Configuration

Image annotations can be checked/unchecked (made visible/invisible). But we also need a concept of "currently selected annotation" so that:

• When the user Replies, the reply is attached to the current annotation
• When the user selects an annotation, the image view is set so that the annotation is fully visible

There are two alternatives to accomplish the latter:

1. Set the view to the bounding box of the annotated region
2. (current decision) Restore the view to what it was when the annotation was created or last edited.
   This is better since the user may want to see more than the annotated region, and if the same view was shared by several annotations then the view won't "jump"
   • The screens of two users may be different, so the available image area may be different (including different aspect ratios).
     We have decided to keep fixed: center of image area, aspect ratio, zoom level; while allowing the image fringes to be cropped

TODO TODO: not yet resolved.

Considerations:

• Use the viewBox attribute that is available on svg, symbol, image
• Use the preserveAspectRatio attribute that is available on svg, symbol, image, marker, view
  This should keep center and aspect ratio fixed, but the zoom still may vary
• <svg> provides attributes to specify the canvas area:
• SVG Linking describes a number of options to link to a particular area of an SVG canvas
• for a deep zoom image, the IIP URL specifies the zoom level and tile origin, so this also relates to the previous section
• IIP represents the zoom level as log2 (eg 5 means 2^5=32), but we should represent it as a real number to accommodate smooth zoom in the future
• Decide where to store the view configuration: best if it's in the SvgConstraint, depending on answer from OAC mlist