View Source

Image Annotation Design
{toc}
h1. OAC Representation
We use the [Open Annotation Collaboration] (OAC) ontology to represent image annotations.
- the target is an [oac:ConstrainedTarget|http://www.openannotation.org/spec/beta/#DM_Constraint] that uses:
-- [cnt:ContentAsText|http://www.openannotation.org/spec/beta/index.html#DM_Constraint_Inline] (Inline Constraint) to embed the SVG info in RDF
-- [oac:SvgConstraint|http://www.openannotation.org/spec/beta/constraints.html#DM_Serialization] to represent the annotated region
-- The OAC spec says "should be a single SVG element, not an entire SVG canvas" so it seems an implicit <svg> tag is assumed around the SVG data:
{code:xml}
<svg version="1.1" xmlns="http://www.w3.org/2000/svg">
...
</svg>
{code}
- the body is an oac:Body and includes the same properties as for [Data Annotation|Open Annotation Collaboration#RS Data Annotation in OAC]
-- if a reply is made, it is attached to the same oac:ConstrainedTarget and has a link rso:reply_to pointing to the original annotation

h2. Example
We start with an example of the basic case, then treat some special requirements.
{code}
@prefix oac: <http://www.openannotation.org/ns/>. # Open Annotation Collaboration
@prefix dcterms: <http://purl.org/dc/terms/>. # Dublin Core Terms
@prefix rso: <http://www.researchspace.org/ontology/>. # ResearchSpace Ontology
@prefix cnt: <http://www.w3.org/2008/content#>. # Content in RDF
@prefix xsd: <http://www.w3.org/2001/XMLSchema#>. # XML Schema Datatypes

<annot> a oac:Annotation;
oac:hasBody <annot/body>;
oac:hasTarget <annot/target>.
<annot/target> a oac:ConstrainedTarget;
oac:constrainedBy <annot/constraint>;
oac:constrains <http://www.researchspace.org/Rembrandt/images/DT219363.jpg>.
<annot/constraint> a oac:SvgConstraint, cnt:ContentAsText;
dc:format "SVG";
cnt:chars """<polyline points="10,20 15,25 25,10 10,20"/>""".
<annot/body> a oac:Body;
rso:P3_has_title "The nose is large and wrinkled";
dcterms:creator rs-users:VladimirAlexiev;
dcterms:created "2002-03-15"^^xsd:date...
{code}
[^image-annotation-example.vsd]
!image-annotation-example.png|width=900!

h1. Special Requirements

h2. Markers
A marker (point, pin) is a point carrying a particular symbol (shape).
OAC does not support markers, since an image annotation target is represented as an [oac:SvgConstraint|http://www.openannotation.org/spec/beta/constraints.html#DM_Serialization], which consists of an [SVG shape|http://www.w3.org/TR/SVG/shapes.html]: path, rect, circle, ellipse, line, polyline, polygon, g(roup) (eg used to create a region with a hole).

A ResearchSpace requirement is to allow *markers* as annotation targets
- A marker is characterized by:
-- Point (coordinates)
-- Symbol (marker type or shape)
--- Can be any geometry or image
--- Should not scale with image zoom (should remain constant in size)
--- RS3.3 will support a small circle and a pin (as on Google Maps)
- It could also be useful to support *multi-markers* consisting of several points, where the symbol is fixed

SVG does not have the concept of a standalone point/marker, but there are several alternatives:

h3. 1. Use-Symbol
Represent the point as [use|http://www.w3.org/TR/SVG/struct.html#UseElement] and the shape as a [symbol|http://www.w3.org/TR/SVG/struct.html#SymbolElement]
- "use" carries x,y,width,height
- "symbol" carries the shape geometry or image
- pro: supports constant size through explicit width,height
- cons: doesn't support multi-markers
- cons: requires <svg><defs> while it's unclear whether OAC supports <svg>
{code:xml}
<svg>
<defs> <symbol id="Triangle"> <polyline points="0,0 10,5 0,10 0,0" /> </symbol> </defs>
<use x="1000" y="750" width="10" height="10" xlink:href="#Triangle" />
</svg>
{code}
- if the symbol contains a single graphics element, then <symbol> can be skipped:
{code:xml}
<svg>
<defs> <polyline id="Triangle" points="0,0 10,5 0,10 0,0" /> </defs>
<use x="1000" y="750" width="10" height="10" xlink:href="#Triangle" />
</svg>
{code}

h3. 2. Polyline-Marker
Represent the point as [polyline|http://www.w3.org/TR/SVG/shapes.html#PolylineElement] and the shape as [marker|http://www.w3.org/TR/SVG/painting.html#Markers]
- Markers are styling elements (symbols) "attached to one or more vertices of ‘path’, ‘line’, ‘polyline’ and ‘polygon’ elements"
-- the marker carries markerWidth,markerHeight
-- use markerUnits="userSpaceOnUse" so the marker size is absolute (the default markerUnits="strokeWidth" scales the marker according to the path's stroke)
-- use the default orient="0" so the marker has absolute orientation (orient="auto" rotates the marker along the path's tangent)
- A single point can be represented as a singular (degenerate) polyline; a multi-marker as a polyline with no stroke
- pro: supports multi-markers
- cons: requires <svg><defs> while it's unclear whether OAC supports <svg>
{code}
<svg>
<defs> <marker id="Triangle"> <polyline points="0,0 10,5 0,10 0,0" /> </marker> </defs>
<polyline points="1000,750" marker="url(#Triangle)" />
</svg>
{code}

h3. 3. Class "marker"
Use a normal graphics element (shape or image), signify it as a marker using [class|http://www.w3.org/TR/SVG/styling.html#ClassAttribute]
- cons: to achieve constant size, requires a [transform|http://www.w3.org/TR/SVG/coords.html#TransformAttribute] or manual size recalculation
- cons: does not support multi-markers
- pro: does not require <svg><defs>
{code:xml}<polyline points="0,0 10,5 0,10 0,0" class="marker" />{code}

This is the current decision, but it's unclear whether it is optimal

h2. Image URLs
Since Image URLs are used to bind the annotation, they need to be stable. The following should be taken into account (see [RS URLs and URIs]):
- the possibility of per-project domains
- server migration from development to testing to production: research and finally publication
- need to support deep zoom images
- possible changes in image format or server technology
Eg an image served by IIP and converted from TIF to JPG may have this URL ([Rembrandt data#Image URLs]):
http://rembrandtdatabase.adlibsoft.com/IIPImageServer/IIPImageServer.exe?FIF=D:/rembrandtdatabase.adlibsoft.com/Images/mh0147_front_nl_2002.tif&HEI=300&CVT=JPEG

TODO TODO: not yet resolved

h2. View Configuration
Image annotations can be checked/unchecked (made visible/invisible). But we also need a concept of "currently selected annotation" so that:
- When the user Replies, the reply is attached to the current annotation
- When the user selects an annotation, the image view is set so that the annotation is fully visible

There are two alternatives to accomplish the latter:
# Set the view to the bounding box of the annotated region
# (current decision) Restore the view to what it was when the annotation was created or last edited.
This is better since the user may want to see more than the annotated region, and if the same view was shared by several annotations then the view won't "jump"
-- The screens of two users may be different, so the available image area may be different (including different aspect ratios).
We have decided to keep fixed: center of image area, aspect ratio, zoom level; while allowing the image fringes to be cropped

TODO TODO: not yet resolved.

Considerations:
- Use the [viewBox|http://www.w3.org/TR/SVG/coords.html#ViewBoxAttribute] attribute [that is available|http://www.w3.org/TR/SVG/coords.html#ElementsThatEstablishViewports] on svg, symbol, image
- Use the [preserveAspectRatio|http://www.w3.org/TR/SVG/coords.html#PreserveAspectRatioAttribute] attribute that is available on svg, symbol, image, marker, view
This should keep center and aspect ratio fixed, but the zoom still may vary
{code:xml}preserveAspectRatio="XMidYMid slice"{code}
- <svg> provides attributes to specify the canvas area:
{code:xml}width="4in" height="2in" viewBox="0 0 4000 2000"{code}
- [SVG Linking|http://www.w3.org/TR/SVG/linking.html#SVGFragmentIdentifiers] describes a number of options to link to a particular area of an SVG canvas
- for a deep zoom image, the IIP URL specifies the zoom level and tile origin, so this also relates to the previous section
- IIP represents the zoom level as log2 (eg 5 means 2^5=32), but we should represent it as a real number to accommodate smooth zoom in the future
- Decide where to store the view configuration: best if it's in the SvgConstraint, depending on answer from OAC mlist

{tasklist}
Vlado: ask in OAC mailing list if an implicit <svg> tag is assumed around the SVG data
Stanislav: test if SVG singular polyline with marker works
Vlado, Jana: figure out [#Image URLs] in the case of IIP
Vlado, Stanislav: work out and document a [#View Configuration] example
{tasklist}