OWLIM-SE Administrative Tasks

Version 1 by dobri.stoilov
on May 20, 2011 10:42.

compared with
Current by barry.bishop
on May 24, 2011 20:15.

This line was removed.
This word was removed. This word was added.
This line was added.

Changes (7)

View Page History
This section contains information about performing day to day administration tasks with a BigOWLIM an OWLIM-SE repository. Most of these standard operations can be achieved using the Sesame software. Some information is repeated here and tailored to the specifics of BigOWLIM. OWLIM-SE.

h1. Dumping a large repository to RDF

The Sesame openRDF workbench Web application has an export function that can be used to export the contents of moderately sized repositories. However, using this with large repositories (more than a hundred million statements or more) causes problems, usually timeouts for the Servlet container (Tomcat) hosting the application. Also, the workbench cannot be used when using BigOWLIM OWLIM-SE without Tomcat.
Therefore, a more straightforward approach for exporting RDF data from repositories is to do this programmatically. The Sesame {{RepositoryConnection.getStatements()}} method can be called with the {{includeInferred}} flag set to {{false}} (in order not to serialise the inferred statements). Then the returned iterator can be used to visit every explicit statement in the repository and one of the Sesame RDF writer implementations can be used to output the statements in the chosen format. If the data will be re-imported, the N-Triples format is recommended, because this can easily be broken in to large 'chunks' that can be inserted and committed separately. The following code snippet shows how an export can be achieved using this approach:

h1. Migration of a data from incompatible versions

The basic procedure is to export the RDF data from the old version of BigOWLIM OWLIM-SE and then reload it in to a new repository instance that uses the new version of BigOWLIM. OWLIM-SE. Exporting is straightforward when using the Sesame workbench -- simply click the 'Export' button, choose the format and click 'download'. To import in to a new repository, click 'add', select a format, specify the file and base URI, then click 'Upload'.
If not using the Sesame workbench, the export must be done programmatically using the {{RepositoryConnection.getStatements()}} API, because the Sesame console does not have an export function. NOTE: It should be possible to export only the explicit statements, as the inferred statements will be recomputed at load time. Fortunately, the Sesame console application does have a 'load' function and this can be used to reload the exported statements.

Once created, the repository configuration is maintained in the Sesame SYSTEM repository. There is no easy generic way of changing this configuration, but there are several possibilities.
Firstly, BigOWLIM OWLIM-SE allows most of the configuration parameters to be overridden at runtime by specifying the parameter values as JVM options. For example, to change the cache-memory configuration parameter, pass \-Dcache-memory=1g option to the JVM that is hosting the application using BigOWLIM. OWLIM-SE.
The second approach is to modify the SYSTEM repository directly. Caution, make sure that the SYSTEM repository is not corrupted and that no other repository configurations are damaged. There are no tools for doing this, but it is probably easiest (if the current configuration is known) to just remove the repository configuration using the Sesame console (drop <repository_id>) or the Sesame workbench WebApp. Then the configuration in the TTL file can be modified and added to the SYSTEM repository (keeping the same repository id).

h1. Restoring the repository after a crash

A BigOWLIM OWLIM-SE repository image can become corrupted if BigOWLIM OWLIM-SE (or the Java application that hosts it) crashes. In particular, the repository gets corrupted if BigOWLIM OWLIM-SE is interrupted after a successful commit but before flushing the update to disk.
By default, if BigOWLIM OWLIM-SE detects that the last shutdown was not normal, it will attempt to restore its internal state back to the end of the last successfully committed transaction. Therefore, users will not normally need to invoke a restore process manually. However, if the {{database-recovery-policy}} parameter has been changed to anything other than {{recover}}, then no attempt will be made to restore any lost data. In this situation, the user may want to invoke a restore process manually as follows.
If an abnormal termination occurs, it is advisable to execute a database restore as soon as possible. This can also help if BigOWLIM OWLIM-SE fails to load a repository during startup for some reason. The BigOWLIM OWLIM-SE jar contains a utility for restoring database images. It can be executed in the root folder of the BigOWLIM OWLIM-SE distribution as follows:

{{java \-cp lib/*:ext/\* com.ontotext.trree.DatabaseRestorer \[path\] \[indexsize\] \[ruleset\]}}