Every feature or bug fix needs to be tracked in an issue on GitHub. Development is done in branches. Based on the milestone (see the issue description on GitHub), the new branch is either created from master (if the code should be in the next major release) or from a bugfix release branch (if the code should be in the next minor release). In order to get the code in production, you need to create a pull request on GitHub of your branch into the target branch (as described before).

In order to contribute to INCEpTION, you need to create a pull request. This section briefly guides you through the best way of doing this:

You can create the pull request any time after your first commit. I.e. you do not have to wait until you are completely finished with your implementation. Creating a pull request early tells other developers that you are actively working on an issue and facilitates asking questions about and discussing implementation details.

Before committing, make sure that you specified your email and name in the git config so that commits can be attributed to you. This can e.g. be done as described in the Git Documentation.

All sources files are stored using UNIX line endings. If you develop on Windows, you have to set the core.autocrlf configuration setting to input to avoid accidentally submitting Windows line endings to the repository. Using input is a good strategy in most cases, thus you should consider setting this as a global (add --global) or even as a system (--system) setting.

After changing this setting, best do a fresh clone and check-out of the project.

On Linux or OS X, having a full JDK installed on your system is generally sufficient. You can skip on to the next section.

On Windows, you need to edit the eclipse.ini file and directly before the -vmargs line, you have to add the following two lines. Mind to replace C:/Program Files/Java/jdk11 with the actual location of the JDK/version on your system. Without this, Eclipse will complain that the jdk.tools:jdk.tools artifact would be missing.

Checkout out the INCEpTION git repository with your favorite git client. If you use the command-line client, use the command

In Eclipse, go to File → Import, choose Existing Maven projects, and select the folder to which you have cloned INCEpTION. Eclipse should automatically detect all modules.

We use a style for formatting the source code in INCEpTION (see Checkstyle and Formatting. The following section describes how to use it with Eclipse.

First, obtain the DKPro code formatting profile from the DKPro website (Section "Code style"). In Eclipse, go to Preferences → Java → Code Style → Formatter to import the file. Apparently, the files can also be used with IntelliJ via the [Eclipse Code Formatter](https://plugins.jetbrains.com/plugin/6546-eclipse-code-formatter) plugin.

Second, install the Checkstyle plugin for Eclipse as well as the Maven Checkstyle plugin for Eclipse. These plugins make Eclipse automatically pick up the checkstyle configuration from the Maven project and highlight formatting problems directly in the source code editor.

We use a style for formatting the source code in INCEpTION (see Checkstyle and Formatting. The following section describes how to use it with IntelliJ IDEA.

First, install the Checkstyle-IDEA plugin. In File | Settings | Other Settings | Checkstyle, navigate to the Checkstyle tab. Start to add a new configuration file by clicking on the + on the right, navigate to inception-build/src/main/resources/inception/checkstyle.xml and apply the changes. Make sure to check the box next to the newly created configuration and apply it as well.

In order to achieve the same formatting and import order as Eclipse, install the Eclipse Code Formatter. Download the DKPro Eclipse Code Style file. In File | Settings | Other Settings | Eclipse Code Formatter, create a new profile using this file.

Also make sure to enable auto import optimization in File | Settings | Editor | General | Auto Import.

To format your source code on save, we also recommend to install the Save Actions plugin and configure it accordingly.

This requires IntelliJ IDEA Ultimate. Using Tomcat allows editing HTML,CSS and JavaScript on the fly without restarting the application. First, download Apache Tomcat from http://tomcat.apache.org/ (we’re using version 8.5). Then, you need to create a Tomcat server runtime configuration in Run | Edit Configurations…. Click on the ` icon, select `Tomcat Server -> Local`. Click on the `Deployment` tab and then on the ` icon to select an artifact to deploy. Choose the exploded war version. Select the Server tab, navigate to the path of your Tomcat server, and update the on Update action to Update classes and resources for both. Make sure that all port settings are different. You now can start or debug your web application via Tomcat. If starting throws a permission error, make sure that the mentioned file, e.g. catalina.sh is marked as executable.

Experimental: If desired, you can also use hot-code replacement via HotswapAgent. This allows you to change code, e.g. adding methods without needing to restart the Tomcat server. For this, follow the excellent HotSwap IntelliJ IDEA plugin guide.

The documentation can be built using a support class in inception-doc/src/test/java/de/tudarmstadt/ukp/inception/doc/GenerateDocumentation.java. To make it usable from Intellij IDEA, you need to build the whole project at least once. Run the class. If it fails, alter the run configuration and add a new environment variable INTELLIJ=true and check that the working directory is the INCEpTION root directory. The resulting documentation will be in target/doc-out.

When changing the database schema, migrations from the current schema to the new one need to be defined. They describe how the schema needs to be modified. This way, INCEpTION can be upgraded to newer versions without needing to manually alter the database schema. The migration process determines the current version of the schema and only applies transformations from there to the newest one. Each module defines its own data base tables and migrations in a file called db-changelog.xml. These are automatically discovered by Liquibase and used when starting INCEpTION.

The original document uploaded by a user into a project. The document is preserved in its original format.

Annotations made by a particular user on a document. The annotation document is persisted separately from the original document. There is one annotation document per user per document. Within the tool, a CAS data structure is used to represent the annotation document.

The layers mechanism allows supporting different types of annotation layers, e.g. span layers, relation layers or chain layers. It consists of the following classes and interfaces:

To add support for a new type of layer, create a Spring component class which implements the LayerSupport interface. Note that a single layer support class can handle multiple layer types. However, it is generally recommended to implement a separate layer support for every layer type. Implement the following methods:

The features mechanism allows supporting different types of annotation features, e.g. string features, numeric features, boolean features, link features, etc. It consists of the following classes and interfaces:

To add support for a new type of feature, create a Spring component class which implements the FeatureSupport interface. Note that a single feature support class can handle multiple feature types. However, it is generally recommended to implement a separate layer support for every feature type. Implement the following methods:

If the new feature has special configuration settings, then implement the following methods:

The Mtas index is implemented in the MtasDocumentIndex and MtasDocumentIndexFactory classes. Furthermore, the MtasUimaParser class provides a parser to be used by Lucene when adding a new document to the index.

The factory allows to build a new MtasDocumentIndex through the getNewIndex method, which is called by the search service.

This class holds the main functionalities of a Mtas index. Its methods are called by the search service and allow to create, open close and drop a Mtas index. It allows to add or delete a document from an index, as well as to perform queries on the index.

Each index is related to only one project, and every project can have only one index from a given search provider.

When adding a document to a Mtas index, the Lucene engine will use the class MtasUimaParser in order to find out which are the tokens and annotations to be indexed.

The parser is responsible for creating a new TokenCollection to be used by Lucene, whenever a new document is being indexed. The token collection consists of all the tokens and annotations found in the document, which are transformed into Mtas tokens in order to be added to the Lucene index. The parser scans the document CAS and goes through all its annotations, finding out which ones are related to the annotation layers in the document’s project - those are the annotations to be indexed. Currently, the parser only indexes span type annotations.

Recommenders provide the ability to generate annotation suggestions. Optionally, they can be trained based on existing annotations. Also optionally, they can be evaluated.

Suggestion supports provide everything necessary to handle annotation suggestions. This includes:

The subsystem is made up of the following main APIs:

This section describes the overall design of internal recommenders in INCEpTION and gives a tutorial on how to implement them. Internal recommenders are created by implementing relevant Java interfaces and are added via Maven dependencies. These are then picked up during application startup by the Spring Framework.

For this tutorial, we will add a recommender for named entities that uses the data majority label for predicting, i.e. it predicts always the label that appears most often in the training data. The full code for this example can be found in the inception-example-imls-data-majority module.

New logging adapters should be created in the module which provides the event they are logging. Logging adapters for events generated outside INCEpTION (i.e. in upstream code) are usually added to the log module itself.

To add support for a logging a new event, create a Spring component class which implements the EventLoggingAdapter interface. Implement the following methods depending on the context in which the event is triggered:

The methods getEvent, getUser and getCreated normally do not need to be implemented.

Most event adapters implement the getDetails method. This method must return a JSON string which contains any relevant information about the event not covered by the methods above. E.g. for an annotation manipulation event, it would contain information helping to identify the annotation and the state before and after the manipulation. In order to generate this JSON string, the adapter typically contains an inner class called Details to which the detail information from the event is copied and which is then serialized to JSON using JSONUtil.toJsonString(…​).

An IRI Schema defines the following attributes that are used for making queries in a knowledge base.

There are multiple classes in the knowledge base module that model the IRI Schema of a knowledge base. All the classes share that they have a single class-attribute for each IRI in the IRI Schema. However each class has a different use case. The relevant classes are shown here.

The module also includes the NamedEntityLinker recommender which can be used to generate annotation recommendations. It gets triggered for any NamedEntity annotations and suggests which KB items to link them to.

A document-rendering editor loads the document and annotation data from the backend and then renders the document including the annotations. This is typically the case for editors using SVG to display the document. The editor has the ability to define the layout of the document in such a ways that the annotations fit nicely.

For document-rendering editors, the plugin.json file offers the following settings:

The external editor mechanism loads the template file within an IFrame that is embedded in the annotation page. Any CSS or JavaScript files needed by the plugin must be referenced by the template file using a relative location. For example let’s assume a file editor.html which needs to load a editor.css style sheet and an editor.js JavaScript file:

Some annotation editors overlay their annotations on an already existing document view. For example, annotations could be overlaid on a HTML or PDF document. In this case, the external editor mechanism can be configured to use a particular DocumentView plugin on the server to render the document and to display it within an IFrame that is embedded in the annotation page. The editor plugin JavaScript and CSS stylesheet files required are then injected into this IFrame as well.

Currently supported views are:

Every editor should provide a policy.yaml right next to the plugin.json. The policy.yaml declares all elements and attributes that are supported by the editor. This policy file should be written as a safelist, i.e. it should say exactly what is permitted instead of saying what is not allowed. Allowing the wrong elements and attributes may cause security problems, e.g. if they can contain executable JavaScript or load data from remote locations.

There are several elements like script, meta, applet, link, iframe as well as a which are and JavaScript event attributes always filtered out.

If an editor does not provide a policy.yaml file, a default built-in policy is used which allows most HTML formatting elements.

There are two types of policies: element policies, and attribute policies.

Editors can be implemented in JavaScript or languages that can be compiled to JavaScript such as TypeScript. To facilitate the implementation, INCEpTION provides a set of interface definitions for TypeScript, in particular AnnotationEditorFactory and AnnotationEditor.

To make use of these, set up a package.json file next to the plugin.json file. In the package.json file, add @inception-project/inception-js-api as a dependency. The example below also already includes TypeScript and ESBuild as dependencies.

The minimal editor implementation consists of three JavaScript/TypeScript files:

There are only few requirements to a PDF annotation tool for integration into INCEpTION. It must provide support for span and relation annotations and it should also be lightweight and easily modifiable to fit into INCEpTION.

There are two PDF annotation tools up for discussion. The first one is PDFAnno and the second is Hypothes.is. Both tools are web-based and open source software available on GitHub.

PDFAnno is a lightweight annotation tool that only supports the PDF format. It was created specifically to solve the lack of free open source software for annotating PDF documents which is also capable of creating relations between annotations. This is described in the publication about PDFAnno by Shindo et al.

Hypothes.is is a project that was created to provide an annotation layer over the web. The idea is to be able to create annotations for all content available on the internet and to share it with other people. Hence Hypothes.is provides the functionality to annotate PDF documents.

PDFAnno compared to Hypothes.is comes with a smaller code base and is less complex. Both editors feature span annotations, however only PDFAnno provides the functionality to create relations between span annotations which is required in INCEpTION. As Hypothes.is was designed to share annotations with others a login mechanism is part of the software.

PDFAnno provides relations, is more lightweight and does not have a login functionality, which would have to be removed. Hence PDFAnno fits the requirements better than Hypothes.is and was chosen as the PDF annotation tool for integration into INCEpTION.

PDFAnno uses PDF.js to render PDF documents in the browser. The tool PDFExtract is used to extract information about the PDF document text. It produces a file in which each line contains information about one character of the text. Information includes the page, the character and the position coordinates of the character in the PDF document, in the given order and separated by a tab character. An example:

There are also draw operations included which are of no relevance for the use in INCEpTION. Characters which have no unicode mapping have the value NO_UNICODE. The PDFExtract file does not contain information about any whitespaces that occur in the PDF document text. PDFAnno requires the PDF document and the PDFExtract file to work. The PDF document can be obtained from the INCEpTION backend. To also provide the PDFExtract file, the tool was slightly modified so that it can be used as a library in INCEpTION.

PDFAnno provides an API for handling annotations. It is possible to import a list of annotations by providing an URL for download. This list has to be in the TOML format. Span annotations require the begin and end positions of the characters it covers. This positions are equal to the line number of characters in the PDFExtract file. A span annotation example in TOML format:

The Brat editor used in INCEpTION works only on plain text. For PDF documents this plain text is obtained by the use of DKPro PDF Reader. The reader extracts the text information from the PDF document and performs operations to ensure correct text ordering and to replace certain character sequences with substitutes from a substitution table.

As the extractions between PDFAnno and INCEpTION differ a mapping of those representations must be implemented to ensure annotations can be exchanged between the frontend and the backend and are usable across all editor modes.

To use a mapping method between the text representation of PDFAnno and INCEpTION at first they must be preprocessed to have a similar structure.

As the PDFExtract file does not only contain the text string, first the characters of the file need to be obtained and appended to a string. All draw operations and NO_UNICODE lines are ignored. As DKPro PDF Reader uses a substitution table to sanitize the document text, the same substitution table is used to sanitize the obtained string.

The PDFExtract file does not contain any whitespaces present in the document text, however DKPro PDF Reader preserves them. The whitespaces are removed from the DKPro PDF Reader string to have a similar structure to the PDFExtract sanitized string content.

Even though both representations now are in a similar shape it can still happen that the content in both strings differs. For example ordering of text areas could be messed up which can especially happen for PDF documents that contain multiple text columns on one page. As both representations are not equal even after preprocessing, a mapping algorithm has to be implemented to find the text of annotations from one representation in the respective other representation.

There are multiple ways to achieve a mapping between PDFAnno and INCEpTION for annotations. Two methods were tested during development: exact string search with context and sequence alignment.

The first option is to make an exact search for the annotation text. However as annotations often cover only one token an exact search for the annotation text would result in multiple occurrences. To get a unique result it is required to add context to the annotation text. As this still can yield multiple occurrences, context is expanded until a unique mapping or no mapping at all is found. Performing this for all annotations results in a lot of string search operations. However the performance can be improved by searching for all annotations in the target string at once with the help of the Aho-Corasick algorithm.

Another approach is to use sequence alignment methods which are popular in bioinformatics. PDF document texts are rather large and most sequence alignment algorithms require O(M x N) memory space, where M and N are the size of the two sequences. This results in a large memory consumption on computing the alignment, hence an algorithm should be used that works with less memory. Such an algorithm is Hirschbergs algorithm. It consumes only O(min(M,N)) memory.

The advantage of the sequence alignment method would be a direct mapping between the representation of PDFExtract and DKPro PDF Reader. However, during testing for larger documents, for example 40 pages, the duration until Hirschbergs algorithm finished was too long and would be unsatisfying for a user. The exact string search however takes increasingly longer to compute mappings the larger the document is and the more annotations have to be mapped. As discussed the Aho-Corasick algorithm reduces the time. However, this still does not scale well for larger documents. To overcome this issue a page wise rendering of annotations was introduced. When navigating through the PDF document in PDFAnno annotations are rendered dynamically per page. In detail, this means whenever the user moves through the document, the current page changes and the user stops movement for 500 ms, the annotations for the previous, current and next page are rendered. This way large documents can be handled by the PDF editor without long wait times for the user.

The exact string search seemed to perform well in terms of finding matching occurrences for annotations in both directions. For the manually tested documents all annotations were found and matched.