ValuesML
Unravelling Expressed Values in the Media for Informed Policy Making
Source: This use-case was summarized by Richard Eckart de Castilho based on information provided on the ValuesML website and the annotation guides published by the project..
The ValuesML project, part of a JRC initiative, aims to enhance policymakers’ understanding of citizens’ values in relation to policy issues. By using machine learning to analyze values expressed in news media and political manifestos, the project connects these values to policies, events, and political contexts. In collaboration with the JRC’s Text Mining & Analytics Competence Centre, ValuesML leverages data from the Europe Media Monitor (EMM) and political manifestos to create a gold-standard dataset for training advanced values detection algorithms.
A key aspect of the project involves annotating documents with personal values and their attainment levels using the INCEpTION annotation tool. This process has resulted in over 2,500 annotated documents across nine languages, involving around 70 scholars. The annotated corpus will help train algorithms to better understand how values influence public opinion and policy.
Detailed informtation on the annotation schema and the INCEpTION configuration can be found in the annotation guidelines. To shortly summarize, the most important functionalities of INCEpTION used in the project were:
- Team-based Annotation: Annotators could annotate texts by highlighting spans, selecting from predefined values, and specifying attainment levels. Key bindings were used to improve annotation efficiency.
- Workload management: As annotators marked a completely annotated document as finished, project managers could track the progress of the project.
- Curation: Curators could merge and resolve annotations from different annotators, ensuring consistent and high-quality data. The system provided clear visual markers (color-coded) for agreement or discrepancies between annotators.
