Work modules

Module 5. Integration and evaluation

This module includes all activities associated with the integration of components (algorithms, techniques and tools) for the development of software applications that serve as technology demonstrators.

Milestones

  • Implementation of software solutions in different formats of the techniques, algorithms and resources generated by the project.
  • Design of technology demonstrators for the selected scenarios to measure the usability and functionality of the softwre solutions generated by the project.

Deliverables

  • Web platform to provide easy access to the technologies obtained.
  • APIs (RESTful) of the technologies obtained, facilitating online integration with third-party applications.
  • Installation of local software packages.
  • Code repository with the implemented techniques that can be reused and replicated by the entire scientific community.
  • Interactive tutorials on the use of the different techniques.

Task 5.1 Module development

The construction of technology demonstrators requires a previous step of development and integration of modules from libraries and own or external resources. Technologies such as programming languages (Python, for example), libraries (NLTK, CoreNLP, Freeling), as well as advanced ML solutions, are necessary for these specific developments. The use of diverse, heterogeneous and multimodal information sources requires the definition of a common data model and the development of a microservices layer (API), which will allow the integration of the front ends of the technology demonstrators. This task will include the integration and development of the necessary modules for the launching of experiments and their evaluation, and for the subsequent integration in demonstrators.

Task 5.2 Construction of demonstrators for each scenario

Determining the usability and performance of the implemented techniques and tools requires the construction of scenario-specific experimental platforms. These platforms will integrate the modules generated from the previous task. To carry out this task with the aim of generating solutions that are transferred to society, i.e., easily reusable and accessible resources will be developed, such as web platforms, APIS, repositories and software packages or tutorials. The dissemination and distribution of scientific knowledge in this way will allow replicability, reusability and full accessibility to all the tools and techniques developed.

Task 5.3 Experimentation and analysis of results

In this task, different experiments will be carried out for each of the specified scenarios. In addition, for each of them the existing evaluation metrics will be analyzed and, if necessary, new metrics will be defined, which will allow us to measure the resources and techniques generated and/or used. To this end, a series of experiments will be developed to perform both intrinsic and extrinsic evaluation. On the other hand, the results obtained during the execution of experiments will be subject to an indepth analysis and discussions by all the researchers of the project. Quantitative and statistical metrics will be extracted that will allow the interpretation of these results in relation to the different configurations applied in the different scientific proposals developed for the project. Through the evaluation and analysis of results it is possible to study, adjust and re-configure the different research proposals to obtain the best configurations of the algorithms proposed during the execution of the project so that the most efficient proposals can be integrated into the technology demonstrators described in task T5.1.

Tarea 5.4 Technological permeability

To ensure progress in the use of language technologies in the final solutions, it is necessary to deepen mechanisms that facilitate the integration of tools and resources generated in integrated platforms and products. In recent years, the scientific community has made a significant commitment to transfer strategies based on repositories and libraries that enable rapid adoption of the innovations created. Projects such as GitHub, Huggingface, Papers with Code or deployments in PIP repositories have become de facto standards that allow the use, by third parties, of highly complex solutions that are at the frontiers of the state of the art. This is a key element that isincreasing the impact of NLP technologies in society. The scientific community has adopted permeability as a further task to increase the impact of research, by making code available in formats and versions that facilitate its use. It means providing experimental developments with the necessary quality for rapid deployment in other third-party execution environments. This increases the reproducibility of the experiments and a wider dissemination of the proposed methods and techniques. Thanks to the simplification of the work to be able to execute a system resulting from a research work, other researchers can apply the solution to new problems or build novel solutions that integrate it.