Upgrading the linguistic ORD-ecosystem - UpLORD

In the context of the requirements of Open Science and of FAIR principles, on the one hand, and of that of more challenging data sets (such as, sensitive or with copyright issues), on the other hand, we identified several gaps regarding the current situation in Switzerland and will address them in this ORD project:

(1) the provision and the usability of the individual corpus platforms across Switzerland: their drawback is that they do not satisfy the Interoperability principle,

(2) the status of several linguistic corpora: most of them do not satisfy all four FAIR principles,

(3) the lack of meaningful metadata and of infrastructures to manage a diversity of annotations of linguistic data, this being linked to the Findable and the Reusable principles,

(4) the proper management of sensitive data, informed consent, copyright and intellectual property issues, which are necessary so that the sets of data adhere to the FAIR requirements,

(5) the metadata schema on SWISSU­base which must be adapted and amended to satisfy the Findable principle,

(6) uploading workflows on SWISSUbase,

(7) the quality control and data curation in SWISSUbase to satisfy the Reusable principles,

(8) the setting of ethical standards for best practices, good habits and frames of mind in linguistic data management and collaboration, as well as the lack of training for the target scientific communities to adopt these standards.

Regarding gaps (1) and (2), this project will move forward the implementation of the LiRI Corpus Platform at the national level following the FAIR principles. This will significatively improve discovery, access, integration, usability and reusability of corpora according to FAIR principles, as well as simplify re-formatting, assembling, harmonizing and standardizing the data and metadata.

• The LiRI Corpus Platform LCP is a software system for handling and querying corpora of different kinds: text, audio and video (see the modules catchphrase, soundscript and videoscope.
• Up to now, numerous tests with importing and quering various corpora have been carried out on the test version. The public version will be available summer 2024, when users will be able to query corpora directly from their browser, and upload their own corpora using a command-line interface. 

Regarding gap (3), the current implementation of VIAN-DH is used as a test case for modelling complex annotation schemes that LiRI Corpus Platform has to offer. Data processed within VIAN-DH is complex interactional data consisting of verbal, paraverbal and non-verbal annotation levels. The project will collect further complex annotation models via the CLARIN-CH and NCCR partners as well as via other clients LiRI is already working with. It is then evaluated whether the LCP@LiRI can also map these. This process goes hand in hand with the development of best practices to show researchers how to deal with complex annotations, but also to showcase the resulting opportunities.

• Complex annotation representation is constantly being tackled during LCP implementation process.
• Up to now, data converters from (TEI-)XML, generic XML and CWB format to CoNLL-U+ have been implemented.

Regarding gaps (4)-(6), the already implemented modular linguistic metadata schema of LaRS@SWISSUbase will be specified and adapted to the needs that are not included for the moment. To upgrade the usability of SWISSUbase, it should be also possible to provide a workflow via an API. In addition, easy workflows between LCP and VIAN-DH will be implemented.

• Version control workflows and launch of API for data upload on SWISSUbase are implemented
• Data Service Unit@UZH established as single point of entry for linguistic data to be deposited on SWISSUbaseswissubase.ch
  • Principles of data curation for linguistics
• Webinar on the use LaRS/SWISSUbase services (February 2023), see also: Webinar: The Language Repository of Switzerland, powered by SWISSUbase (youtube.com)

Regarding gap (7), this project will set up national CLARIN-CH working groups whose role will be to develop specific metadata for various types of linguistic data (e.g., sociolinguistics data, experimental psycholinguistics, neurolinguistics data, conversational analysis data, lexicography data, computational data, acquisitional data, multimodal data) that satisfy the FAIR principles. 

• To develop specific metadata, a a mixed approach was adopted: (1) a CLARIN-CH WG on Learner corpora, (2) individual discussions with researchers (e.g. for sign language data) and (3) the reuse of CLARIN CMDI metadata schema.
• Mapping of existing metadata schema on SWISSUbase to CLARIN CMDI profiles is ongoing
• Adapting the metadata schema according to the needs and requirements of the linguistic community:
  • Challenge of balancing individual and more general needs/requirements: one additional focus that has emerged as very important is on functionality of SWISSUbase as a catalogue for data and also for services (linkage functionality only).
• A survey to collect information about recommended/standard data formats is currently in preparation. After an initial pre-test with data experts working at LiRI, SWISSUbase and NCCR Evolving Language (April 2024), the survey is currently circulated in the CLARIN-CH community (May-June 2024). Find the survey here. 

Regarding gap (8), this project will develop showcases, best practices and documentation for data management according for FAIR principles (such as, planning and writing data management plans, using sustainable file formats, setting solid file-naming conventions, finding data storage backup schemes, creating informative metadata and documentation) and will promote ORD practices in the CLARIN-CH, the NCCR  "Evolving Language" and other linguistic communities. Special attention will be given to finding solutions that overcome disciplinary boundaries.

• Existing LaRS@SWISSUbase guides:
  • For Researchers: Prepare your data for deposit in SWISSUbase
  • User guide for Linguistics studies and data
  • Linguistics Metadata guide
• Documentation on CLARIN-CH Documentation platform 

  • Research data lifecycle
    Data Management Planning
    Collecting data
    Processing data
    Sharing data
    Archiving data
    Reusing data

  • FAIR data

  • Copyright

  • Licenses

  • Data protection

  • Data access and security

  • Metadata standards

  • Standard data formats

• A national event, CLARIN-CH Day, on the theme ORD: Challenges and Opportunities is planned for September 9, 2024, with an open call for data pitches, and exchanges between researchers and experts.
• 9 webinars on the topics of data protection (legal and technical perspectives), data anonymisation, intellectual property rights in linguistic data, legal aspects of sharing and reusing linguistic data organised in the frame of the CLARIN-CH WG on Management of Sensitive and Personal data, Ethical and Legal issues for linguistic data

1. Bubenhofer, N., Malits, A., Strebel, S., Gräen, J., Buerli, S., & Grisot, C. (2023, December). Building and consolidating a FAIR-compliant ecosystem of infrastructures. In CLARIN Annual Conference Proceedings (p. 95-99). 
2. Schaber, J., Graën, J., McDonald, D., Mustac, I., Rajovic, N., Schneider, G., ... & Kontino, T. (2023, October). The LiRI Corpus Platform. In CLARIN annual conference proceedings (pp. 145-149). 
3. Schaber, J., Graën, J., Mustač, I., Rajović, N., Schneider, G., Zehr, J., & Bubenhofer, N. Swissdox@ LiRI–a large database of media articles made accessible to researchers. CLARIN annual conference proceedings (pp. 111-115). 

4. Poster at Open Access Week 2023 at UZH 

5. Presentation at 2023 SWISSUbase Annual event at UZH (November 2023)