README (Please): A Small File, a Big Help When Working with Research Data
Have you ever downloaded a research dataset, opened the files and realised you had no idea what they contained or how to use them? This is where a README file comes in.
Terabytes Without the Command Line: Onedata Connects Storage with Computing Resources
Research data often move between several storage locations and work environments over the course of a single project: from an instrument to storage, from there to colleagues or computing resources, and finally to a repository. For datasets containing hundreds of gigabytes or even terabytes of data, none of these transfers can be taken for granted. The new Onedata service, introduced at the CESNET Conference 2026, is designed to simplify this process for e-INFRA CZ users. It makes data stored in different locations accessible within a single shared environment and facilitates their transfer to computing resources.
Today, data are generated on specialised instruments, stored in central storage systems and then transferred to computing resources, which may be located in a different city or institution. A single project often involves several teams, while files are continuously added, modified and returned for further processing. Each storage system may use different technology and require a different method of access. Users therefore need to know which tool to choose and how best to transfer large datasets.
“A typical example is a team collecting cryo-electron microscopy data in Brno and processing them on a supercomputer in Ostrava. The files are hundreds of gigabytes in size, new data are added continuously, colleagues need access to them, and sometimes the same data are worked with in both locations. A one-off copy is simply not enough in this situation,” explains Tomáš Svoboda from the Institute of Computer Science at Masaryk University, who is designing the new service. “Users often need to figure out how to move data from data storage to MetaCentrum for computation. Technically, this is possible, but the existing workflows can create unnecessary barriers,” he adds.
One Shared Space for the Whole Team
Onedata allows a research team to create a shared workspace, known as a space. This can be connected to multiple physical storage systems, even if they are located in different places or use different storage technologies. The team sees the data within a single environment, can manage access permissions and, as needed, determine where files should be transferred or replicated. “The data can remain where they were generated or be moved to where they are needed for further processing. Users can still access them through the same interface,” explains Josef Smrčka, an IT administrator at Masaryk University who is also involved in deploying the service.
More experienced users can work with specialised clients and application programming interfaces, but for basic tasks a web interface is sufficient. It presents the data as ordinary files and folders, even when they are physically stored, for example, in S3 object storage. Researchers do not need to use the command line or install and configure additional tools. If the storage systems located near the instrument and the computing resources are connected to the same space, data can be replicated through Onedata directly to where they will be processed, without first downloading them locally and then uploading them again elsewhere.
Onedata also facilitates long-term collaboration between teams that continuously collect and process data. Newly added files become available within the shared space to all authorised users, without the need to share or send them separately each time. The service is not limited to any particular scientific discipline; its usefulness depends primarily on the volume of data and the way researchers work with them. For spreadsheets, office documents or datasets of a few hundred megabytes, standard cloud services are usually sufficient. The benefits of Onedata become apparent when datasets reach hundreds of gigabytes or terabytes and need to be regularly shared or transferred between storage systems, institutions and computing resources. “The software is primarily intended for active data that are continuously generated, modified and shared during the research process. It therefore does not replace repositories designed for the long-term preservation of completed datasets. However, the National Repository Platform project envisages linking the two, making it possible to transfer a completed dataset from Onedata directly to the appropriate repository,” adds Svoboda.
The service will expand the existing portfolio of e-INFRA CZ storage solutions with a tool designed for working with large, active datasets that need to be processed and shared.
How to Get Started
Onedata is available to e-INFRA CZ users, who can first try the system in a test space. Those wishing to deploy it within their own project can contact support to discuss their specific requirements. The team will then create a dedicated space and connect it to one or more of the available storage systems. e-INFRA CZ is involved not only in operating the service, but also in the further development of the system. Together with the developers of Onedata, it is preparing the Onedata Drive application, which will also make the data accessible to Windows users.
