ENTSO-E boosts data collections with PEMMDB platform

ENTSO-E runs about 10 annual data collections for 40 transmission system operators on the PEMMDB platform built by RTE international. The system now covers operators in 36 countries.

PEMMDB is based on OperatorFabric, an open-source coordination system already used by RTE across its eight control centres. RTE international built PEMMDB with workflows embedded directly in that software, rather than through a separate application.

ENTSO-E uses the data for planning studies including ERAA, TYNDP and the Seasonal Outlook. Those studies inform grid investment decisions, supply security assessments and European Union energy funding policy.

Before PEMMDB, ENTSO-E relied on a process built around Excel templates, shared folders and Python scripts. Staff had to prepare templates manually, duplicate them across country folders and run checks only at the end of each collection cycle. The workflow consumed months of preparation and post-processing time and relied on external consultants.

Under that approach, users could overwrite each other's files, and many formatting errors were found only after submissions were complete. ENTSO-E staff often had to correct files directly, creating further complications when updated versions were uploaded later.

How it works

PEMMDB introduces single sign-on and a live status view for each data collection. ENTSO-E can see in real time which transmission system operators have submitted data, which submissions are still outstanding and which entries have triggered anomalies.

Each submission passes through five quality-control checks before it is committed to the database. The checks cover schema validation, completeness, business rules and consistency, with alerts raised before data is accepted.

The platform also includes integrated Elastic and Kibana tools, allowing users to compare current submissions with earlier ones. That gives participating operators another way to review data quality before it enters planning models.

The scale of the system reflects the breadth of ENTSO-E's data requirements. Demand forecasts, supply scenarios, network parameters and generation capacity data must be collected from 40 organisations operating across different systems and time zones.

Broader use

After its initial deployment, ENTSO-E is extending PEMMDB to other projects and use cases. The organisation has also incorporated the application into additional system development studies, including the collection and validation of network models, as part of a broader move away from a legacy tool.

RTE international said the work also feeds back into the underlying open-source software. Developments made for ENTSO-E can therefore be reused by other grid operators and organisations working on similar coordination and data-exchange tasks.

OperatorFabric was originally developed by RTE and is hosted by LF Energy. It is designed as a coordination and notification layer for utility control-room users, sitting alongside operational systems rather than replacing them.

RTE international was selected by ENTSO-E to design and build the platform, working with Aprico Consultants. The project drew on RTE's existing production use of OperatorFabric across its control-centre network.

Gabin Garrot, Project manager at RTE international, described the wider effect of the work on the platform. "Every project we deliver on the platform moves it forward," Garrot said.

He added that the ENTSO-E work has an impact beyond a single customer deployment. "The improvements we make for ENTSO-E return to the open-source community, so what we build for one client strengthens the whole ecosystem," Garrot said.

Aurélien Watare, Director of digital solutions at RTE international, said the PEMMDB project shows how the company is applying open-source software in energy infrastructure. "RTE international embraced open source as early as 2020, and PEMMDB is a concrete demonstration of its value applied to critical energy infrastructure. This approach delivers robustness and reliability, while remaining transposable to other players in the sector and continuously enriching the wider ecosystem," Watare said.