From Beamline to Smartphone: Using Matrix to monitor experiments at DESY

How the ROCK-IT project uses Matrix to keep researchers informed during long-running experiments.

When experiments run for 17 hours, who keeps an eye on them?

Experiments at synchrotron beamlines can run for many hours, sometimes even overnight. During this time, unexpected events may occur that require a scientist’s attention. However, continuously monitoring an experiment is not always practical nor desirable.

This was one of the challenges we faced in the ROCK-IT project at DESY. During a demonstrator experiment involving catalysis at the P65 beamline, measurements lasted approximately 17 hours. According to the beamline staff, experiments of this duration are quite common.

The consequences of unexpected events can be significant. During an earlier experiment at P65, H₂O levels rose unexpectedly overnight, compromising the measurements and forcing the team to repeat the experiment. The incident could also have put the equipment at risk.

Our goal was therefore simple: How can we automatically notify beamline users when something unexpected happens, without requiring them to stay at the beamline?

Our solution combines automated data analysis to detect sudden increases or decreases in mass spectrometer signals with a small software component called the Notifier, which connects the experiment’s data processing workflow to Matrix, an open communication protocol used by the Helmholtz Chat Service provided by HIFIS.

By sending alerts to a Matrix room, researchers can receive information about their experiment on their computers or smartphones, wherever they happen to be. This allows them to react quickly instead of discovering problems the next morning.

How the Notifier works

The Notifier acts as a bridge between the alert-generating system and Matrix.

Whenever an unexpected event is detected, the monitoring or data processing workflow can produce an alert described in a JSON document. This document is placed in a queue, where the Notifier retrieves it and publishes the corresponding message to a designated Matrix room.

For our demonstrator, we used ASAP::O as the alert queue because it was already part of the ROCK-IT data processing pipeline. However, the Notifier’s approach is not tied to a particular queue implementation. Other messaging systems, such as Apache Kafka, could serve the same purpose.

This separation is useful: the software detecting unexpected events does not need to know anything about Matrix. It only needs to publish an alert, leaving communication with users to the Notifier.

Organizing notifications with Matrix rooms and spaces

Beamlines serve many different experiments and scientific teams. We therefore wanted notifications to be organized in a way that keeps the information relevant to each group.

Our proposed structure uses:

• One Matrix room per beamtime, containing notifications specific to that experiment.
• One Matrix space per beamline, grouping the rooms belonging to that beamline.

The Notifier can create rooms and spaces, associate rooms with spaces, and invite participants automatically.

Ideally, once a beamtime has been scheduled, the corresponding Matrix room could be created and all relevant participants invited without additional manual work from beamline staff.

This is an important aspect of the design: automation should make life easier for scientists and beamline staff, not introduce another administrative task.

Technical lessons learned

Although sending a message to Matrix is relatively straightforward, integrating Matrix into an automated experimental workflow raised several interesting questions.

Giving the Notifier its own Matrix identity

To interact with Matrix, the Notifier needs an account with permission to create rooms, invite users and send messages. Our initial implementation uses a dedicated Matrix account controlled by the application through an access token and device ID. This allows the Notifier to perform the required operations programmatically.

We also investigated using a Hookshot bot offered through HIFIS. However, our initial experiments suggested that Hookshot would need to be invited to each room first, leaving part of the room creation and setup process to beamline staff.

We therefore continued with a dedicated account for the demonstrator and are now considering a functional account as a more sustainable identity for the service, potentially also facilitating integration with the Helmholtz Matrix infrastructure.

Testing Matrix integration automatically

Another interesting challenge was testing the Notifier in our continuous integration (CI) pipeline. Because the Notifier interacts with a Matrix server, integration tests require an actual server to be available. We addressed this by running Synapse, a Matrix homeserver implementation, in a Docker container as part of the GitLab CI pipeline.

A startup script configures and launches the server inside the CI environment. Once the server is ready, the tests can exercise the Notifier’s Matrix functionality. This setup allows us to test interactions with Matrix automatically, without relying on a permanently running external test server. The configuration is available in the project’s GitLab CI file and the corresponding Synapse startup script.

Making notifications effortless for researchers

One of the main lessons from our first experiments is that technical integration is only part of the challenge. Matrix offers a convenient way to deliver notifications using existing software, including mobile applications. This reduces the need to develop and maintain our own notification infrastructure. However, researchers must have a Matrix account and be familiar with the application. During our P65 demonstrator, the beamline staff were not particularly enthusiastic about having to set up another communication tool, although they considered the benefits worthwhile.

At present, we collect participants’ Matrix IDs manually. In the future, this information could potentially be collected during the beamtime proposal process. This would open the door to a fully automated workflow: participants submit their proposals, the relevant Matrix room is prepared, and they receive notifications during their experiment without any additional setup by beamline staff.

We are also considering whether notifications could be displayed in our web-based data visualization interface, either as an alternative or a complement to Matrix. Reliable notifications on mobile devices would be an important requirement for such an approach.

What’s next?

The current Notifier demonstrates how an existing communication platform can become part of a scientific experiment’s monitoring infrastructure. Our next steps include improving the account setup, reducing manual onboarding, and investigating how notifications could become a standard feature of the beamtime workflow. We would also like to make the software more discoverable, e.g. by registering it in the Helmholtz Research Software Directory.

While we are still evaluating how Matrix fits into the everyday workflows of beamline users, the demonstrator has shown a promising direction: connecting scientific infrastructure to familiar communication tools can help researchers stay informed without having to remain physically present throughout an experiment.

The source code of the Notifier is available in the DESY GitLab repository.