HIFIS Conference
The first conference on HIFIS took place at DESY Hamburg.
The first conference on HIFIS took place at DESY Hamburg.
The platform software.hifis.net (meanwhile moved to hifis.net) is launched as the future home for software development services within Helmholtz and HIFIS. The static site is built with Jekyll. Its sources are available on GitLab. Feel free to build upon it.
In preparation of providing Helmholtz-wide training events 12 Carpentries instructors are trained as part of the membership of the HZDR and GFZ in the Carpentries.
The first Helmholtz-wide training events are conducted. Read the announcement post for information on how the initial course portfolio will look like.
The first steps, progress and ongoing works of HIFIS Cloud, Backbone, and Software Clusters have been reported in the first annual report of HIFIS. This report has been improved and approved by the ICT Federation Board and further served as a basis for feedback by the Scientific Advisory Board in April.
The HIFIS Scientific Advisory Board gathered in a video conference to discuss the progress of HIFIS and provide advice. Comments and suggestions were compiled in a comprehensive report.
From June 2019 to May 2020, a service survey throughout all 19 Helmholtz centres has been conducted. The target of this survey was to determine which services are desired to be offered as services in HIFIS. A second target was to identify which centres are willing to act as service providers for the services to be offered in the Helmholtz Cloud.
In preparation of the launch of the Consulting Services the HIFIS Software helpdesk has been launched basing on Zammad. Therefore, a reusable Ansible role has been implemented.
The blueprint for the new official Helmholtz-wide Authentication and Authorization Infrastructure (Helmholtz AAI), derived from former HDF AAI, is online and available for services and identity providers. It is drafted by FZJ and KIT, and is compatible to the AARC blueprint.
HIFIS fostered a service agreement with the German National Research and Education Network (DFN) on operating a a Helmholtz-wide Virtual Private Network (Helmholtz Backbone), covering the majority of Helmholtz centres. By September, the contract is expected to be signed from the participating partners.
A common contact point for all queries and support requests shall be introduced. This will be realized using a ticketing system, allowing to distribute tasks related to the different HIFIS clusters, especially Cloud Services and Software Services.
HIFIS Software offers free-of-charge consulting as a service to research groups within the Helmholtz umbrella. We can help you deal with specific licensing issues, migrating code between different environments and languages, setting up new projects, and other problems that you need to solve.
After multiple iterations, the first set of selected federated services, forming the initial Helmholtz Cloud Service Portfolio, will be announced and started to be integrated. This set will be continuously enhanced and the performance of the integrated services will be monitored.
A meeting of all HIFIS members is planned to foster collaboration between the HIFIS clusters. Non-HIFIS Helmholtz-centres as well as all incubator platforms shall be incorporated.
The future software management platform will be based on reusable Ansible recipes. This allows other research centers to reuse the implemented solution. The open-source recipes are publicly available on gitlab.com/hifis/ansible.
Having finished the list of the initial service portfolio, the integration of the services will start. At first, the exact details of each service and its tasks for service integration will be examined. This goes hand in hand with the first integration steps. It will also result in a specific roadmap to be established by the end of the year.
The software management platform is planned to be integrated into the Helmholtz Cloud as soon as the technical platform is available. Continuous Integration will be part of the software management platform from the very beginning.
One important element of the software management platform is a solution for continuous integration (CI). Reusable Ansible recipes for a GitLab CI infrastructure will be provided. The Ansible role is developed here.
First Virtual Private Network (Helmholtz Backbone) connections have been set up and are in productive use.
We are happy to announce that the group of the 10 most mature services has reached a maturity level of more than 80% - more than sufficient for them to be integrated into the Cloud Portal in the coming weeks. This top group includes services like GitLab (HZDR, FZJ), Nextcloud (HZB, KIT), Mattermost (HZDR), Rocket.Chat (FZJ) but also OpenStack (FZJ, KIT), JupyterHub (FZJ), Zammad (HZDR) and B2Share (FZJ).
Find more information of Initial Service Portfolio here.
The community work package publishes further details for the Helmholtz Hacky Hour for the first half of 2021 on the events page.
The GitLab service will be deployed to a new Ansible-managed infrastructure providing an enhanced maintainability and scalability. Access to this Helmholtz-wide GitLab will be granted to all people affiliated with the Helmholtz Association and partners to foster cross-center collaboration. If users encounter any issues, they can report them via the HIFIS helpdesk.
The education and training work package publishes the offered training events for the first half of 2021 on </events.html>.
After the ramp-up phase during the first year, the initial Cloud, Backbone and Software Services are expected to be in production by early 2021. We will report on the integration progress and impact of these services, especially with respect to user acceptance and added value for the scientific community.
The Helmholtz Cloud Portal allows harmonized access to all Helmholtz Cloud services. It contains all necessary information to access the available services. The alpha version of the portal is planned to be available by end 2020 and the publicly accessible beta version is planned for February 2021.
While the integration of the initial service portfolio will continue, the evaluation of further services will start. This process will presumably involve the review of postponed services from the initial service selection as well as service selection criteria, focusing on service usage and optimal user experience.
Starting data collection from Helmholtz employees about their used IT-related workflows and software tools. The data will be analysed to further improve our services. Results will be published later in the year.
HIFIS Community evaluates the reusability of the Netherlands RSD as RSD for all Helmholtz centers.
The education and training work package publishes the offered training events for the second half of 2021 on </events.html>.
Deprovisioning of users will be supported by the Helmholtz AAI. This includes a framework to query deprovisioning information from IdPs to allow automatic deprovisioning in Helmholtz AAI. In case of failure, user shall be contacted to confirm their account manually.
The success of the services is essentially reflected in their use and application. From the middle of the year, key figures on the use of the individual services are therefore determined and collected.
The automatic deprovisioning of users is documented and technically implemented for at least one service and at least one service using centre.
The HIFIS data transfer service will be improved such that any combination of interested centres can participate by only needing to install a lightweight client that can manage third party copy.
Developed concept and alpha implementation for automated communication with services, for example using Local Agent or via Cloud Portal. This requires cooperation with the Service Integration working group of the Cloud Cluster.
Services, that were postponed during the initial service selection, as well as newly applied services were transferred into the onboarding process for new Helmholtz Cloud services.
We consider Continuous Integration (CI) an important aspect in a modern software engineering workflow. Easy-to-use CI resources should be available for all Helmholtz scientists. Beside the Helmholtz-wide GitLab many Helmholtz centers run their own GitLab instances on dedicated infrastructure in their center. In order to bundle resources and enable research software projects to easily use CI for their projects, CI will be offered as a service. In this milestone, we will set up CI as a Service and test it first with few pilot centers, in order to make it available to all Helmholtz communities.
Please note: The milestone was originally set for the end of the second quarter. Due to the underlying compute resources not being available in time, this milestone needed to be rescheduled to the third quarter of 2021.
The integration of several services of the initial service portfolio is completed and they can be used according to their description and service level. Therefore, these services immediately become fully integrated services when the legal framework is signed.
Data collection and primary analysis of the HIFIS survey 2021 has been finished. First results will be presented in respective meetings (e.g., HIFIS conference, Incubator workshop).
The production version of the Helmholtz Cloud Portal will be made available. The new search function and filter options help to find a suitable service. The availability of the service is tested automatically and is visible in the service card. The service description contains all main information such as support, service level and limitations. User comments are welcome in the new free text field and will help to continuously improve the usability of the portal.
The Process Framework for Helmholtz Cloud Service Portfolio is published in version 1.0. It focuses on the explanation of the processes regarding the Service Portfolio Management for Helmholtz Cloud, giving an overview of which processes exist, how they interconnect, which roles are involved in each process and what is included in each process step.
Obtained deprovisioning information shall be provided to services. This requires cooperation with the Service Integration working group of the Cloud Cluster.
The legal framework describes the necessary coordination and responsibilities so that the Helmholtz Cloud can perform its task for and within the Helmholtz community. It covers the relevant aspects, such as data sovereignty, intended use, GDPR or IT security.
At least one of the prominent services (such as Nextcloud, Open Stack or Jupyter) or a combination of those is extended to a meta service: the meta service, which is provided by several Helmholtz Centers, is working as a federation or cooperation instead of separated local solutions.
Allowing scientists or reviewers to easily interact with computational environments can be considered a key aspect for reproducible and easy-to-use research software. Integrating tools like Binder or ShinyR into the Helmholtz-wide GitLab will allow scientists to easily make their research software product available to others without the need for installing and configuring tools locally. In this milestone, a concept will be elaborated on how this integration may be realized.
Publish results of HIFIS survey 2021.
The primary goal is to check whether the services in and the processes around the Service Portfolio continue to fulfill defined requirements. Consequently, the review may lead to changes in the Service Portfolio itself as well as the Process Framework. The results will enter the annual HIFIS report and further Service Portfolio Maintenance.
In Cloud Portal, logged-in users shall be able to have a specific view on the services they are allowed to use, as well as service usage conditions. Also, a user can see and manage their relevant service notifications.
(This work package was postponed from September 2021 to January 2022, as no user-specific information will be available by then.)
The education and training work package publishes the offered training events for the first half of 2022 on events.hifis.net.
Service Providers can now provide all service information asked for during the Onboarding process in Plony, our data management system based on Plone.
The Assembly of Members of the Helmholtz Association have received a draft for the definition of technical, administrative and data protection regulations to simplify the use of the Helmholtz Cloud and its Services, and to ensure that applicable conditions are met.
An initial course on using GitLab CI will be held. At this time also the initial course material will be published.
Improve the consulting handbook in all aspects, based on past experience. and publish it.
The education and training work package publishes the offered training events for the second half of 2022 on events.hifis.net.
Virtual Organisations (VOs) have been introduced in Helmholtz AAI during 2020. Requests to create new top-level VOs are currently being processed on end user request, using the HIFIS Helpdesk and considerable manual interaction for VO creation and documentation. In 2022, a transparent process shall be set up and implemented to allow VO handling in larger scales than so far.
The basis for the software catalogue will be the Research Software Directory developed at Netherlands eScience Center. In close collaboration with the original maintainers, an adapted version for Helmholtz will be developed. The development of the HIFIS Research Software Directory will be coordinated at GFZ in collaboration with DLR and UFZ, and aided by the HIFIS Software Technology work package. In this milestone a basic prototype with the initial functionality will be deployed, marking the beginning of the test phase with selected test users.
Until summer, HIFIS will give all stakeholders the opportunity to strengthen our community and to contribute to the further development of HIFIS, especially in light of the upcoming review. As a part of this, HIFIS will organize an All-HIFIS conference in Berlin, possibly as presence meeting, if circumstances allow.
Service categories and keywords categorise the Helmholtz Cloud Services and simplify the selection of a suitable service.
The hifis.net homepage will be relaunched, with improved and aligned design. Based on the progress that HIFIS meanwhile made, having implemented and distributed numerous services to the scientific communities, we will increasingly concentrate on the presentation of scientific use cases.
All steps of the Onboarding Process for new Cloud Services are completely integrated into Plony, the Plone-based database. This enables Service Providers and HIFIS administrators to maintain the Service Catalogue online and collaboratively. The Service Description is automatically transferred to Cloud Portal.
After a bit more than three years of initial phase, the progress and further perspectives of HIFIS will be reviewed. Earlier in 2022, potential perspectives will be proposed and formulated in a pre-review report. End of September, the main review meeting will take place at DESY.
The GitLab CI as a Service offer is planned to be expanded to at least 3 more partners within the Helmholtz association.
During 2021, the framework to automatically request user deprovisioning information from IdPs via attribute query has been principally set up for the central AAI (development) instance and a small set of Helmholtz IdPs. During 2022, the task is to remove remaining inconsistencies and set up the productive deprovisioning process for all IdPs, at least with the already implemented fallback of manual user based confirmation of active accounts.
In close cooperation with the data protection officers of the Helmholtz Association, we have developed a joint controller agreement that covers the Helmholtz Cloud core components that operate the cloud.
For 2022, a set of policy rules, regulating the basic participation conditions of the Helmholtz Backbone network, shall be worked out and put into action for the participants.
So far the Helmholtz Codebase service does not support the Windows operating system for executing jobs. As part of this milestone the Helmholtz Codebase service will support the Windows operating system to run CI jobs.
As part of this milestone it should be possible to
In 2021 the proposed goal of introducing a post-consulting survey for KPI-measurement was achieved by the Consulting work package. Therefore, this year the work package will focus on improving the KPIs by analysing software projects of clients, measure potential improvements and the long-term effects after a HIFIS consultation.
Following the overall very positive HIFIS evaluation results received end of 2022 and in line with our ongoing annual reporting, we will define the short- and mid-term steps to focus on during 2023 and parts of 2024.
During HIFIS evaluation, we were given advise to elaborate on some of the extended future plans of HIFIS, to be put in place from 2025 onwards. These further details are planned to be worked out until fall 2023.