Dynamic Deployment of Transatlantic Computational Testbeds via the Infrastructure Manager

Germán Moltó, Miguel Caballer, Kate Keahey, Estíbaliz Parcero, and Amanda Calatrava. Dynamic Deployment of Transatlantic Computational Testbeds via the Infrastructure Manager. In EGI2025, 2025.

Download

[HTML] 

Abstract

The Infrastructure Manager (IM) is an open-source Cloud orchestrator to deploy complex virtual applicationarchitectures (defined in TOSCA) on multiple Cloud back-ends (AWS, Azure, OpenStack, etc.), featuring a web-based UI (IM Dashboard) a CLI and a REST API. The IM is being used in production as the orchestrator in theEGI Federated Cloud, the largest distributed computing platform in Europe, composed of OpenStack sites. Itis also the Application Management Layer / Deployment service of the EOSC EU Node, a public procurementaction from the European Commission to build the EOSC (European Open Science Cloud), a platform to createa web of FAIR data and services for science in Europe.There is a lack of transatlantic computational testbeds to foster science across borders on computing-relatedcollaborations between US and EU institutions. To this aim, we decided to integrate the IM with ChameleonCloud, a configurable experimental environment for large-scale edge-to-cloud research managed by ArgonneNational Lab (ANL) and including Cloud-based resources from institutions such as the University of Chicago(UC) and the Texas Advanced Computing Center (TACC). The goal is to dynamically deploy TOSCA-basedapplication architectures on virtualized computing resources that span across large-scale distributed comput-ing infrastructures such as EGI Cloud Compute and Chameleon Cloud. Funding was secured to do a researchexchange of a UPV member at Argonne National Lab during April 2025.This contribution summarizes the main integration activities and lessons learned, illustrating the feasibilityof deploying complex application architectures via Infrastructure as Code approaches, even across multipleorganizational domains via the federating capabilities offered by the Infrastructure Manager.This exchange was funded by the DISCOVER-US project, financed by the European Union’s Horizon Europeresearch and innovation funding programme under grant agreement number 101135064. Results presentedin this paper were obtained using the Chameleon testbed supported by the National Science Foundation. GMacknowledges Grant PID2020-113126RB-I00 funded by MICIU/AEI/10.13039/501100011033. This work wassupported by the project AI4EOSC‘‘Artificial Intelligence for the European Open Science Cloud’’that hasreceived funding from the European Union’s Horizon Europe Research and Innovation Programme underGrant 101058593. Also, the project iMagine‘‘AI-based image data analysis tools for aquatic research’’thathas received funding from the European Union’s Horizon Europe Research and Innovation Programme underGrant 101058625

BibTeX Entry

@inproceedings{Molto2025ddt,
   abstract = {The Infrastructure Manager (IM) is an open-source Cloud orchestrator to deploy complex virtual application
architectures (defined in TOSCA) on multiple Cloud back-ends (AWS, Azure, OpenStack, etc.), featuring a web-
based UI (IM Dashboard) a CLI and a REST API. The IM is being used in production as the orchestrator in the
EGI Federated Cloud, the largest distributed computing platform in Europe, composed of OpenStack sites. It
is also the Application Management Layer / Deployment service of the EOSC EU Node, a public procurement
action from the European Commission to build the EOSC (European Open Science Cloud), a platform to create
a web of FAIR data and services for science in Europe.
There is a lack of transatlantic computational testbeds to foster science across borders on computing-related
collaborations between US and EU institutions. To this aim, we decided to integrate the IM with Chameleon
Cloud, a configurable experimental environment for large-scale edge-to-cloud research managed by Argonne
National Lab (ANL) and including Cloud-based resources from institutions such as the University of Chicago
(UC) and the Texas Advanced Computing Center (TACC). The goal is to dynamically deploy TOSCA-based
application architectures on virtualized computing resources that span across large-scale distributed comput-
ing infrastructures such as EGI Cloud Compute and Chameleon Cloud. Funding was secured to do a research
exchange of a UPV member at Argonne National Lab during April 2025.
This contribution summarizes the main integration activities and lessons learned, illustrating the feasibility
of deploying complex application architectures via Infrastructure as Code approaches, even across multiple
organizational domains via the federating capabilities offered by the Infrastructure Manager.
This exchange was funded by the DISCOVER-US project, financed by the European Union’s Horizon Europe
research and innovation funding programme under grant agreement number 101135064. Results presented
in this paper were obtained using the Chameleon testbed supported by the National Science Foundation. GM
acknowledges Grant PID2020-113126RB-I00 funded by MICIU/AEI/10.13039/501100011033. This work was
supported by the project AI4EOSC‘‘Artificial Intelligence for the European Open Science Cloud’’that has
received funding from the European Union’s Horizon Europe Research and Innovation Programme under
Grant 101058593. Also, the project iMagine‘‘AI-based image data analysis tools for aquatic research’’that
has received funding from the European Union’s Horizon Europe Research and Innovation Programme under
Grant 101058625},
   author = {Germán Moltó and Miguel Caballer and Kate Keahey and Estíbaliz Parcero and Amanda Calatrava},
   booktitle = {EGI2025},
   title = {Dynamic Deployment of Transatlantic Computational Testbeds via the Infrastructure Manager},
   url = {https://indico.egi.eu/event/6638/contributions/20436/},
   year = {2025}
}