Transient Compute - ARC as Cloud Front-End

Overview:

For research there is a new type of player on the HPC provider market. In addition to commercial providers now also national computing centres, national research and education networks (NREN) and local IT departments expose their resources via cloud platforms. Funding agencies are adapting to the situation with explicit regulations accepting requests for cloud usage costs and outsourcing. The research is thus challenged with the technological and conceptual change needed to benefit from this new situation. We report on the Swiss experience, in particular on how compute clusters are created in no time for single applications and how cloud providers are plugged into traditional grids like WLCG with almost no effort. Two tools, the long existing Advanced Resource Connector (ARC) from the Nordugrid collaboration and the newer elasticluster package from the University of Zurich, are used to achieve this. The extremely reduced time from demand formulation to running solution, from several months to some hours, changes the perception of compute clusters as a persistent infrastructure to a service. The compute element becomes a transient  concept. Obstacles towards this change arise due to the highly subsidised provision of traditional compute resources and the full cost recovery provision of the cloud resources. We discuss a real Swiss case in terms of prices. The presentation, in addition to being a report, provides a recipe on how to create compute clusters on demand and alternatively plugging them into traditional grids.

Target Audience:

Research IT supporters, researchers and projects in need of compute clusters on clouds, compute resource providers.

Benefits for Audience:

How to create clusters on demand, how to plug on-demand clusters into traditional grids

Topic 1: Challenges facing users and service providers