Executable Infrastructure for Africa and Arabia

E-Infrastructure has, over the years, proven its worth in enabling scientific collaboration, even at regional and global scales. The adoption of common platforms such as HPC, data management, etc has made
sharing of scientific data, applications and research outputs easier and is accelerating scientific investigation, especially in regions where these were previously unavailable. This is particularly important in Africa, and areas of the Arab-speaking world, which is still at a disadvantage in terms of infrastructure. The Africa-Arabia ROC (AAROC) acts as a coordination point, first for grid infrastructures, but has been expanded to more general collaboration infrastructure services such as science gateways, federated identity providers, document and data repositories, etc. This expansion of services offering to ever-more demanding research communities places unreasonable strain on a fully-distributed model, where every site administrator is expected to understand and operate these new services. This is particularly true in the African and Arabian regions, where knowledge networks are sparse.

This contribution describes a development and deployment philosophy which adopts a “DevOps” paradigm which aims to encode models of services using application and service orchestration tools, heavy use of continuous integration and delivery pipelines. Site and service configuration has been encoded abstract representations, providing a reproducible models of services, rather than prescriptive guides. These can be customised as desired on a per-site or per-instance basis, providing a good level of flexibility. Most importantly, this model is executable, meaning that any number of sites and services can be effectively deployed remotely, by a core team. Continuous integration is done at every commit of code, by executing the playbooks on a cloud-based development site, which provides transparency to the remote site administrators.

The adoption of this methodology helps to solve the problems of sustainably maintaining service configuration, improving communication between site operations and service developers, ensuring the proper
state of services, and verifying the state of deployment. Some of the main benefits of this approach are speeding up the deployment of new services, reliably applying updates and recovering from disaster. In this contribution, we show that the “traditional” HPC and grid service deployment can be reproduced and improved, by adopting a more modern operations stack. We also highlight how this has helped to deploy advanced services- federated identity infrastructure, science gateways, application repositories, cloud sites, and Open Access repositories throughout the region in short time, and how it is now playing a crucial role in the strengthening of technical and scientific communities of practice in the region.