D4.4 API and runtime (complete with documentation and basic unit testing) for IO employing fast local storage

Abstract

This deliverable presents the activities performed on the ExaQUte project task 4.5 Development of interface to fast local storage. The activities have been focused in two aspects: reduction of the storage space used by applications and design and implementation of an interface that optimizes the use of fast local storage by MPI simulations involved in the project applications.

In the rst case, for one of the environments involved in the project (PyCOMPSs) the default behavior is to keep all intermediate les until the end of the execution, in case these les are reused later by any additional task. In the case of the other environment (HyperLoom), all les are deleted by default. To unify these two behaviours, the calls \delete object" and \detele le"have been added to the API and a ag \keep" that can be set to true to keep the les and objects that maybe needed later on. We are reporting results on the optimization of the storage needed by a small case of the project application that reduces the storage needed from 25GB to 350MB.

The second focus has been on the de nition of an interface that enables the optimization of the use of local storage disk. This optimization focuses on MPI simulations that may be executed across multiple nodes. The added annotation enables to de ne access patters of the processes in the MPI simulations, with the objective of giving hints to the runtime of where to allocate the di erent MPI processes and reduce the data transfers, as well as the storage usage.

Full Document