The SHIWA project (Sharing Interoperable Workflows for large-scale scientific simulations on Available DCIs) within grid computing was a project led by the LPDS (Laboratory of Parallel and Distributed Systems) of MTA Computer and Automation Research Institute. The project coordinator was Prof. Dr. Peter Kacsuk. It started on 1 July 2010 and lasted two years. SHIWA (project number 261585) was supported by a grant from the European Commission's FP7 INFRASTRUCTURES-2010-2 call under grant agreement n°261585.
The SHIWA project developed and deployed the SHIWA Simulation Platform (SSP) to enable infrastructure and workflow interoperability at two levels: coarse-grained interoperability, referring to the nesting of different workflow systems in order to achieve execution frameworks interoperability; and fine-grained interoperability, referring to the transformation workflow representations in order to achieve workflows migration from one system to another.
After the project ended the SHIWA Technology was overtaken by the ER-flow support action project to provide sustainability and to extend the user community base.
Background and motivations
Scientists of all disciplines have invested tremendous effort in the exploitation of Distributed Computing Infrastructures (DCIs) for their ability to support compute-intensive in-silico experiments and virtual organisations. Many DCIs with large user communities have emerged during the last decade, such as the Distributed European Infrastructure for Supercomputing Applications (DEISA) [Niederberger and Mextorf 2005], EGEE Grid (Enabling Grids for e-Science) [EGEE n.d.], the German D-Grid initiative (D-Grid) [Gentzsch 2006], UK National Grid Service (NGS) [NGS n.d.] and the North American TeraGrid (TG) [TeraGrid n.d.]. They are based on different middleware stacks that provide an abstraction layer between computer resources and applications. For example, NGS and TeraGrid are built on the Globus Toolkit [Foster 2006], EGEE on gLite [gLite n.d.], DEISA relies on both the Globus Toolkit and Unicore [Erwin and Snelling 2002], while D-Grid is executed under gLite, the Globus Toolkit and Unicore. In Europe, this momentum is climaxing in 2010 with the emergence of the European Grid Initiative (EGI) that will federate all major European organisations related to distributed-computing and National Grid Initiatives (NGIs). In its effort to create the next generation of pan-Europe DCI, EGI will face unprecedented challenges related to the heterogeneity of national grid infrastructures, resources and operating middleware. Production DCIs are commonly built on a large number of components, such as data resources, metadata catalogues, authentication and authorisation methods, and software repositories. Managing the execution of applications on DCIs is consequently a complex task. Moreover, solutions developed for one particular Grid are difficult to port to other infrastructures. In order to shield this complexity from researchers and to facilitate the design of in-silico experiments, workflow systems are widely used as a virtualisation layer on top of the underlying infrastructures. They have become essential to integrate expertise about both the application (user domain) and the DCI (infrastructure domain) in order to optimise and support research of the scientific computing community. In the current multi-DCI panorama, users need to access different infrastructures in order to enlarge and widen the variety of resources usable, as well as share and reuse domain specific resources. Interoperability among DCIs is hardly achieved at the middleware level though. SHIWA considers the EGI production infrastructure a major DCI of great interest for the European Scientists to design and simulate experiments in-silico. It directly addresses the challenges related to (i) scientific experiments design through simulation workflows description and (ii) middleware heterogeneities encountered among the many existing DCIs through workflow interoperability techniques.