XChange: Coupling parallel applications in a dynamic environment

TitleXChange: Coupling parallel applications in a dynamic environment
Publication TypeConference Paper
Year of Publication2004
AuthorsH Abbasi, M Wolf, K Schwan, G Eisenhauer, and A Hilton
Conference NameProceedings - IEEE International Conference on Cluster Computing, ICCC
Date Published01/2004

Modern computational science applications are becoming increasingly multi-disciplinary, involving widely distributed research teams and their underlying computational platforms. A common problem for the grid applications used in these environments is the necessity to couple multiple, parallel subsystems, with examples ranging from data exchanges between cooperating, linked parallel programs, to concurrent data streaming to distributed storage engines. This paper presents the XChange
mxn middleware infrastructure for coupling componentized distributed applications. XChange
mxn implements the basic functionality of well-known services like the CCA Forum's MxN project, by providing efficient data redistribution across parallel application components. Beyond such basic functionality, however, XChange
mxn also addresses two of the problems faced by wide area scientific collaborations, which are (1) the need to deal with dynamic application/ component behaviors, such as dynamic arrivals and departures due to the availability of additional resources, and (2) the need to 'match ' data formats across disparate application components and research teams. In response to these needs, XChange
mxn uses an anonymous publish/subscribe model for linking interacting components, and the data being exchanged is dynamically specialized and transformed to match end point requirements. The pub/sub paradigm makes it easy to deal with dynamic component arrivals and departures. Dynamic data trans-formation enables the 'in flight' correction of data or needs mismatches for cooperating components. This paper describes the design and implementation of XChange
mxn, and it evaluates its implementation compared to those of less flexible transports like MPI. It also highlights the utility of XChange
mxn's 'in flight' data specialization, by applying it to the SmartPointer parallel data visualization environment developed at our institution. Interestingly, using XChange
mxn did not significantly affect performance but led to a reduction in the size of the code base. © 2004 IEEE.