DEWE: A Framework for Distributed Elastic Scientific Workflow Execution

Leslie, L.M., Sato, C., Lee, Y.C., Jiang, Q. and Zomaya, A.Y.

As cloud computing is increasingly adopted, the open, on-demand nature of public clouds makes explicit consideration of the underlying environment highly advantageous in terms of decreasing cost and increasing elasticity. In this paper, we address the exploitation of such cloud capabilities and present DEWE, a framework for the distributed, elastic execution of scientific workflows. DEWE is designed to be easily extensible and customizable, and to provide a simple interface to automate deployment and elasticity in public clouds. Using Montage, an astronomical image mosaic engine, as a case study, and Amazon Web Services (AWS) as the cloud environment, we demonstrate the benefits DEWE can provide to scientists seeking to design job scheduling, data management, and resource allocation strategies with potentially unlimited on-demand resources at hand. Further, DEWE’s visualization tool much leverages the analysis and evaluation of those strategies.

Cite as: Leslie, L.M., Sato, C., Lee, Y.C., Jiang, Q. and Zomaya, A.Y. (2015). DEWE: A Framework for Distributed Elastic Scientific Workflow Execution. In Proc. 13th Australasian Symposium on Parallel and Distributed Computing (AusPDC 2015) Sydney, Australia. CRPIT, 163. Javadi, B. and Garg, S.K. Eds., ACS. 3-10

(from crpit.com) (local if available)