Demaq/Transscale: Automated distribution and scalability for declarative applications

摘要

The goal of the Demaq/TransScale system is to automate the distribution of applications to multiple hosts. Today, in order to create highly scalable architectures, developers have to manually restructure the application logic, creating balanced application fragments and data partitions to distribute across the available host machines. Our approach automates much of this manual work. We show how a novel, messaging-based programming model allows to implement distribution as a source-level transformation that turns a non-distributed application specification into a set of programs that can be executed on the various machines of a cluster. The challenge is the identification of application fragments that can be executed independently of each other, such that each fragment can be executed using only local transactions. Further, our method allows developers to increase scalability (i.e. the number of possible fragments) by choosing from a library of automatically applicable rewrites. As a running example, and for the reported experimental results, we use TPC-App as a benchmark application.