SP-ant: An ant colony optimization based operator scheduler for high performance distributed stream processing on heterogeneous clusters

作者：

Highlights：

• Presented a novel scheduler for high performance distributed data stream processing.

• Applied Ant Colony Optimization for adaptive scheduling on heterogeneous clusters.

• Reduced convergence time by rescheduling only a subset of operators.

• Implemented our SP-Ant scheduler on the standard Apache Storm.

• Reduced system response time by 50% compared to when using the R-Storm scheduler.

摘要

•Presented a novel scheduler for high performance distributed data stream processing.•Applied Ant Colony Optimization for adaptive scheduling on heterogeneous clusters.•Reduced convergence time by rescheduling only a subset of operators.•Implemented our SP-Ant scheduler on the standard Apache Storm.•Reduced system response time by 50% compared to when using the R-Storm scheduler.

论文关键词：Ant colony optimization,Distributed data stream processing,Scheduling,Apache Storm,Heterogeneous clusters

论文评审过程：Received 7 January 2021, Revised 16 November 2021, Accepted 26 November 2021, Available online 3 December 2021, Version of Record 8 December 2021.

论文官网地址：https://doi.org/10.1016/j.eswa.2021.116322