Dynamic Load Balancing for I/O-Intensive Tasks on Heterogeneous Clusters. Xiao Qin, Hong Jiang, Yifeng Zhu, and David R. Swanson. In Proceedings of the 10th International Conference on High Performance Computing (HiPC 2003), Dec.17-20, 2003, Hyderabad, India. (An extended version will be available as a journal paper accepted by Journal of Cluster Computing)

Abstract
Since I/O-intensive tasks running on a heterogeneous cluster need a highly effective usage of global I/O resources, previous CPU- or memory-centric load balancing schemes suffer significant performance drop under I/O-intensive workload due to the imbalance of I/O load. To solve this problem, we develop two I/O-aware load-balancing schemes, which consider system heterogeneity and migrate more I/O-intensive tasks from a node with high I/O utilization to those with low I/O utilization. If the workload is memory-intensive in nature, the new method applies a memory-based load balancing policy to assign the tasks. Likewise, when the workload becomes CPU-intensive, our scheme leverages a CPU-based policy as an efficient means to balance the system load. In doing so, the proposed approach maintains the same level of performance as the existing schemes when I/O load is low or well balanced. Results from a trace-driven simulation study show that, when a workload is I/O-intensive, the proposed schemes improve the performance with respect to mean slowdown over the existing schemes by up to a factor of 8. In addition, the slowdowns of almost all the policies increase consistently with the system heterogeneity.
BibTeX Entry
  @inproceedings{yzhu:hipc03,
author = "Xiao Qin and Hong Jiang and Yifeng Zhu and David Swanson",
title = "{Dynamic} Load Balancing for {I/O}-Intensive Tasks on Heterogeneous Clusters",
booktitle = " Proceedings of the 10th International Conference on High Performance Computing ({HiPC} 2003)",
location = "Hyderabad, India",
year = "2003",
month = dec
}
Full Paper
 
Last modified on October 16, 2003