Xen and the Art of Cluster Scheduling
- 1 November 2006
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 12, 4
- https://doi.org/10.1109/vtdc.2006.18
Abstract
In shared use clusters, scheduling systems must schedule both serial and parallel jobs in a fair manner, while at the same time optimizing overall cluster efficiency. Since serial and parallel jobs conflict considerably, scheduling both types of jobs concurrently is a difficult task. Two major strategies are in common use: partitioning the cluster (thus avoiding the problem) and reservation combined with backfilling. Both have major drawbacks in overall performance, ease of use and fairness depending on the particular configuration, which can lead to heated debates between the users of the different types of jobs. In this paper, we introduce an approach to create dynamic virtual cluster partitions using para-virtualization techniques, to deal with the conflicts between parallel and serial jobs. The system dynamically adjusts to different types of job loads and offers easy and transparent use and configuration to both users and administrators, without resorting to complicated runtime prediction or backfilling algorithms. A proof-of-concept implementation based on the Sun Grid Engine scheduling system and Xen is presented.Keywords
This publication has 17 references indexed in Scilit:
- Countering security threats in service-oriented on-demand grid computing using sandboxing and trusted computing techniquesJournal of Parallel and Distributed Computing, 2006
- VSched: Mixing Batch And Interactive Virtual Machines Using Periodic Real-time SchedulingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- Diagnosing performance overheads in the xen virtual machine environmentPublished by Association for Computing Machinery (ACM) ,2005
- Performance Implications of Virtualization and Hyper-Threading on High Energy Physics Applications in a Grid EnvironmentPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- Xen and the art of virtualizationPublished by Association for Computing Machinery (ACM) ,2003
- Multiple-Queue Backfilling Scheduling with Priorities and Reservations for Parallel SystemsPublished by Springer Nature ,2002
- Multiple-queue backfilling scheduling with priorities and reservations for parallel systemsACM SIGMETRICS Performance Evaluation Review, 2002
- Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfillingIEEE Transactions on Parallel and Distributed Systems, 2001
- Predicting application run times using historical informationPublished by Springer Nature ,1998
- The ANL/IBM SP scheduling systemPublished by Springer Nature ,1995