RUcore Resource Object
RUcore Resource Object
PDF-1(1.04 MB)
TitleInvestigating the use of autonomic cloudbursts within the MapReduce framework
NameHegde, Samprita (author), Parashar, Manish (chair), Marsic, Ivan (internal member), Pompili, Dario (internal member), Rutgers University, Graduate School - New Brunswick,
Degree Date2011-05
Date Created2011
SubjectElectrical and Computer Engineering,
Computer systems
DescriptionToday MapReduce framework is increasingly becoming a popular programming paradigm for data intensive computing, especially when there is ad-hoc data to be processed. In MapReduce programming paradigm, computation is done in two stages - a map stage and a reduce stage. The users simply have to provide a ‘map’ and a ‘reduce’ function and the underlying framework handles parallelizing and distributing the computation to worker nodes. Currently, the existing MapReduce frameworks work like a batch processing system where the cluster size is assumed to be static. We have developed a new objective-based scheduler which: 1. Provides both deadline and budget based scheduling capability 2. Provides cloudbursting capability where a computation can “burst” out to cloud whenever the existing datacenter is not capable of meeting the objective. Using these features, it is possible to run any MapReduce application subject to a user objective on any existing cluster by leveraging utility cloud resources. In this thesis, we use the Comet coordination engine and the MapReduce framework which is built on top of Comet Engine. The new autonomic scheduler works with the MapReduce Framework and manages the cluster as well as cloud in order to meet computation requirements. We have investigated the use of cloudbursting for MapReduce applications. We found that it is possible to run the application subject to both time and budget based objectives and successfully complete a job by efficiently using datacenter as well as cloud infrastructures.
NoteM.S.
NoteIncludes bibliographical references
Noteby Samprita Hegde
Genretheses
Persistent URLhttp://hdl.rutgers.edu/1782.1/rucore10001600001.ETD.000061267
Languageeng
CollectionGraduate School - New Brunswick Electronic Theses and Dissertations
Organization NameRutgers, The State University of New Jersey
RightsThe author owns the copyright to this work.