Title: Automatic Dependency Management for Scientific Applications on Clusters
Abstract: Software installation remains a challenge in scientific computing. End users require custom software stacks that are not provided through commodity channels. The resulting effort needed to install software delays research in the first place, and creates friction for moving applications to new resources and new users. Ideally, end-users should be able to manage their own software stacks without requiring administrator privileges. To that end, we describe vc3-builder, a tool for deploying software environments automatically on clusters. Its primary application comes in cloud and opportunistic computing, where deployment must be performed in batch as a side effect of job execution. vc3-builder uses workflow technologies as a means of exploiting cluster resources for building in a portable way. We demonstrate the use of vc3-builder on three applications with complex dependencies: MAKER, Octave, and CVMFS, building and running on three different cluster facilities in sequential, parallel, and distributed modes.
Publication Year: 2018
Publication Date: 2018-04-01
Language: en
Type: article
Indexed In: ['crossref']
Access and Citation
Cited By Count: 5
AI Researcher Chatbot
Get quick answers to your questions about the article from our AI researcher chatbot