Increasing Demand for Research Computing

Computational based research has grown rapidly together with the traditional theoretical and experimental based research in the past 20 years in NUS. During the late 80’s, we only had around a dozen researchers accessing the national supercomputing resources. In the 90’s, we had 200 to 300 researchers using the university's High Performance Computing (HPC) resources. Today, the figure has grown to more than 1,000. The growth in computational based research activities also means that more researchers have to work out the cost of their computing requirements whenever they submit a research proposal or apply for research grant, which can be challenging at times. We hope to offer some tips on what to consider in estimating the cost of research computing.

Research Computing Cost Estimation

The type of resources and the quantity have to be identified first before the cost can be estimated. In general two basic types of resources are required for research computing; namely the compute system and the storage system.

For compute system, two commonly used HPC systems are the cluster system and the single-image multi-processor system. Single-image multi-processor systems generally cost more than the cluster systems for the same number of processors. These are needed for large memory applications that can not be parallelized to run on a distributed memory cluster environment. The cluster system is a more cost effective option for applications that can be parallelized using message passing programming model such as MPI. The quantity needed (the number of processor cores) is dependent on the application. Most applications are scalable with good speedup using an average of 16 processor cores. However, for some time-critical research projects, even achieving only 20% speedup with 100% increase in processor cores is considered a worthy investment. To determine the total number of processor cores required, researchers need to know the number of simulations they need to run and whether multiple simulations have to be executed at the same time. For example, if 16 processor cores are required for each simulation and three simulations will be executed at any point in time, then a system with 48 processor cores will be needed.

Today, a standard cluster node for HPC comes with twelve (12) processor cores. The cost of a blade cluster with high-performance network interconnection with five (5) years warranty is estimated to cost $10,000 per node. This figure covers only the hardware cost while the total cost of ownership (TCO) may include other areas such as hosting space, systems administration, utility (power & cooling) and software tools such as compilers, numerical libraries and job management tools. Based on a TCO analysis of a recent cluster implementation, around 40% of the TCO can be attributed to hardware cost, 30% to utility and hosting, 15% to systems administration and around 15% to software tools and other shared resources. Not all these cost components are relevant to everybody. For example, for researchers managing their own system and using open-source software tools, the cost of the last two components can be excluded in the estimation.

As research simulations get bigger and more data intensive, storage requirements will become a major consideration in cost estimation. These are work/scratch space and data archiving. The requirement for storage can range from gigabytes to tens or even hundreds of terabytes, depending on the application and the area of research. For example, a weather change simulation project we currently support requires terabytes of work space and probably more for archiving. Usually, the more expansive parallel file system or high-performance network attached storage (NAS) system is used as work/scratch space and cheaper storage system is used for archiving. Based on our experience, a parallel file system with 120 terabytes of storage may cost around $3,000 to $4,000 per terabyte with a 5-year hardware/software warranty while a similar capacity commodity based archiving storage system generally costs about half of that. For some systems, the per-terabyte cost may decrease with the increase in capacity. Again the TCO of a storage system is expected to be at least double the hardware cost.

Alternative Sourcing

Researchers can also estimate the research computing cost according to the on-demand utility-based services, such as those provided by Amazon (http://aws.amazon.com/ec2/pricing/). For HPC, the relevant service will be the Cluster Compute Instances service, which costs US$1.60/8-core/hour with 1.69 terabytes of storage. The storage-only cost is also available at http://aws.amazon.com/s3/#pricing.