Migration to Cloud Based HTC and HPC Systems |Build HPC Cluster

Long-term processing of varied information from myriad touch-points is often required for effective analysis. Creating a system of computing power to gain insights across multiple data inputs is challenging. High-throughput computing (HTC) systems and High Performance Computing (HPC systems) systems combine many computational resources to accomplish this task. Through the use of parallel processing, running advanced applications can be done reliably, rapidly and efficiently. The financial sector is a key industry for said integration, being a prime industry for said migration.

HTC (High-Throughput Computing) serial systems are more suited to running multiple independent instances of software on multiple processors at the same time.

HPC (High Performance Computing) systems enable users to run a single instance of parallel software over many processors.

This system is intended for testing and running large codes, parallel-processing codes, visualization and scientific applications. HPC systems do not mean to speed the time it takes for individual processes to complete. Instead, this type of computing system allows many versions of the same program to run concurrently. The power of this intent lies in the ability to use different data for each copy of the program, also known as jobs. Applications that might involve such processing include:

– patient data in large scale biomedical trials
– parts of a genome or protein sequence in bioinformatics applications
– random numbers in a simulations based on Monte Carlo methods
– model parameters in ensemble simulations or explorations of parameter spaces

Legacy Technology
“Legacy data centers were designed when these modern densities and configurations did not exist. These designs make it difficult to scale, as they did not consider the software, underlying sensor fabric and mechanical automation that are required to remove heat with a scalpel-like efficiency.” (Data Center journal) The original AT&T UNIX operating system began the multitasking operating system. Today, the UNIX® system takes the form of Single UNIX Specification, known to allow multiple implementations. “The Open Group is committed to working with the community to further the development of standards conformant systems by evolving and maintaining the Single UNIX Specification and participation in other related standards efforts. Recent examples of this are making the standard freely available on the web, permitting reuse of the standard in open source documentation projects, providing test tools, developing the POSIX and LSB certification programs.” (UNIX)

Benefits of Cloud Based HTC/HPC
Shifting a legacy capital expense to an operational expense is the main advantage in this migration. Cloud computing takes grid computing to a whole new level by using virtualization to summarize an operating system instance. Companies can form their own OS model and run via cloud whenever they need the resources. On-Demand computing power allows companies to scale up or down depending on need. This fact based agility is key to unlocking financial scale in what used to be considered a capital expense.


Click here for an outline of how to setup, step-by-step, done so by Rackspace.