Financial solutions

Results

There are many possible Virtual Machine (VM) types or flavours provided by the various Cloud Providers. Properly choosing the most optimal flavour for specific task requires additional calculation. It is done automatically by MELODIC. The list of the most suitable VM flavours for AI Investments is depicted in table below:

VM type VCPUs CPU RAM GPU GPU RAM Price/hour
Amazon Web Services – EU: Ireland – OS: Linux
p2.xlarge 4 61 NVIDIA K80 0.972$
g3.4xlarge 16 122 NVIDIA Tesla M60 8 1.210$
Azure—West Europe—OS: Ubuntu Advantage Standard
Nd6 6 122 NVIDIA Tesla P40 24 2.287EUR
Google Compute Engine—EU: Netherlands
nl-standard-8 8 30 NVIDIA Tesla T4 1.3684$

In the table the analysis of the different infrastructure scenarios for AI Invest-ments is presented. Each row contains one scenario. The private infrastructure gives the number of servers each hosting two GPUs providing a certain GPU time for training the models. According to the above cost assumptions the cost of this private infrastructure is calculated. If the number of GPUs provided by the on-premise servers is less than what is needed for the average 10 models to be trained, the addi tional GPUs must be sourced from a Cloud base infrastructure. Finally, additional GPU resources must be acquired form a Cloud burst infrastructure when the burst of 50 models must be trained, i.e. 40 models more than normally handled by the private infrastructure. Finally, the table also shows the total monthly cost of all servers in the given scenario adding together the cost of the private infrastructure, the Cloud base and the Cloud burst servers. Assuming the above maintenance cost, the TCO over a three years’ period is computed. The cost is minimal for the cases of having 5 private servers, i.e. not being continuously dependent on using Cloud servers, and only use the Cloud to manage the bursts.

VM typeVCPUsCPU RAMTotal monthly costTCO over 3 years
ServersGPUTimeCostGPUTimeCostGPUTimeCost
12144018185760559940113311016880247695
24288036164320419940113311015661203807
36432054242880279940113311014442159918
48576072221440140040113311013223116030
51072009030004011331101200472141
612864010830003810761046212976659
714100801264000361020991225581177
81611520144400034963936238085695
91812960162500032906881250690213
102014499180600030850826263194731
25503600045140000004514162500

As presented above, the difference between the most optimal and worst scenario costs is 175 554 usd in the 3 years TCO period. This level of savings is possible thanks to optimized multi-cloud deployment.