How Thunder Compute works (GPU-over-TCP)

We created proprietary technology to virtualize GPUs

Thunder Compute uses network-attached GPUs instead of physically-attached GPUs. This means that we can allocate GPUs much more efficiently than traditional cloud platforms. As a result, our prices are much lower for the same hardware.

Under the hood, these GPUs are network-attached over TCP. From your perspective, the resulting instances behave like they have GPUs without requiring that a GPU is physically connected.

Virtual GPUs act like physical GPUs for lower cost

From a user's perspective, all instances on Thunder Compute are on-demand instances, exactly like you would find on AWS, GCP, or Azure. Under the hood, these instances are connected to GPUs over a network. Intuitively, the cloud instances you interact with on Thunder Compute have all of the functionality of EC2 instances that you would find on Amazon or Google Cloud. In fact, they are hosted on Amazon or Google Cloud.

Many cloud instances share each physical GPU

Here is a diagram of how instances communicate with GPUs:

Virtual GPUs are over 5x more efficient

Now that you understand the distinction between a Thunder Compute instance and a GPU instance on EC2, it is worth explaining why we use virtualization. Primarily, virtualization allows us to serve more customers with fewer GPUs. This lets us pass through (much) lower pricing to you than you see anywhere else for comparable hardware.

This technology is early and continues to evolve

There are a few limitations of this virtualized approach:

1. Performance: TCP is slower than PCIe. In early prototypes, virtual GPUs were thousands of times slower than native GPUs. Through extensive optimization, we have brought average runtime very close to native. Exact measurements vary by use case, but most of our users notice no performance difference. Over time we expect this to continue to improve.

2. Limited Compatibility: Today, Thunder Compute is focused on AI/ML workloads. Eventually, our virtual GPUs will support the full functionality of physical GPUs, including rendering, graphics, and simulation.

Until now, our testing has shown data science and AI development workflows to be the most performant and stable. Thunder Compute is free to try and incredibly cheap. Try it and see what we're about. Reach out to us in our discord if you run into issues.