gRPC versus REST: A Performance Comparison

TL;DR:
We benchmarked REST vs gRPC under identical local setups using .NET. While gRPC is widely believed to outperform REST, our results show REST can be equally, or even slightly faster in specific data-heavy operations.

As microservices multiply across modern systems, the choice of API protocol can make or break performance. REST has ruled for years, but gRPC promises to be faster, leaner and more efficient. So why hasn't it taken over? Microservices can be heavily dependent on each other, which means speed and stability is key. When gRPC claims to be faster than REST, why isn't it the de facto standard? In this blog we will put gRPC and REST head to head, to see which is actually faster.

gRPC is a superior technology to REST! At least that is what multiple sources claim. According to various blogs, gRPC performs better and faster than REST on several metrics. In this blog we will test specifically, how fast a REST client can handle different requests and responses, and compare it to how fast a similar gRPC client handles the same requests and responses. This begs the question...

Is gRPC faster than REST?

We hypothesize that gRPC is able to send and receive requests faster than a traditional REST. To test this, the following experiments have been developed.

The Experiment

To test the hypothesis, two experiments were created, one utilizing gRPC and one utilizing REST. These experiments have to adhere to the following:

Rules

To ensure accurate measurements, the results must be obtained from the same computer.
Multiple data structures will be tested.
The setup for both APIs has to be as similar as possible.
The time used for measuring should be obtained from the client.

Set up

To adhere to the multiple data structures rule, a local database has been created, this database will provide a single instance of an object, as well as multiple instances of objects that will be stored in a list.
Each API will have 12 methods to call.
- Three for a single instance which takes a parameter of Id.
  - Each one with a larger payload
- Three for a single instance which takes a parameter of Id.
  - Each one with a deeper payload
- Three for a collection of 100 instances, which takes no parameters.
  - Each one with a larger payload
- Three for a collection of 100 instances, which takes no parameters.
  - Each one with a deeper payload
Each API will be tested with a client written in C#, as a console app.
- Time will be measured with .NET Stopwatch.
- The stopwatch will begin when the method is called and end when the API returns the full data.
To minimize anomalies and outliers, each operation will be executed 100 times, and the average call speed will be evaluated.

Specs

I7-9700k, 8 core, 4.6GHz

Samsung SSD 840 EVO 250GB

NVIDIA GeForce GTX 1070

2x8 GB HyperX Fury 2666MHz DDR4 Memory

REST

What is REST?

We have decided to work with the common implementation of REST and not the full implementation of a RESTful API.

The key features to take note of when using REST:

Separation of client and server

Server and client can be implemented independently without knowing each other.
Server code can be changed without affecting the client.
Client code can be changed without affecting the server.
Both server and client are aware of methods available.

Statelessness

Stateless means that the server is not required to know the current state of the client and vice versa.
Either end can understand any method calls, without knowing the previously called methods.

Invocation

We invoke a method on the server via HTTP operations
- GET
- POST
- PUT
- DELETE

Setting Up the Experiment for the REST API

The architecture for this experiment is a simple one:

REST Architecture

Sample Project and Metrics

If you want to replicate this experiment yourself, the database setup can be found in the repository and the source code for the rest-api can be found in the repository.

Running our setup yielded the following results:

Single payload

The difference between a single small payload and a single large payload is small in the context of a daily task. A single small payload has a mean response time of 0.0181 whilst a single large payload has a mean response time of 0.0204 seconds. But in relation to each other, it's a 12.7% increase in response time.

API Single Payload

To put this into perspective a small payload contains 10 values of data. A large payload contains (4+(69))6+4 or 352 values. This means that we have requested 3420% more data and it only took 12.7% longer.

To test different scenarios we also created a "deep" payload that contains a different amount of nested objects. The deepest payload contains a total of eight nested objects, however, the total amount of values is far less in comparison to the previous payload. The previously mentioned payload peaked at 352 values whereas the deepest payload peaks at (4+(64))+(4+(74))+(4+(8*4)) values, or 96 values in total. In other words, the deep payloads are much smaller in size but different in structure.

To give a concrete example, a large payload is structured like so:

large_payload {
    id,
    string_Value,
    int_value,
    double_value,
    medium_...(2743 chars omitted)...s.

gRPC

What is gRPC?

gRPC is a modern, open-source remote procedure call (RPC) framework that can run anywhere. It enables client and server applications to communicate transparently and develop connected systems.

Some key features we would like to highlight:

HTTP/2 support

HTTP/2 is HTTP/1's successor, which is what most websites and frameworks utilize today. In many ways, HTTP/2 is an improved version of HTTP/1, and HTTP/3 is already in the works.

Language independent

gRPC is language independent, which means it doesn't matter which language you develop in. The framework supports a handful of popular languages. This is quite an advantage when you're developing microservices, which might have services developed in different languages and frameworks.

Contract First

gRPC is strictly contract first which is a design approach that works especially well in larger development teams. It also excels when developing microservices, as a contract would be created before any actual implementations can be done. The contract is designed in the .proto file, which is also where gRPC gains some of its speed from, seeing as .proto files are...

Strongly typed As a by-product of a strongly typed proto file, which is used as a contract between client and server, but also used as an extensible mechanism for serializing structured data.

Setting Up the gRPC Project

For the gRPC architecture we use the same as the REST, we have a client and a server running locally. The client calls the methods exposed by the proto file. The method then gets executed on the server and queries the database, once the data has been obtained it replies to the client. When the client has received all the data, we stop and log the time elapsed since the call started.

gRPC Architecture

Sample Project and Metrics

If you want to replicate this experiment yourself, database setup can be found in the repository and the source code for the grpc-project can be found in the repository.

Running our setup yielded the following results:

Single payloads

gRPC Wide Payload Results

The test on single payloads yielded quite odd results, where the medium payload proved to be the fastest on average, and the largest payload only being slightly slower than the smallest. Just to recap the numbers; a larger payload contains 3400% more data than a small payload, and yet it only took 2.36% longer to get that data.

gRPC Deep Payload Results

Even more odd were the results of the deep payloads. Once again the payload containing a "medium" amount of data, was the fastest, just like previously. But unlike previously, the deepest payload was significantly faster than the deep payload, to be precise; the deepest payload, which contains 300% more data than a deep payload was 28.63% faster. As the results are rather unexpected we have to take a close look at possible errors that could have occurred.

Collection of payloads

gRPC Wide Payload Results

Collections paint a different picture, a small payload collection averaged 0.01911 seconds, while a collection of large payloads took 0.7025. This means a large payload on average took 267.6% longer to get. These results are much closer to what we would expect. We did the same test with a collection of deep payloads.

gRPC Deep Payload Collection Results

The results of this test, were as you would expect, as the payloads incrementally increase in size, they also increase incrementally in response time.

Conclusion

When we put the two charts next to each other, it's easy to see which one has an edge. The REST API is represented by the blue blocks, whilst gRPC is represented by the red blocks, just like previously.

Wide Payload results comparison

Deep payload results comparison

This is the case for both single instances of objects as well as collections of objects.

Deep payload collection result comparison

Wide payload collection result comparison

We hypothesized that gRPC would be faster than REST, based on the numerous blogs claiming this to be true, with their own tests. Contrary to popular belief, our experiments suggest that under certain conditions, REST can outperform gRPC in raw speed, reminding us that performance isn't universal but contextual.

These results might not seem as much, but it has been proven that people on average don't wait around for data to load and will abandon a web page or program if loading times are too long. When moving large amounts of data, a small amount of time can be the difference between keeping or losing a customer.

This prompts the question: When to use gRPC and when to use REST

We would argue that gRPC fits into a setting, where you need to have multiple programs or services talking to each other across different languages, especially when the task that needs to connect to an endpoint is an action that needs to be executed; one such action could be TurnOnTheWater(). This argument is based on the research made into gRPC, rather than the results of these particular tests.

REST on the other hand operates on the four aforementioned HTTP operations, these operations indicated data transfers of one sort or the other. While REST can execute the same actions as gRPC, the action TurnOnTheWater() doesn't fit into what a REST API was designed for. We would instead use REST where we required data transfers and other typical CRUD mechanics.

Ultimately, the right choice depends on your use case: REST for simplicity and interoperability, gRPC for high-efficiency internal microservice communication.

Possible Errors

Network outage during some of the tests
The gRPC serverside logging was set to critical, tweaking this option might yield different results.

What's Next?

This blog has only been about the differences in speed between REST and gRPC, but in reality, many other factors are present, if we were to truly compare the two frameworks. gRPC has claimed to not only be faster, but also more reliable, stable, and secure, and all of these metrics, as well as other metrics, would be interesting to cover, they are however out of scope in this particular blog-post.

Technologies Used

gRPC
.NET Web Api
.NET console app
MySQL Database

gRPC versus REST: A Performance Comparison

gRPC versus REST: A Performance Comparison

The Experiment