Why Stream Switched from Python to Go

Name an online company today and there is a good chance all or part of its back end uses Python. Like JavaScript, most businesses use it as the foundation for all the framework underlying customization and integration. However, as systems grow evermore complex companies are looking at other languages to streamline processes. For example, Thierry Schellenbach looks at how the company Stream announced recently that it has changed its core service language from Python to Go. The company uses Go for all performance intensive code, and it still uses Python where its programmers feel makes sense.

To understand the importance of this recent change, knowing the company's product is crucial. Stream is an API for building, scaling, and personalizing news feeds and activity streams. Stream serves about one billion API requests per month for over 300 million end users. The company emphasizes performance and reliability, two factors that drive every technical decision it makes.

Choosing fast tools is very important (Stream optimizes Cassandra, PostgreSQL, Redis, and many other technologies). However, at times the team finds that Python is the bottleneck in the company's system, its programming language of choice.

Computationally heavy tasks like serialization, ranking, and aggregation were sometimes taking much longer than retrieving the data from a data store over the network. Stream's team knew they could do better. Switching from Python to Go erased a great deal of time, making the application code once again more like the glue between services and not a major bottleneck in optimization.

Reading through Go code is very straightforward. Stream migrated from a Python codebase with many different authors and lots of opinions regarding code style and frameworks, each of which added a small amount of magic. Go, on the other hand, forces a clean style and nudges authors away from being clever. Developers can get up to speed quickly when reading each other’s code (or their own code from some time ago).

Perhaps the easiest sell for Go was its performance, both at runtime and compile time. It's comparable to Java or C++ in most computing benchmarks. In real-world usage, developers found that Go is approximately 30 times faster than Python.

It’s important to note that Stream still uses Python where it makes sense. For instance, the company's dashboard, website, and machine learning for personalized feeds use Python because the tooling is much better. Stream won’t be saying goodbye to Python anytime soon, but going forward developers will write all performance-intensive code in Go.

The Go compiler (which itself is written in Go) is also incredibly fast. Stream’s most complex microservice written in Go takes just six seconds to compile, and that's a major win compared to toolchains like Java and C++, which are infamous for sluggish compile times of minutes or even hours for more complex full builds.

In closing, Go is a great language for writing microservices. It's very fast, has native concurrency primitives, excellent support for existing tools, and is downright fun to use when developing. Writing Go might take longer to write up front compared to scripting languages like Ruby or Python, but the maintenance costs are far lower and you'll save lots of time that would otherwise be spent on optimizing code.

Be sure to read the next API Strategy article: How SwaggerHub Helps Your API Development Program Scale

Original Article

From Python to Go: Why Stream switched to Go

 

Comments

Scalini-Torhout

Great article, I didn't know about Go.

Thanks for the insight!