Switching from Python to Go? Here's What You Need To Know

Continued from page 2. 

In addition, Go has first-class support for Protocol Buffers and GRPC-based APIs. These two tools work well together for building microservices, which need to communicate via RPC. Developers only need to write a manifest that defines the RPC calls that can be made and what arguments they take. Then both the server and client code are generated automatically from this manifest. This resulting code is fast, has a very small network footprint, and is easy to use. This example shows how easy creating an API Endpoint is for left-padding a string:
 

Example showing how easy creating an API endpoint is for left-padding a string

From this manifest, developers can generate client code for different languages, such as C++, Java, Python, and Ruby. This means no more ambiguous REST endpoints for internal traffic. Google provides a full tutorial on how to use gRPC titled Protocol Buffer Basics: Go.

Python Versus Go

One experiment Stream conducted to compare Python and Go was taking its ranked Feed functionality in Python and rewriting it in Go. Ranked feeds enable our users to have full control over how their feeds are sorted. They do this by specifying a scoring Function in a JSON config. Have a look at the example below:

{
  "functions":{
    "simple_gauss":{
      "base":"decay_gauss",
      "scale":"5d",
      "offset":"1d",
      "decay":"0.3"
    },
    "popularity_gauss":{
      "base":"decay_gauss",
      "scale":"100",
      "offset":"5",
      "decay":"0.5"
    }
  },
  "defaults": {
      "popularity": 1
  },
  "score":"simple_gauss(time)*popularity"
}

To support this ranking method both the Python and Go code need to:

  1. Parse the expression for the score. In this case, we want to turn the string "simple_gauss(time)*popularity" into a function that takes an activity as input and returns a score as output.
  2. Create partial functions based on the JSON config. For example, we want "simple_gauss" to call "decay_gauss" with a scale of five days, offset of one day, and a decay factor of 0.3.
  3. Parse the "defaults" configuration so you have a fallback if a certain field is not defined on an activity.
  4. Use the function from Step 1 to score all activities in the feed.

Developing the Python version of the ranking code took roughly three days of writing the code, unit tests, and Documentation. Next, the team spent approximately two weeks optimizing the code. One of the optimizations made was translating the score expression (simple_gauss(time)*popularity) into an abstract syntax tree. The team also implemented caching logic, which pre-computed the score for certain times in the future.

In contrast, developing the Go version of this code took roughly four days, and the performance didn't require any further optimization. While the initial bit of development was actually faster in Python, the Go version ultimately required substantially less work.

This huge difference savings in time needed when optimizing the codebase is because of Go's language performance. With Python we had to parse the expression into abstract syntax trees and optimize/profile every function that we exposed via ranking. Because Go is so much faster than Python we didn't need to invest in more optimizations. The end result was that the vanilla Go code performed roughly 40 times faster than the well-optimized Python code.

Some other components of Stream's system took substantially more time to build in Go than in Python. As a general trend, developing Go code takes slightly more effort, but teams will spend much less time optimizing the code for performance.

Stream's Takeaways from Switching to Go

Go is a very performant language with great support for concurrency. It's almost as fast as languages like C++ and Java. While building things using Go takes more time when compared to Python or Ruby, you'll save a ton of time spent on optimizing the code.

The small development team at Stream powers the feeds for over 200 million end users. Go's combination of a great ecosystem, easy onboarding for new developers, fast performance, solid support for concurrency, and a productive programming environment make it a great choice for us.

Stream still leverages Python for our dashboard, site, and Machine Learning for personalized feeds. We won't be saying goodbye to Python anytime soon, but going forward we'll write all performance-intensive code in Go.

To learn more about Stream, this interactive tutorial is a great starting point. You can even follow the tutorial leveraging our Go SDK.

Be sure to read the next Developers article: GitHub Launches Preview of Organization Project Permissions API