How Python Is Becoming Faster?

Even though Python is a dynamic, easy-to-learn language with simple syntax, it is relatively slow when compared to languages like Java, Go, C and C++.

A comparison of popular frameworks built with Python (Django) and languages like Go (Gin) shows that the Go framework runs more requests per second (114 963) than Django (8 945); this shows that Go is about 12 times faster than Python.

The fundamental reason behind this slow execution of Python code is because it is a dynamically written language.

Java, for example, is a statically typed language that runs all necessary checks and compiles the code before runtime; this optimizes the program and makes it run faster.

On the other hand, Python is a language that is compiled at run time because, as a dynamically written language, any variable type or value can change while the program is running. For this reason, Python code cannot be compiled beforehand, and because of that, the code cannot be optimized at runtime as we have in low-level languages like Java or C

Another reason behind the slow execution of Python compared to languages like Java or C is because Python is an interpreted language - while Java is a machine language.

An interpreted language like Python needs to be converted to a machine language before its execution; this compilation process also explains another reason python is slow.

Even if Python is slower, it has a simple syntax and a large number of libraries and contributors. This can partly explain why it is being used in many common and less common fields like GPU-dedicated tasks like machine learning and artificial intelligence.

So, is there no way to make Python faster?

There's a couple of ways you can go about making your Python program faster using:

Multi-processing module

Python is a language that doesn’t allow concurrency through multithreading. Multithreading enables different program portions to run on separate CPU cores simultaneously, making the program run faster.

Python has a global interpreter lock which disallows it from multithreading; this is because python is a dynamic language that compiles at runtime; therefore, running multiple portions of a Python program at the same time will make the code encounter some issues.

The Python multiprocessing module bypasses the global interpreter lock and allows you to have multiple interpreters running concurrently to make your Python program execution faster. However, you may run into issues with shared and locking memory.

C extensions

Other than the multiprocessing module, writing a C extension for your Python code can significantly improve the run time of your program.

The default Python implementation, CPython, is natively written in C.

Because of this, you can write C code as an extension to your python code.

C is a fast low-level language that will help make your Python program run faster.

But, both of these methods also have their downsides. Running the multiprocess module effectively is hard to achieve because of memory sharing and locking while using C extensions; you need to know the C programming language.

However, all these are about to change

The creator of the Python language, Guido Van Rossum, unveiled plans for making Python faster in contribution to the 2021 virtual Python Language Summit held in May.

To improve the recently released Python 3.10, the plan to speed up Python to up to 2x will begin with Python 3.11.

In the published presentation, Van Rossum explained that the Python performance improvement project is handled by a "small team funded by Microsoft" as part of Microsoft’s way of giving back to the Python community.

He assured that the team would also take care of maintenance and support, and the project will be open source.

According to the plan, Van Rossum indicated that the project of speeding up Python would be sequential, targeting 2x in Python 3.11 and up to 5x speed in subsequent Python releases.

But will the team be able to achieve this speed?

Van Rossum stated that they are far from certain that they will reach 2x, but they are "optimistic and curious".

He gave some constraining factors they must work by because achieving 2x speed in version 3.11 might be hard.

He declared that any changes and improvements to Python must not break the stability of application binary interface (ABI) compatibility; It must not break the limited API compatibility. It must not break or slow down extreme cases. He added that the modification must "keep the code maintainable."

How do they plan on achieving 2x speed in Python 3.11?

Within the constraining factors, Rossum and the Python improvement project team identified some aspects they freely change to reach a 2x speed in version 3.11.

Since the bytecode compiler and interpreter are components that change in each release version, it is a great candidate for speed optimization without breaking anything.

He proposed introducing "Adaptive, specializing bytecode interpreter"; which will use bytecode instructions specific to a particular data type to execute that portion of the code. This will act like an inline cache which will speed up the execution.

Other proposed speed optimization for Python 3.11 includes optimizing the frame stack, proving the speed of function calls, changing pyc file format, and enacting a more efficient exception handling.

Depending on the speed improvements the team can achieve in Python 3.11, Rossum declared that achieving 5x speed may be possible, but "we'll have to be creative."

Is there an efficient way to Improve Python speed before Python 3.11?

Python 3.11, which is proposed to come with improvements that will make Python faster, will not be released until 2022. Before then, we need to find a way to speed up our Python programs.

To speed your Python programs, we can implement the Python multiprocessing modules or use C code as a Python extension, as explained earlier.

You can also use a JIT compiler such as Numba if you're using NumPy.

Numba is a just-in-time JIT compiler that uses decorators to convert Python and NumPy codes to machine code. It works by compiling Python codes, replacing the Python interpreter, then making Python functions compile directly to machine code, thereby improving the speed at run time.

Numba translates Python functions to optimized machine code at runtime using the industry-standard LLVM compiler library.

When using Numba, you don't need to replace the Python interpreter, run a separate compilation step, or even have a C/C++ compiler installed. You just need to apply one of the Numba decorators to your Python function, and Numba does the rest.

API developers may also use FastAPI, a modern, fast (high-performance), a web framework for building APIs with Python 3.6+ based on standard Python type hints. TechEmpower benchmarks show FastAPI applications running under Uvicorn as one of the fastest Python frameworks available.

If you are familiar with Flask, learning FastAPI will not take too much of your time:

For non API development, if you want to swap the popular Django or Flask in some of your time-sensitive workloads you may consider less used frameworks like Falcon, Bottle and apidaora. According to Web Frameworks Benchmark project, Falcon (3), Bottle (0.12) and apidaora (0.28) are the top 3 fastest Python frameworks for Python 3.9.