Why hasn't Python's Type Hint become popular?

I saw an interesting question on Zhihu, Why is TypeScript so popular, but it's rare to see people writing Python with type annotations?

Although I couldn't resist answering on Zhihu, just in case, I decided to expand and update it on my blog.

BTW, I've been really exhausted recently with the recent launch, so I'm writing an article to relax.

Introduction#

The answer to this question is actually quite simple, it's due to historical baggage and ROI. Before understanding why this phenomenon exists, we first need to understand what Type Hint can bring us, and then we need to understand the history of Type Hint.

As of now (March 2020), Type Hint can bring us visible benefits:

Through annotation, combined with IDE support, it can improve our coding experience.
With the support of tools like mypy/pytype, we can integrate static type checking into our CI/CD process.
With the support of pydantic and many modern frameworks, we can reduce a lot of repetitive work.

You might think that Python Type Hint has matured since the introduction of PEP 484 in Python 3.5. But in reality, it has come a long way in a much shorter time than you might imagine.

Alright, let's take a look at the key milestones in the development history of Type Hint.

PEP 3107 Function Annotations
PEP 484 Type Hints
PEP 526 Syntax for Variable Annotations
PEP 563 Postponed Evaluation of Annotations

PEP 3107#

As mentioned earlier, most people's first encounter with Type Hint should be when it was proposed in September 2014 and accepted in May 2015 in PEP 484. But in fact, the embryonic form of Type Hint existed much earlier, and the syntax of PEP 484 actually originated from the syntax designed in 2006 and introduced in PEP 3107 for Python 3.0, see PEP 3107 -- Function Annotations.

In PEP 3107, there is a description of the goals of this proposal:

Because Python's 2.x series lacks a standard way of annotating a function's parameters and return values, a variety of tools and libraries have appeared to fill this gap. Some utilise the decorators introduced in "PEP 318", while others parse a function's docstring, looking for annotations there.
This PEP aims to provide a single, standard way of specifying this information, reducing the confusion caused by the wide variation in mechanism and syntax that has existed until this point.

In simple terms, in order to add additional metadata to a function's parameters or return values, people came up with various methods, such as using decorators introduced in "PEP 318" or parsing a function's docstring to find annotations. This PEP aims to provide a standardized way of specifying this information, reducing the confusion caused by the wide variation in mechanism and syntax that existed until that point.

The resulting syntax is as follows:

def foo(a: 'x', b: 5 + 6, c: list) -> max(2, 9):
    pass

Does it look familiar? Yes, PEP 3107 actually laid the foundation for subsequent Type Hint.

It can be annotated.
It is part of the function/method information and can be inspected.
It is evaluated at runtime.

But a new question arises, why is this proposal often overlooked? Or do we need to look at it from a specific point in time?

This proposal can be traced back as early as 2006 and was confirmed to be introduced in Python 3 in PEP 3000, which is probably the most famous proposal in Python history (declaring the birth of Python 3) and was officially released in 2008.

At that time, PEP 3107 faced two problems:

From 2006 to 2008, the community's main focus was on discussing why we needed Python 3 and why we should migrate to Python 3.
PEP 3107 only told everyone that you can annotate and easily access annotation information, but how to abstractly represent a type, such as a list of integers, still relied on the community's own development. In other words, it was left to the community to figure it out.

Problem 1 had no solution and could only be pushed forward with time. Problem 2 led to the birth of PEP 484.

PEP 484#

You should have some understanding of PEP 484, so I won't go into the specifics of the proposal here.

The significance of PEP 484 lies in the fact that it built a reasonable abstraction of Python's type system on top of the syntax and foundation laid by PEP 3107. This is also the important product of typing. Only at this point did Python's type hint have a basic official specification and achieve basic usability. This happened in September 2015 (September 13, Python 3.5.0 was officially released).

But in reality, PEP 484 at that time could only be said to be basically usable. Let me give you a few examples that were criticized.

Let's look at a piece of code:

from typing import Optional

class Node:
    left: Optional[Node]
    right: Optional[Node]

This code is actually quite simple, it describes a standard binary tree node. However, in the context of PEP 484, this code exposes two problems:

Variables cannot be annotated. As I mentioned earlier, PEP 484 is essentially an extension of PEP 3107, and at this time, the scope of hint was limited to functions/methods only. In the above code, during the 3.5 period, I couldn't annotate my left and right variables. The fact that variables, one of the basic elements of a programming language, couldn't be annotated with Type Hint meant that this type hint functionality was not complete to some extent.
Circular references, quite literally, how to solve the problem of circular references in Type Hint on the community/StackOverflow was a headache. The community: What the fuck?

Fortunately, the Python community realized this problem and introduced two proposals to solve these problems.

PEP 526#

Problem 1 led to the birth of PEP 526 -- Syntax for Variable Annotations, which was proposed in August 2016 and accepted in September 2016. It was implemented in BPO-27985 in January 2018. In my impression, this should be one of the PEPs in the Python community with the least controversy, fastest acceptance, and fastest implementation.

In PEP 526, Python officially allowed annotations on variables, whether they are class attributes or regular variables.

class Node:
    left: str

This is allowed.

def abc():
    a:int = 1

This is also allowed.

Based on this proposal, the Python official also promoted the implementation of PEP 557 -- Data Classes. But that's another story.

Going back to PEP 526, it only solved problem 1 mentioned above and did not solve problem 2. This issue would be addressed by PEP 563.

PEP 563#

To solve the problem of circular references, Python introduced PEP 563 -- Postponed Evaluation of Annotations in September 2017, proposed by the community, accepted in November 2017, and implemented in GH-4390 in January 2018.

After PEP 563, we can write our code like this:

from typing import Optional

class Node:
    left: Optional["Node"]
    right: Optional["Node"]

Yes, PEP 563 finally solved the two problems in PEP 484.

Summary#

With PEP 563 as an important dividing point, Python only began to have a complete ecosystem and production usability after January 2018. If we consider the release version, it should be after June 2018, after the official release of Python 3.7.

After Python 3.6/7, the community began to build an ecosystem around Type Hint. For example, using PEP 526 to efficiently validate data formats, see pydantic.

By the way, this library is also the foundation of the currently popular framework FastAPI, which is also my favorite framework.

Major companies have also started to follow suit, such as Google's pytype and Microsoft's pyright for support in VSCode.

There are also many excellent libraries like starlette.

Only at this point did the true power of Python + Type Hint begin to emerge. This is when we can start answering the question, "Why should I switch to Type Hint?" I guess it's not just because it's enjoyable to write in an IDE.

When making technical decisions, we do so because the decision brings us enough benefits, in other words, a sufficient ROI, rather than simply because we like it.

From this perspective, it has only been a year and a half to two years. For a user habit formation cycle, this is too short. Not to mention that there are still a lot of Python 2 code lying around.

Going back to the comparison, TypeScript was released in October 2012 with version 0.8, and at that time, TypeScript had a relatively complete type system.

TypeScript took 8 years, and Python probably still has a long way to go.

Of course, this answer only discusses the question from a technical and historical perspective. Other factors, including the community's bargaining and compromise, are not within the scope of this answer. If you are interested, you can go to python-ideas, python-dev, and discuss-python to find discussions about these proposals in history, which are very interesting.

Finally, the success of TypeScript also has another reason, it has a good father and its father has money (escape).

Well, that's about it. I've been exhausted with work recently, so I can only write some junk articles to relieve stress and calm my mind.