Issue tracker

Considering that issues opened about translations may be written in the translation language, which can be considered noise but at least is inconsistent, issues should be placed outside bugs.python.org (b.p.o).

As all translation must have their own GitHub project (see Repository for Po Files), they must use the associated GitHub issue tracker.

Considering the noise induced by translation issues redacted in any languages which may beyond every warnings land in b.p.o, triage will have to be done. Considering that translations already exist and are not actually a source of noise in b.p.o, an unmanageable amount of work is not to be expected. Considering that Xiang Zhang and Victor Stinner are already triaging, and Julien Palard is willing to help on this task, noise on b.p.o is not to be expected.

Also, language team coordinators (see Language Team) should help with triaging b.p.o by properly indicating, in the language of the issue author if required, the right issue tracker.

Branches

Translation teams should focus on last stable versions, and use tools (scripts, translation memory, …) to automatically translate what is done in one branch to other branches.

The three currently stable branches that will be translated are 12: 2.7, 3.5, and 3.6. The scripts to build the documentation of older branches needs to be modified to support translation 12, whereas these branches now only accept security-only fixes.

The development branch (master) should have a lower translation priority than stable branches. But docsbuild-scripts should build it anyway so it is possible for a team to work on it to be ready for the next release.

Domain Name, Content negotiation and URL

Different translations can be identified by changing one of the following: Country Code Top Level Domain (CCTLD), path segment, subdomain or content negotiation.

Buying a CCTLD for each translations is expensive, time-consuming, and sometimes almost impossible when already registered, this solution should be avoided.

Using subdomains like “es.docs.python.org” or “docs.es.python.org” is possible but confusing (“is it es.docs.python.org or docs.es.python.org?”). Hyphens in subdomains like pt-br.doc.python.org is uncommon and SEOMoz 23 correlated the presence of hyphens as a negative factor. Usage of underscores in subdomain is prohibited by the RFC1123 24, section 2.1. Finally, using subdomains means creating TLS certificates for each language. This not only requires more maintenance but will also cause issues in language switcher if, as for version switcher, we want a preflight to check if the translation exists in the given version: preflight will probably be blocked by same-origin-policy. Wildcard TLS certificates are very expensive.

Using content negotiation (HTTP headers Accept-Language in the request and Vary: Accept-Language) leads to a bad user experience where they can’t easily change the language. According to Mozilla: “This header is a hint to be used when the server has no way of determining the language via another way, like a specific URL, that is controlled by an explicit user decision.” 25. As we want to be able to easily change the language, we should not use the content negotiation as a main language determination, so we need something else.

Last solution is to use the URL path, which looks readable, allows for an easy switch from a language to another, and nicely accepts hyphens. Typically something like: “docs.python.org/de/” or, by using a hyphen: “docs.python.org/pt-BR/”.

As for the version, sphinx-doc does not support compiling for multiple languages, so we’ll have full builds rooted under a path, exactly like we’re already doing with versions.

So we can have “docs.python.org/de/3.6/” or “docs.python.org/3.6/de/”. A question that arises is: “Does the language contain multiple versions or does the version contain multiple languages?”. As versions exist in any case and translations for a given version may or may not exist, we may prefer “docs.python.org/3.6/de/”, but doing so scatters languages everywhere. Having “/de/3.6/” is clearer, meaning: “everything under /de/ is written in German”. Having the version at the end is also a habit taken by readers of the documentation: they like to easily change the version by changing the end of the path.

So we should use the following pattern: “docs.python.org/LANGUAGE_TAG/VERSION/”.

The current documentation is not moved to “/en/”, instead “docs.python.org/en/” will redirect to “docs.python.org”.

Language Tag

A common notation for language tags is the IETF Language Tag 3 4 based on ISO 639, although gettext uses ISO 639 tags with underscores (ex: pt_BR) instead of dashes to join tags 5 (ex: pt-BR). Examples of IETF Language Tags: fr (French), ja (Japanese), pt-BR (Orthographic formulation of 1943 - Official in Brazil).

It is more common to see dashes instead of underscores in URLs 6, so we should use IETF language tags, even if sphinx uses gettext internally: URLs are not meant to leak the underlying implementation.

It’s uncommon to see capitalized letters in URLs, and docs.python.org doesn’t use any, so it may hurt readability by attracting the eye on it, like in: “https://docs.python.org/pt-BR/3.6/library/stdtypes.html”. RFC 5646 (Tags for Identifying Languages (IETF)) section-2.1 7 states that tags are not case sensitive. As the RFC allows lower case, and it enhances readability, we should use lowercased tags like pt-br.

We may drop the region subtag when it does not add distinguishing information, for example: “de-DE” or “fr-FR”. (Although it might make sense, respectively meaning “German as spoken in Germany” and “French as spoken in France”). But when the region subtag actually adds information, for example “pt-BR” or “Portuguese as spoken in Brazil”, it should be kept.

So we should use IETF language tags, lowercased, like /fr/, /pt-br/, /de/ and so on.

Fetching And Building Translations

Currently docsbuild-scripts are building the documentation 8. These scripts should be modified to fetch and build translations.

Building new translations is like building new versions so, while we’re adding complexity it is not that much.

Two steps should be configurable distinctively: Building a new language, and adding it to the language switcher. This allows a transition step between “we accepted the language” and “it is translated enough to be made public”. During this step, translators can review their modifications on d.p.o without having to build the documentation locally.

From the translation repositories, only the .po files should be opened by the docsbuild-script to keep the attack surface and probable bug sources at a minimum. This means no translation can patch sphinx to advertise their translation tool. (This specific feature should be handled by sphinx anyway 9).

Mailing List

The doc-sig 30 mailing list will be used to discuss cross-language changes on translated documentation.

There is also the i18n-sig list but it’s more oriented towards i18n APIs 1 than translating the Python documentation.

Chat

Due to the Python community being highly active on IRC, we should create a new IRC channel on freenode, typically #python-doc for consistency with the mailing list name.

Each language coordinator can organize their own team, even by choosing another chat system if the local usage asks for it. As local teams will write in their native languages, we don’t want each team in a single channel. It’s also natural for the local teams to reuse their local channels like “#python-fr” for French translators.

Repository for PO Files

Considering that each translation team may want to use different translation tools, and that those tools should easily be synchronized with git, all translations should expose their .po files via a git repository.

Considering that each translation will be exposed via git repositories, and that Python has migrated to GitHub, translations will be hosted on GitHub.

For consistency and discoverability, all translations should be in the same GitHub organization and named according to a common pattern.

Given that we want translations to be official, and that Python already has a GitHub organization, translations should be hosted as projects of the Python GitHub organization 31.

For consistency, translation repositories should be called python-docs-LANGUAGE_TAG 22, using the language tag used in paths: without region subtag if redundant, and lowercased.

The docsbuild-scripts may enforce this rule by refusing to fetch outside of the Python organization or a wrongly named repository.

The CLA bot may be used on the translation repositories, but with a limited effect as local coordinators may synchronize themselves with translations from an external tool, like transifex, and lose track of who translated what in the process.

Versions can be hosted on different repositories, different directories or different branches. Storing them on different repositories will probably pollute the Python GitHub organization. As it is typical and natural to use branches to separate versions, branches should be used to do so.

Translation tools

Most of the translation work is actually done on Transifex 15.

Other tools may be used later like https://pontoon.mozilla.org/ and http://zanata.org/.

Documentation Contribution Agreement

Documentation does require a license from the translator, as it involves creativity in the expression of the ideas.

There’s multiple solutions, quoting Van Lindberg from the PSF asked about the subject:

1. Docs should either have the copyright​ assigned or be under CCO. A permissive software license (like Apache or MIT) would also get the job done, although it is not quite fit for task.
2. The translators should either sign an agreement or submit a declaration of the license with the translation.
3. We should have in the project page an invitation for people to contribute under a defined license, with acceptance defined by their act of contribution. Such as:

“By posting this project on Transifex and inviting you to participate, we are proposing an agreement that you will provide your translation for the PSF’s use under the CC0 license. In return, you may noted that you were the translator for the portion you translate. You signify acceptance of this agreement by submitting your work to the PSF for inclusion in the documentation.”

It looks like having a “Documentation Contribution Agreement” is the most simple thing we can do as we can use multiple ways (GitHub bots, invitation page, …) in different context to ensure contributors are agreeing with it.

Language Team

Each language team should have one coordinator responsible for:

• Managing the team.
• Choosing and managing the tools the team will use (chat, mailing list, …).
• Ensure contributors understand and agree with the documentation contribution agreement.
• Ensure quality (grammar, vocabulary, consistency, filtering spam, ads, …).
• Redirect issues posted on b.p.o to the correct GitHub issue tracker for the language.

Simplified English

It would be possible to introduce a “simplified English” version like Wikipedia did 10, as discussed on python-dev 11, targeting English learners and children.

Pros: It yields a single translation, theoretically readable by everyone and reviewable by current maintainers.

Cons: Subtle details may be lost, and translators from English to English may be hard to find as stated by Wikipedia:

> The main English Wikipedia has 5 million articles, written by nearly 140K active users; the Swedish Wikipedia is almost as big, 3M articles from only 3K active users; but the Simple English Wikipedia has just 123K articles and 871 active users. That’s fewer articles than Esperanto!