Planet Mozilla L10N

November 05, 2018

Mozilla L10n Blog

L10n Report: November edition

Please note some of the information provided in this report may be subject to change as we are sometimes sharing information about projects that are still in early stages and are not final yet.

Hello everyone,

We would like to thank you for reading the l10n reports and keeping updated on what’s happening in the Mozilla l10n community. And we would also like to make it even better! If you have a few minutes before or after reading this month’s report, could you please take this short, anonymous survey?

Yes, I take the survey

Thank you, and enjoy this new report!

Welcome!

New localizers

Derek started contributing to Navajo locale for Focus Android project. Welcome Derek!

Almar and Valdimar just recently joined a community workshop in Iceland, and immediately started contributing to Icelandic locale. Welcome to you both, we are glad to have you on board!

Lobsang is not only a new contributor, he also kicked off localization for Tibetan locale! Focus for Android will be the first browser ever localized in Tibetan 🙂 Congrats and welcome!

Are you a locale leader and want us to include new members in our upcoming reports? Contact us!

New community/locales added

Navajo was just recently added to Pontoon.

Tibetan was also added to our localization effort.

New content and projects

What’s new or coming up in Firefox desktop

Firefox 63 is currently available in release, while 64 is in beta, and 65 in Nightly. The last day available to ship updates for beta will be November 27.

Most of the strings landed towards the end of the Nightly cycle are still related to Privacy (Content Blocking, Trackers, etc.) and Security (certificate error pages), but there’s a also a new about:performance page, using Fluent for localization.

Talking about Fluent, as anticipated in the September edition of this report, expect a lot of strings moving to FTL files in the coming weeks. You can also visit arewefluentyet.com to visualize the work done around this area, and get an idea of how many strings are left.

A new project, strictly connected to Firefox, is also available in Pontoon for localization: Firefox Monitor. There are 2 distinct parts:

Both projects are scheduled for launch in the first half of November.

What’s new or coming up in mobile

With version 63 slowly rolling out to users in the coming week, you will be now able to use Firefox for Android in English for Canada (en-CA) and Ligurian (lij) on the release version! Congratulations to those teams for completing and shipping this work.

We’ve also shipped Firefox for iOS v14 a few days ago, and the main new features are:

A new experiment just came out in Indonesia, and you can try it out too: ScreenshotGO. This project is shipping on Google Play in Indonesia only at the time, but you can give it a try by installing it from its GitHub project page. The main idea is to provide better screenshot management on Android devices. Users can add a “Go” button their device to speed up the taking of screenshots.

What’s new or coming up in Foundation projects

Fundraising

The October fundraiser went out over the last few days and is doing pretty great so far, thanks a lot to everyone who contributed! It’s definitely helping the Foundation raise money to fund next year’s programs.

Fundraising email schedule for November and December is shaping up, starting on Giving Tuesday:

Email #1 Giving Tuesday (11/27)
Email #2 12/03
Email #3 12/17
Email #4 12/27
Email #5 – Mitchell message 12/31

It’s likely we will get help from a vendor for some of those emails, especially if the copy gets approved right before the holidays. Of course we will keep you updated.

Advocacy

The Advocacy team is working on test campaigns around misinformation and plan to launch one or two of them early this month. The campaigns can potentially be launched globally, but will always keep a focus on Europe. Markets and locales will heavily depend from the campaign targets.

The campaigns will have 3 main goals:

What’s new or coming up in Support

– New and updated Firefox 63 content awaits, giving your talent another chance to shine. Click the links below, see if your locale is waiting for your involvement and go for it!

– If you need an introduction or refresher on how localizing works for support.mozilla.org, please take a look at our step-by-step explanation (with videos).

– Warm thanks to the Bengali community for organizing a fun and exciting event that included localizing Support content.

What’s new or coming up in Pontoon

Errors & Warnings. We launched support for Errors & Warnings in Pontoon, which allow you to easily identify and fix translations breaking Firefox builds or exceeding the length limit on Mozilla.org. Note that Warnings, unlike Errors, mean that we’re not completely sure that the string contains critical issues, so it might be OK to just leave them unchanged. For more details, check out the documentation or read more about Errors & Warnings and a handful of other novelties in Pontoon.

Changing Machine Translation provider. We switched to Google Cloud Translation API as our machine translation provider. Previously, we used a free plan of Microsoft Translator Text API, which only worked with the old version of the API. That version is now being deprecated and lately started behaving unreliably, often not returning any results. One of the benefits of using Google Translate is that the number of Mozilla locales with machine translation support more than doubled (from 48 to 103).

Events

 

Friends of the Lion

Image by Elio Qoshi

Shout-out to Stoyan (one of our Bulgarian localizers) for working on a patch that adds several Firefox add-ons to help localizers test all permission strings. Update includes all recent webext permissions. Check it out here!

 

Know someone in your l10n community who’s been doing a great job and should appear here? Contact on of the l10n-drivers and we’ll make sure they get a shout-out (see list at the bottom)!

Useful Links

Questions? Want to get involved?

 

Did you enjoy reading this report? Let us know how we can improve by reaching out to any one of the l10n-drivers listed above.

November 05, 2018 07:19 AM

September 28, 2018

Mozilla L10n Blog

Bringing critical information closer to localizers

One of the missions of Pontoon since its inception has been to host critical information needed for localization under one roof. Over the course of the last few months several pieces of that puzzle have been solved.

Translation errors and warnings

Until recently, a vital piece of information was missing from Pontoon dashboards, namely translation errors and warnings. Localizers had to go hunt them on the standalone product dashboard, where they appeared 20-30 minutes after being submitted in Pontoon. If they wanted to fix an error, they had to go back to Pontoon, submit a (hopefully) legit translation, and wait another 20-30 minutes to see product dashboard go green.

As you can imagine, very few localizers actually did that. So in practice it was mostly project managers that were looking for errors, submitting suggestions with fixes and pinging localizers to approve them.

The first step towards bringing errors and warnings closer to localizers’ attention was made back in May when we started using compare-locales checks in Pontoon. These checks are split into two groups. The first consists of checks that need to be passed by translations landing in localized products (e.g. Firefox builds). Pontoon prevents translations failing those checks from even being saved. The second group contains more loose checks, which aren’t required in the build process and can be bypassed in Pontoon.

Since May we extended compare-locales support from the basic translation workbench to all places where translations can enter Pontoon, including batch actions, file upload, and import from VCS. Finally, we performed checks on all translations previously stored in Pontoon.

Now that we had the data about failing checks, we started integrating them into the localizers’ workflow, so they could be fixed easily. We introduced two new translation statuses: Errors for translations failing checks from the stricter group, and Warnings for translations failing checks that can be bypassed.

Next, we started calculating Errors & Warnings counts and revealing them on dashboards. Shortly after corresponding filters have been implemented for quick access to strings with failing checks.

Errors (red) and warnings (orange) are designated with their respective colors in the string list. The list of failing checks appears in the editor upon openning the string.

Finally, the effort spanning multiple quarters started paying off. In the 7 days since the (silent) launch, the number of errors across all localizations went down by 42%. The number of warnings (which include false positives) decreased by 10%.

A huge shout-out to jotes, the powerhouse behind landing translation errors and warnings in Pontoon! As part of this effort he not only fixed 8 bugs with a total of 3,000 lines worth of changesets, but also designed the development process we used.

Including all localizations

Another important piece of information that made it to Pontoon recently are localizations that don’t take place in Pontoon. We introduced the ability to enable such localizations in read-only mode, so project managers can now access full project stats in dashboards and the API. Additionally, all Mozilla translations are now accessible in the Locales tab, and Translation Memory of teams that localize some of their projects outside Pontoon now also includes those.

Inversely, we removed system projects (Tutorial and Pontoon Intro) from dashboards, which means they now contain not only all, but also only the relevant stats.

Resource deadlines and priorities

The most critical piece of information coming from the tags feature is resource priority. It used to be pretty hard to discover, because it was only available in the Tags tab, rarely used by localizers. Back in July we fixed that by exposing resource priority in the Resources tab of the Localization dashboard. Similarly we also added a resource deadline column.

As a consequnce of these two changes localizers can now see resource priorities and deadlines without leaving Pontoon for the standalone web dashboard.

The road ahead

We hope these changes will increase efficiency of the Mozilla localization community and we’re happy to see numbers already started confirming that. Which is a great motivation for the road ahead of us. There are plenty of improvements we’d still like to make to our dashboards, ranging from smaller ones like adding word count to stats to bigger efforts like tracking community activity over time.

You’re welcome to submit your own idea or start hacking on one of the good first bugs.

September 28, 2018 02:35 PM

September 21, 2018

Mozilla L10n Blog

L10N Report: September Edition

Please note some of the information provided in this report may be subject to change as we are sometimes sharing information about projects that are still in early stages and are not final yet.

Welcome!

New localizers

Are you a locale leader and want us to include new members in our upcoming reports? Contact us!

New community/locales added

Javanese and Sundanese locales have been added to Firefox Rocket and are now launching in Indonesia. Congrats to these new teams and to their new localizers! Come check out the Friends of the Lion section below for more information on who they are.

New content and projects

What’s new or coming up in Firefox desktop

There are currently few strings landing in Nightly, making it a good time to catch up (if your localization is behind), or test your work. Focus in particular on certificate errors and preferences, given the amount of changes happening around Content blocking, Privacy and Security. The deadline to ship any localization update in Beta (Firefox 63) is October 9.

It’s been a while since we talked about Fluent, but that doesn’t mean that the migration is not progressing. Expect a lot of files moving to Fluent in the coming weeks, thanks to the collaboration with students from the Michigan University.

What’s new or coming up in mobile

Firefox Focus for Android is launching a new version this week, and it’s shipping with 9 new locales: Aymara (ay), Galician (gl), Huastec (hus), Marathi (mr), Punjabi (pa-IN), Náhuat Pipil (ppl), K’iche’ (quc), Sundanese (su), Yucatec Maya (yua).

What’s new or coming up in web projects

Activate Mozilla

The Activate Mozilla campaign aims at the grassroots of volunteer contributions. The initiative wants to bring more clarity on what are the most important areas to contribute to at Mozilla right now by providing guidance to mobilizers on how to recruit contributors and create community around meaningful Mozilla projects.

The project is added to Pontoon in a few required languages, with opt-in option for other languages. Once the completion reaches 95%, the locale will be enabled on production. There is no staging server, and any changes in Pontoon will be synced up and pushed to production directly.

Mozilla.org

Mozilla.org has a new tracking protection tour that highlights the content blocking feature. The update is ready for localization, and the tour will be available on production in early October.

Common Voice

The new home page was launched earlier this month. It is only available for 50% of the users during the A/B testing phase. The new look will roll out to all users in the next sprint. The purpose of this redesign is to convert more visitors to the site to contribute. We hope the new looking will help improve the conversion rate we currently have, which is between 10-15%. Though the change doesn’t increase more localization work at string level, the team believes it will bring more people to donating their voices to established locales.

If your language is not available on the current site, the best way to make a request is through Pontoon by following these step-by-step instructions. To learn more about Common Voice project and discussions, check them out on Discourse.

What’s new or coming up in Foundation projects

A very quick update on the fundraising campaign: the September fundraiser is being sent out in English and localization will start very soon

On the Advocacy side, the copyright campaign will continue after the unfortunate vote from the EU Parliament and the next steps are being discussed. The final vote is scheduled towards the end of year.

Stay tuned!

What’s new or coming up in Support

What’s new or coming up in Pontoon

Guided tour. Another product of Google Summer of Code is a guided tour of Pontoon, designed and developed by an experienced Pontoon contributor Vishal Sharma. It’s linked from the homepage as a secondary call to action, and consists of two pieces: the actual tour, which explains the translation user interface, and the tutorial project, which demonstrates more details through carefully chosen strings.

 

Pontoon tips

Here’s a quick Pontoon tip that a lot of people already know, but can still help some.

Pontoon has a feature that automatically identifies specific elements in strings, highlights them and makes them clickable. Here’s an example:

This feature is meant to allow you to easily copy placeables into your translation by clicking them. This saves you time and reduces the risk of introducing typos if you manually rewrite them, or partially select them while copy/pasting them.

One common misconception is to think those elements should always be kept in English. While it’s certainly true in multiple cases (variables, HTML tags like in the screenshot above…), there are several places where Pontoon highlights parts of a string that could or should be translated.

Here’s an example where all the highlighted elements should be translated:

Here Pontoon thinks those words are acronyms, and that you could potentially keep them in your translation. It turns out here they are not acronyms, it’s just a sentence in full caps, so we can simply ignore the highlights and translate it like any other string.

Here’s a last example where Pontoon successfully detects an acronym, and it could have been kept but the localizer decided to translate it anyway (and it’s okay):

To summarize the feature, Pontoon does its best to guess what parts of a string you are likely to keep in your translation, but these are suggestions only.

Also remember, you’re not alone! If you have a doubt, you can always reach out to the l10n PM owning the project. They will clarify the context for you and help you better identify false positives.

Events

Friends of the Lion

Know someone in your l10n community who’s been doing a great job and should appear here? Contact on of the l10n-drivers and we’ll make sure they get a shout-out (see list at the bottom)!

Useful Links

Questions? Want to get involved?

Did you enjoy reading this report? Let us know how we can improve by reaching out to any one of the l10n-drivers listed above.

September 21, 2018 12:09 PM

August 14, 2018

Mozilla L10n Blog

L10N Report: August Edition

Please note some of the information provided in this report may be subject to change as we are sometimes sharing information about projects that are still in early stages and are not final yet.

Welcome!

After a quick pause in July, your primary source of localization information at Mozilla is back!

New content and projects

What’s new or coming up in Firefox desktop

As localization drivers, we’re currently working on rethinking and improving the experience of multilingual users in Firefox. While this is a project that will span through several releases of Firefox, the first part of this work already landed in Nightly (Firefox 63): it’s a language switcher in Preferences, hidden behind the intl.multilingual.enabled preference, that currently allows to switch to another language already installed on the system (via language packs).

The next step will be to allow installing a language pack directly from Preferences (for the release version), and install dictionaries when user chooses to add a new language. For that reason, we’re creating a list of dictionaries for each locale. For more details, and discover how you can help, read this thread on dev-l10n.

We’re also working on building a list of native language names to use in the language switcher; once again, check dev-l10n for more info.

Quite a few strings landed in the past weeks for Nightly:

What’s new or coming up in mobile

It’s summer time in the western hemisphere, which means many projects (and people!) are taking a break – which also means not many strings are expected to land in mobile land during this period.

One notable thing is that Firefox iOS v13 was just released, and Marathi is a new locale this time around. Congratulations to the team.

On Firefox for Android front, Bosnian (bs), Occitan (oc) and Triqui (trs) are new locales that shipped with on current release version, v61. And we just added English from Canada (en-CA) and Ligurian (lij) to our Nightly v63 multi-locale build, which is available through the Google Play Store. Congratulations to everyone!

Other than that, most mobile projects are on a bit of a hiatus for the rest of the month. However, do expect some new and exciting projects to come in the pipeline over the course of the next few weeks. Stay tuned for more information!

What’s new or coming up in web projects

AMO

About two weeks ago, over 160 sets of curated add-on titles and descriptions were landed in Pontoon. Once localized, they will be included in a Shield Study to be launched on August 13. The study will run for about 2 months. This is probably the largest and longest study the AMO team has conducted.

The current Disco Pane (about:addons) lists curated extensions and themes which are manually programmed. TAAR (Telemetry Aware Add-on Recommender) is a new machine-learning extension discovery system that makes personalized recommendations based on information available in Firefox standard Telemetry. Based on TAAR’s potential to enhance content discovery by surfacing more diversified and personalized recommendations, the team wants to integrate TAAR as a product feature of Disco Pane.  It’s called “Disco-TAAR”.

The localized titles and description will increase users’ likelihood to install and install more than one. To be part of this study, you need to make sure your locale has completed at least 80% of the AMO strings by August 12.

Common Voice

Like many of you, the team is taking a summer break. However, when they come back, they promise to introduce a new home page at the beginning of next month. There should be no localization impact.

There are three ways to contributing to this project:

  1. Web part (through Pontoon)
  2. Sentence collection
  3. Recording

We now have 70 locales showing interest in the project. Many have reached 100% completion or close to it. Congratulations to reaching the first milestones. However, your work shouldn’t stop here. The sentence collection is a major challenge that all the teams face before the fun recording part can begin. Ruben Martin from the Open Innovation team addresses the challenges in this blog post. If you want to learn more about Common Voice project, sign up to Discourse where lots of discussions take place.

What’s new or coming up in Foundation projects

August fundraising emails will be sent to English audience only, the team realizes a lot of people, especially Europeans are gone on holidays and won’t be reading emails. A lot of localizers should be away as well, so they decided it was best to skip this email and focus on September fundraiser.

The Internet Health Report team has started working on next year’s report and is planning to send a localized email in French, German and Spanish to collect examples of projects that improve the health of the internet, links to great new research studies or ideas for topics they should include in the upcoming report.

As for the localized campaign work, it is slowing down in August for several reasons, one of them being an ongoing process to hire two new team members to expand the advocacy campaigns in Europe: a campaign manager, and a partnership organizer. If you know potential candidates, that would be great if you could forward them these offers!

That being said, you can expect some movement on the Copyright campaign towards the end of the month as the next vote is currently scheduled on September 12th.

What’s new or coming up in Pontoon

File priorities and deadlines

The most critical piece of information coming from the tags feature is file priority. It used to be pretty hard to discover, because it was only available in the rarely used Tags tab. We fixed that by exposing file priority in the Resources tab of the Localization dashboard. Similarly, we also added a deadline column to the localization dashboard.

Read-only locales

Pontoon introduced the ability to enable locales for projects in read-only mode. They act the same as regular (read-write) locales, except that users cannot make any edits through Pontoon (by submitting or reviewing translations or suggestions, uploading files or performing batch actions).

That allows us to access translations from locales that do not use Pontoon for localization of some of their projects, which brings a handful of benefits. For example, dashboards and the API will now present full project status across all locales, all Mozilla translations will be accessible in the Locales tab (which is not the case for localizations like the Italian Firefox at the moment) and the Translation Memory of locales currently not available in Pontoon will improve.

Expect more details about this feature as soon as we start enabling read-only locales.

Translation display improvements

Thanks to a patch by Vishal Sharma, translations status in the History tab is now also shown to unauthenticated users and non-Translators. We’ve also removed rejected suggestions from the string list and the editor.

Events

Accomplishments

Common Voice

Many communities have made significant progress in the third step: donating their voices. In July, recording were made in the following languages and more.

For the complete list of all the languages, check the language status dashboard.

Useful Links

Questions? Want to get involved?

 

Did you enjoy reading this report? Let us know how we can improve by reaching out to any one of the l10n-drivers listed above.

August 14, 2018 12:16 PM

August 07, 2018

Mitchell Baker

In Memoriam: Gervase Markham

Gerv was Mozilla’s first intern.  He arrived in the summer of 2001, when Mozilla staff was still AOL employees.  It was a shock that AOL had allocated an intern to the then-tiny Mozilla team, and we knew instantly that our amazingly effective volunteer in the UK would be our choice.

When Gerv arrived a few things about him jumped out immediately.  The first was a swollen, shiny, bright pink scar on the side of his neck.  He quickly volunteered that the scar was from a set of surgeries for his recently discovered cancer.  At the time Gerv was 20 or so, and had less than a 50% chance of reaching 35.  He was remarkably upbeat.

The second thing that immediately became clear was Gerv’s faith, which was the bedrock of his response to his cancer.  As a result the scar was a visual marker that led straight to a discussion of faith. This was the organizing principle of Gerv’s life, and nearly everything he did followed from his interpretation of how he should express his faith.

Eventually Gerv felt called to live his faith by publicly judging others in politely stated but damning terms.  His contributions to expanding the Mozilla community would eventually become shadowed by behaviors that made it more difficult for people to participate.  But in 2001 all of this was far in the future.

Gerv was a wildly active and effective contributor almost from the moment he chose Mozilla as his university-era open source project.  He started as a volunteer in January 2000, doing QA for early Gecko builds in return for plushies, including an early program called the Gecko BugAThon.  (With gratitude to the Internet Archive for its work archiving digital history and making it publicly available.)

Gerv had many roles over the years, from volunteer to mostly-volunteer to part-time, to full-time, and back again.  When he went back to student life to attend Bible College, he worked a few hours a week, and many more during breaks.  In 2009 or so, he became a full time employee and remained one until early 2018 when it became clear his cancer was entering a new and final stage.

Gerv’s work varied over the years.  After his start in QA, Gerv did trademark work, a ton of FLOSS licensing work, supported Thunderbird, supported Bugzilla, Certificate Authority work, policy work and set up the MOSS grant program, to name a few areas. Gerv had a remarkable ability to get things done.  In the early years, Gerv was also an active ambassador for Mozilla, and many Mozillians found their way into the project during this period because of Gerv.

Gerv’s work life was interspersed with a series of surgeries and radiation as new tumors appeared. Gerv would methodically inform everyone he would be away for a few weeks, and we would know he had some sort of major treatment coming up.

Gerv’s default approach was to see things in binary terms — yes or no, black or white, on or off, one or zero.  Over the years I worked with him to moderate this trait so that he could better appreciate nuance and the many “gray” areas on complex topics.  Gerv challenged me, infuriated me, impressed me, enraged me, surprised me.  He developed a greater ability to work with ambiguity, which impressed me.

Gerv’s faith did not have ambiguity at least none that I ever saw.  Gerv was crisp.  He had very precise views about marriage, sex, gender and related topics.  He was adamant that his interpretation was correct, and that his interpretation should be encoded into law.  These views made their way into the Mozilla environment.  They have been traumatic and damaging, both to individuals and to Mozilla overall.

The last time I saw Gerv was at FOSDEM, Feb 3 and 4.   I had seen Gerv only a few months before in December and I was shocked at the change in those few months.  Gerv must have been feeling quite poorly, since his announcement about preparing for the end was made on Feb 16.  In many ways, FOSDEM is a fitting final event for Gerv — free software, in the heart of Europe, where impassioned volunteer communities build FLOSS projects together.

To memorialize Gerv’s passing, it is fitting that we remember all of Gerv —  the full person, good and bad, the damage and trauma he caused, as well as his many positive contributions.   Any other view is sentimental.  We should be clear-eyed, acknowledge the problems, and appreciate the positive contributions.  Gerv came to Mozilla long before we were successful or had much to offer besides our goals and our open source foundations.  As Gerv put it, he’s gone home now, leaving untold memories around the FLOSS world.

August 07, 2018 06:19 PM

August 03, 2018

Mozilla L10n Blog

intl_pluralrules: A Rust Crate for Handling Plural Forms with CLDR Plural Rules

intl_pluralrules is a Rust crate, built to handle pluralization. Pluralization is the foundation for all localization and many internationalization APIs. With the addition of intl_pluralrules, any locale-aware date-, time- or unit-formatting (“1 second” vs “2 seconds”) and many other pluralization-dependent APIs can be added to Rust.

Rust joins the family of mature languages such as C++ and Java (via ICU), JavaScript (via ECMA 402), and Python (via Babel) which make writing multilingual software possible. All of those APIs use the same unified database for storing international plural rules, called Unicode CLDR.

intl_pluralrules determines the CLDR plural category for numeric input by leveraging Unicode Language Plural Rules and plural rules from Unicode CLDR. That category can be used to identify the correct string variant that should be used in localization.

The crate is available on crates.io, and can be used as a library in any Rust program. For example, the Rust implementation of the Fluent Project is the first system using intl_pluralrules for handling pluralization.

Why care about plurals?

In short, numbers can change the way words appear in a string. In a simple example, the English words “page” and “pages” are different because of pluralization. English plural forms are fairly simple; in other languages, the rules can be more complex. If software is built with one plural paradigm in mind, localizing for several languages (each with its own unique paradigm) becomes a complicated process–and unnecessarily so.

Flod’s blog post on the advantages of Fluent describes the problems and potential benefits intl_pluralrules crate addresses in his section, Plural Forms. If you would like to know more about why crates like intl_pluralrules are vital to i18n and l10n in Rust, start with Flod’s post.

How intl_pluralrules does plurals

The intl_pluralrules crate accepts numeric input and produces the appropriate plural category for that input in a given locale. In completing this process, intl_pluralrules performs several steps outlined here:

Making plural operands from numbers

Unicode’s Language Plural Rules provides a defining set of characteristics that affect the plural category the number belongs to in certain languages. These characteristics are called operands. The set comprises an absolute value, integer value, fraction value with and without trailing zeros, and the number of fraction digits with and without trailing zeros.

The intl_pluralrules crate creates a set of operands from the input and uses those operands when determining the plural category.

Notice the difference between the v value (number of visible fraction digits) for 1 and 1.0. Although 1 and 1.0 represent the same literal value (both represent a singular value), the presence of a decimal can change the plural category in some languages.

For example, although in English it is unlikely to see whole count nouns measured in float style (with a trailing zero decimal), the correct plural form for 1.0 is “other”, not “one.” This means that the pages example from the previous section would read, “You have 1.0 open pages” rather than “You have 1.0 open page.” This distinction may seem strange because, as mentioned, this is an unusual use for 1.0 in English. Nonetheless, you will find that it is the proper plural form.

Because Rust’s float types do not preserve trailing zeros when stringified, the from method uses Rust’s ToString trait on its input when generating plural operands. This allows a user of the crate to send string or numeric input to the system, and, so long as it is a valid float or integer value, it will be accepted.

CLDR resource parser and code generator

intl_pluralrules depends on two associated crates that reside in the same GitHub repository and are also available on crates.io: cldr_pluralrules_parser and make_pluralrules.

cldr_pluralrules_parser parses the plural rules from the JSON CLDR repository and builds an AST representation of the rules. The following code snippet shows the English and Russian plural rules from CLDR.

make_pluralrules generates a Rust file from that AST. The following code snippet shows the generated Rust plural logic for English.

The Rust file generated by these crates is used in intl_pluralrules to determine the plural form of a number.

intl_pluralrules crate

Using intl_pluralrules is a two-step process.

  1. The user must create an IntlPluralRule instance by providing a BCP 47 language tag* and a plural type (cardinal or ordinal) to the create method. This will use the generated Rust file to create an IntlPluralRule object.
  2. A number needs to be passed to the select method on the IntlPluralRule instance created in step 1.

The value returned from step 2 is the plural category.

*You can use the get_locales method to see what languages are available in the crate

Performance and implications

First, intl_pluralrules has landed in fluent-rs, meaning that the Rust implementation uses the crate for handling all plural-concerned instances. Because intl_pluralrules leverages the available data from Unicode, Fluent’s selection process for plural-concerned strings in any FTL file is completely automated. So long as the provided CLDR file has rules for your locale, the developer will not need to hard code plural logic into the software and localizers won’t need to report a bug in order for the correct plural string to be activated.

Second, intl_pluralrules is fast. The crate is still in prerelease because, although fully functional, some optimization features are still being discussed. In spite of intl_pluralrules’ WIP status regarding optimization, the system is still incredibly performant. Compared to ICU’s C PluralRules, intl_pluralrules is approximately 20 times faster in a simple benchmark test.

Intl_pluralrules’ comparative speed is due to the decision to store plural rules as compiled Rust code, rather than as CLDR syntax to be parsed at runtime. Using cldr_pluralrules_parser and make_pluralrules to generate the Rust version of the CLDR rules, the plural rules are compiled into the crate. This makes the crate slightly larger but also quicker because CLDR rules are not parsed at run time (as they are in ICU), which is the main source of the speed disparity. As intl_pluralrules moves towards 1.0, it is expected that performance will only increase.

In the bigger picture, the release of intl_pluralrules means that the Rust ecosystem gains a higher-level internationalization and localization API, hopefully the first of several. Conversely, the internationalization and localization ecosystem gains use of this API, which leverages the performance benefits of the Rust Language.

Relevant Links:

intl_pluralrules Developers:

August 03, 2018 11:30 PM

July 12, 2018

Axel Hecht

Localization, Translation, and Machines

TL;DR: Is there research bringing together Software Analysis and Machine Translation to yield Machine Localization of Software?

I’m Telling You, There Is No Word For ‘Yes’ Or ‘No’ In Irish

from Brendan Caldwell

The art of localizing a piece of software with a Yes button is to know what that button will do. This is an example of software UI that makes assumptions on language that hold for English, but might not for other languages. A more frequent example in both UI and languages that are affecting is piecing together text and UI controls:

In the localization tool, you’ll find each of those entries as individual strings. The localizer will recognize that they’re part of one flow, and will move fragments from the shared string to the drop-down as they need. Merely translating the individual segments is not going to be a proper localization of that piece of UI.

If we were to build a rule-based machine localization system, we’d find rules like

Now that’s rule-based, and it’d be tedious to maintain these rules. Neural Machine Translation (NMT) has all the buzz now, and Machine Learning in general. There is plenty of research that improves how NMT systems learn about the context of the sentence they’re translating. But that’s all text.

It’d be awesome if we could bring Software Analysis into the mix, and train NMT to localize software instead of translating fragments.

For Firefox, could one train on English and localized DOM? For Android’s XML layout, a similar approach could work? For projects with automated screenshots, could one train on those? Is there enough software out there to successfully train a neural network?

Do you know of existing research in this direction?

July 12, 2018 01:25 PM

June 27, 2018

Mozilla L10n Blog

L10N Report: June Edition

Please note some of the information provided in this report may be subject to change as we are sometimes sharing information about projects that are still in early stages and are not final yet.

Welcome!

New localizers

Are you a locale leader and want us to include new members in our upcoming reports? Contact us!

New community/locales added

New content and projects

What’s new or coming up in Firefox desktop

Firefox 61 has been released on June 26th. This also means that Firefox 62 is in Beta, while 63 is on Nightly, and that’s where you should focus your work for localization and testing.

This development cycle will be a little longer than usual, to account for Summer holidays in the Northern hemisphere: the deadline to provide your localization for Beta is August 21st, with Firefox 62 planned for release on September 5th.That’s a Wednesday and not a Tuesday, which is the common release day for Mozilla products, because September 3rd is a public holiday in US (Labor Day).

Now, diving into updates and new content. There are several new in-product pages:

One important note about Activity Stream (about:newtab and about:welcome): unlike other strings in Firefox, once translated they are not used directly in the following Nightly, they need a new build of Activity Stream (for now). That normally happens once a week. For this reason, if you’re targeting Beta, make sure to prioritize localization of activity-stream/newtab.properties.

What’s new or coming up in Test Pilot

Two new Test Pilot experiments were published on the website about 3 weeks ago: Color and Side View. While the experiments themselves were not localized, new content was added to the website. Unfortunately, that happened with poor timing, resulting in English content being displayed on the website. The following cycle is longer than usual (normally they are 2 weeks long), because most people were traveling for the bi-annual Mozilla All Hands, making things worse.

We realize that this is less than ideal, and we are trying to set up a system to avoid repeating these mistakes in the future.

What’s new or coming up in mobile

As mentioned in the section about Firefox desktop, Firefox 61 has been released on June 26th, which means that Firefox for Android 61 updates will start rolling out to users progressively. Please refer to the section about Firefox desktop above since the Firefox for Android release cycle is the same, and the comments concerning localization apply as well.

We’ve also recently decided to stop updating the What’s New content for Firefox Android (due to low visibility and interest in localizing this section, and the fact that it does not affect the number of downloads – amongst other things). This means the locales that were supported by the Play Store will no longer need to localize the updated version for Beta each time we are releasing a new version. Instead, we’ve opted for displaying a generic message.

On the Firefox iOS front, English from Canada (en-CA) was added to the ever-growing list of shipping locales with the new v12. Congratulations on that! Next update (and so, new strings) is slowly creeping up, so stay tuned for more.

Focus Android locales are also continuously growing, with Afrikaans (af), Pai-pai (pai), Punjabi (pa-IN), Quechua Chanka (quy) and Aymara (ay) teams having started to localize. Note that there are public Nightlies available on the Play Store that you can test your work on. Instructions on how to do that are here.

And finally, after months of silence localization-wise, Focus iOS will get new strings soon! However at the moment, we are not opening the project back up to new locales as there is no clearly defined schedule, and we cannot guarantee by when new locales can be added once completed.

What’s new or coming up in web projects

Legal documentation:

Common Voice:

What’s new or coming up in Foundation projects

Copyright campaign! The battle to fix copyright has started a few days ago, and while we were disappointed by the JURI Committee vote, we’re now mobilizing citizens to ask a larger group of MEPs to reject Article 13 during the EU Parliament plenary on July 5th. We can still win!

In the second half of 2018, the Advocacy team will have a bigger focus on both Europe and company misbehavior around data, so we can expect more campaigns being localized. It’s also an opportunity to mobilize internal resources to plan and build localization support on the new foundation website.

If you would like to learn more, you can watch Jon Lloyd, Advocacy Campaigns Manager, during the Foundation All-Hands in Toronto talk briefly about the recent campaign wins and the strategy for the next 6 month:

On fundraising, the current plan for the coming month is to communicate an update to existing donors, do various testing around it, then send a broader fundraising ask towards end of July/beginning of August. The donate website will soon get a makeover that will fix some layout issues with currencies, and generally provide more space for localization. Which is good news!

Several foundation website will also soon get a unified navigation header to match the one on foundation.mozilla.org.

What’s new or coming up in Pontoon

Making unreviewed suggestions discoverable

Reviewing pending suggestions regularly is important and making them discoverable is the first step to get there. That’s why we got rid of the misleading Suggested count, which didn’t include suggestions to Translated strings, and started exposing Unreviewed suggestions in a new sortable column in dashboards. It’s represented by a lightbulb, which is painted blue if unreviewed suggestions are present.

Tags help you prioritize your work

To help you prioritize your work, we rolled out Tags. The idea is pretty simple: we define a set of tags with set priority, which are then assigned to translation resources (files). Effectively, that assigns priority to each string. Currently, tags are only enabled for Firefox, which you’ll notice by the Tags tab available in the Firefox localization dashboard and filters.

Pontoon Tools 3.2.0

Pontoon Tools is a must-have add-on for all Pontoon users, which allows you to stay up-to-date with localization activity even when you don’t use Pontoon. Michal Stanke just released a new version, which is now also available for Chrome and Chromium-based browsers. Additionally, it also brings support for the aforementioned Unreviewed state and enables system notifications by default.

Support for localizing WebExtensions

Extensions for Firefox are built using the WebExtensions API, a cross-browser system for developing extensions. The WebExtensions API has a rather handy module available for internationalizing extensions – i18n. It stores translations in messages.json files, which are now supported in Pontoon. For more technical details on internationalizing WebExtensions, see this MDN page.

Redesigning Pontoon homepage

As mentioned in the last month’s report, Pramit Singhi is working on Pontoon homepage redesign as part of the Google Summer of Code project. Based on the impressive amount of feedback collected during the research among Pontoon users, he came up with a proposal of the new homepage and would like to hear your thoughts. Please consider the proposal a wireframe, so he’s more interested in hearing what you think about the overall page structure and content and less so how you like fonts and colors.

Events

Friends of the Lion

Image by Elio Qoshi

Francis who speaks many languages, joined the l10n community not long ago, thanks to the Common Voice project. He has been actively involved in bringing new contributors and introduce new localization communities to Mozilla. He also makes sure the new contributors have a simple onboarding process so they can contribute right away and see the fruit of their work quickly. Additionally, Francis files issues and fixes bugs through the project on GitHub. Thank you!

Know someone in your l10n community who’s been doing a great job and should appear here? Contact one of the l10n-drivers and we’ll make sure they get a shout-out (see list at the bottom)!

Useful Links

Questions? Want to get involved?

 

Did you enjoy reading this report? Let us know how we can improve by reaching out to any one of the l10n-drivers listed above.

June 27, 2018 07:39 PM

June 25, 2018

Mozilla L10n Blog

Making unreviewed suggestions discoverable

Last year we introduced a new review process in Pontoon with the ability to explicitly reject suggestions. That allowed us to accurately filter out suggestions in need of a review from approved and rejected ones. For more details, check out this blog post.

Today, we’re shipping the final piece of the puzzle to make unreviewed suggestions truly discoverable: we’re exposing them on dashboards. We’re also getting rid of the misleading Suggested count, which didn’t include suggestions to Translated strings. Suggested strings will be treated as Missing from now on – as they are in the product.

To make unreviewed suggestions stand out even more, we’re also introducing a new sortable column in dashboards. It’s represented by a lightbulb, which will be painted blue if unreviewed suggestions are present.

Unreviewed suggestions are represented by a lightbulb

A huge shout-out to Adrian Gaudebert, who contributed this important feature. Reviewing pending suggestions regularly is important and making them discoverable is the first step to get there.

June 25, 2018 11:03 AM

May 31, 2018

Mozilla L10n Blog

The importance of reviewing suggestions

While at Mozilla we want to ensure consistent and high-quality translations, we also want to make sure that contributing is a rewarding and pleasant experience for everyone. Translating in a timely manner is important, however there are other essential things to take into consideration. For example, leaving non-urgent projects with missing strings so that new localizers can get involved is one of them. Reviewing pending suggestions regularly is another – and the main topic of this post.

In the last few months, the l10n-drivers Project Management group (Delphine, Peiying, Flod, Jeff, and Théo) has taken a good look at the state of pending suggestions across teams and projects in Pontoon, and has decided to take more direct action in order to ensure that localizers (newcomers and old timers alike) get timely and constructive reviews. In fact, a lack of timely replies – or worst, a total lack of replies – creates an unwelcoming environment and risks losing contributors.

We started reaching out to some teams individually – but given the amount of locales concerned, it seems a broader “call to action” at this moment is a better way to kick things out.

As a first step, we’d like communities that have more than 300 pending suggestions to go through these and take some time to bring that number down. In fact, our goal this year is not only to get a better picture of the health of the localization communities, but also to take action in order to ensure our localization communities are thriving and healthy. This is just one of many steps to get there.

The reality is that when a locale has hundreds or thousands of unreviewed suggestions, and that remain unreviewed for months, it creates an unwelcoming environment for newcomers who could have a high impact on the project with some mentorship. Seeing these high numbers will make new potential localizers think that no one is paying attention, that they will not be able to get involved effectively and that they won’t be able to make an impact. In addition to being unwelcoming, this slowly creates a closed localization community, as the lack of reviews and transparency restrict involvement to only a handful of localizers (or in some cases, one localizer alone).

In the case of teams that have thousands of unreviewed strings, one way that they can start thinking about this is if there are any projects that have become less important, that they are less passionate about, and that they do not want to work on anymore. In this case, they can simply reach out to the Project Manager in charge in order to discuss the situation. The Project Manager can also easily be found on the Pontoon UI under each project page, next to the “Contact Person” field. If still in doubt, simply reach out to the dev-l10n mailing list and we will gladly respond from there.

So, what are our next steps here? Starting in July, we will reach out to teams that still have more than 300 pending suggestions and ask that they bring that number down. In lack of response, or if no action on pending suggestions is taken, we will carefully evaluate which contributors to give more permissions to and assign translator rights to new particularly active localizers. We will also talk with them on how to coordinate the localization activity for the locale.

So this is a call to action for every locale that has pending suggestions: please help us ensure that we all have a great experience in localization at Mozilla, by not only welcoming and onboarding newcomers in your team – but also making sure everyone gets a timely review of their work.

Pontoon documentation explains what a healthy localization workflow can look like, and we invite you to take a look at it here.

May 31, 2018 10:01 PM

May 23, 2018

Mozilla L10n Blog

Tags are now available in Pontoon to help you prioritize your work

Almost a couple of years ago I started working on a concept called string tiers. The goal was twofold: on one side help locales, especially those starting from scratch, to prioritize their work on a project as large as Firefox, with currently over 11 thousand strings. On the other hand, give project managers a better understanding of the current status of localization.

Given the growth in complexity and update frequency of Developer Tools within Firefox (currently almost 2,600 strings), finding a solution to this problem became more urgent. For example, is a locale in bad shape because it misses thousands of strings? The answer would not automatically be ”yes”, since the missing strings might have a low priority.

The string tiers concept assigns priority to strings based on their target – who is meant to see them – and their visibility. The idea is quite simple: a string warning the user about an error, or requiring an action from them, is more important than one targeting developers or website owners, and buried in the Error Console of the browser.

If you’re interested in knowing more about string tiers, the full document is available here.

In the past few months the Pontoon team – Ryan and Matjaz in particular – has been working hard to implement the back-end for supporting string tiers. Since the system can be used for more than just priority, we decided to use a more generic term: tags. Translation resources – files in the case of Firefox, not individual strings – can be associated to one or more tags.

A dashboard is available in each localization page, for example for the German localization of Firefox:

At a glance you have access to all the information you need: the priority associated to a tag, its translation status and latest activity. You can use the progress bar to access the strings, for example to translate missing strings or review suggestions, but note that tags are also available in filters.

Pontoon’s documentation is already up to date with information about tags.

Project managers can now see the overall translation status of each tag, but also a breakdown with the status of each locale for a specific tag.

This is a brand new feature, and required a lot of code changes in Pontoon. If you find bugs, or want to add features, feel free to file a bug.

May 23, 2018 03:29 PM

May 09, 2018

Mozilla L10n Blog

L10N Report: May Edition

Please note some of the information provided in this report may be subject to change as we are sometimes sharing information about projects that are still in early stages and are not final yet.

Welcome!

New localizers

Are you a locale leader and want us to include new members in our upcoming reports? Contact us!

New community/locales added

New content and projects

What’s new or coming up in Firefox desktop

Activity Stream

Activity Stream has become an integral part of Firefox, officially replacing the existing New Tab and soon integrating code for displaying snippets and onboarding content. For this reason, we’re working on moving translations to mozilla-central.

Currently, Activity Stream is managed as a stand-alone project in Pontoon, and store its translations in a GitHub repository. Once this meta bug is fixed, Activity Stream’s strings will be exposed as part of the Firefox project.

While this makes the relation between Activity Stream and Firefox more obvious for localizers, it will also allow to make some improvements in the future, like reducing the lag between translations landing in repositories and actually being available for testing in Firefox.

Release schedule

There are some changes in the release schedule for the Summer, with the Nightly cycle for 63 lasting 10 weeks instead of 8, and the effects of this change rippling to the end of year.

Does this impact you as a localizer? Luckily, not really, thanks to cross-channel. Just don’t forget to keep an eye on the deadlines in Pontoon for the Firefox project, and you’ll be all set.

If you’re curious, you can always check the official Release Calendar.

Fluent migrations

We were waiting for the 61 Nightly cycle to end before resuming migrations. The first bug on the schedule for 62 landed on Tuesday and migrated almost 30 strings used in Privacy related permission dialogs (camera, location, pop-ups, etc.), with a second one already in review (over 20 strings, used in JavaScript across subdialogs in Preferences).

Shield studies

Shield studies are A/B tests, deployed to a specific portion of Firefox users, designed to test features or changes and analyze their impact.

So far, these studies have been targeting only English users (with one exception), and didn’t require localization. This week we started localizing our first study: it’s an extension that will be enabled initially only for Nightly, and for a limited number of locales (4) that have a significant population on the Nightly channel. We will also expose the opt-out preference for Shield studies, together with the about:studies page, to all languages in Firefox in the coming days.

If you’re interested in Shield studies, you can find more details in this wiki page.

What’s new or coming up in mobile

There’s been quite a few things going on in mobile world recently, as you’ve probably noticed.

We’ve recently exposed strings for the upcoming release of Firefox iOS v12, that aims to launch around June 12th. The tentative deadline for localizing and testing is May 23rd. Screenshots are updated twice a week.

Firefox for Amazon Fire TV is shipping an update very soon, probably some time next week. We’ll be getting updated screenshots for testing very soon, stay tuned!

Same for Focus Android v5 that is coming up, and will feature three new locales: Angika (anp), Cornish (kw) and Purépecha (tsz). Congrats to the localizers involved in this effort!

Firefox for Android is updating to v60 as we speak, and is being progressively rolled out to users.

What’s new or coming up in web projects

Common Voice was launched on May 2nd in 29 languages. Thanks to many communities’ active participation to get to the finish line, especially with quite a bit of last minute changes. More languages will be added as they reach the completion criteria.

The team has been continually impressed with the thoroughness of communities around the globe down to the wire, to test, to identify issues, and they have not stopped just because the product or their language is launched. The bugs keep coming, not just for translation but for site stability and UX issues. Here is one example.

Currently, only the website is localizable. The next step is to start recording voices in new languages. To that end, we need sentences for people to read in these new languages. The Common Voice team is participating the Mozilla’s Global Sprint to help collect and review new sentences for people to read. Only after that, the team will start collecting voice data in a new language.

What’s new or coming up in Foundation projects

On the advocacy and fundraising side, we’ve had an opportunity to respond to the Clear History feature announced by Facebook during the F8 conference, but the team is mostly focusing on the long term strategy on how to move the conversation to the broader data and privacy topic.

What’s new or coming up in Pontoon

We’re happy to announce that Vishal Sharma and Pramit Singhi will be joining us for the Google Summer of Code project this year! They will help us improve the path to first contribution, including the homepage redesign and guiding new users to submitting their first translation. That should help us grow l10n communities.

Pontoon now runs compare-locales checks when you submit a translation – the same library that is used during the Firefox build process. That allows Pontoon to prevent you from submitting translations that can break Firefox. Read more about the updates we made to the way quality checks work.

Pontoon now allows you to translate strings across all projects in a single translation interface, which helps you work on all untranslated strings in one go, or review all pending suggestions submitted by your team contributors. We also changed the way search works by bringing it closer to the default behaviour of search engines.

More relevant communications for Engagement projects

https://irccloud.mozilla.com/file/Amhzza0T/Schermata%202018-05-09%20alle%2012.28.53.png

We realized that several communications we send to the dev.l10n.web mailing list concern only a small subset of locales, and are meant to be ignored by the vast majority of people. In order to keep the signal-to-noise ratio low, we are changing the way we communicate project updates like snippets and newsletters. From now on, we will try to rely as much as possible on Pontoon built-in notification system, allowing us to send notifications only to existing contributors in the target locales. This way, communications in dev.l10n.web should be more relevant for everyone.

This also means people contributing to Engagement projects won’t get notification directly in their inbox, but using the Pontoon Tools add-on from Michal Stanke will get you almost real-time notifications in your browser. If you’re usually contributing to those projects, please keep an eye on Pontoon notifications if you don’t want to miss schedule, deadline or context about new content.

Events

If you’re planning to meet your fellow teammates this year, or planning an event with other locales, make sure to check our plans for l10n meetups in 2018 and the process to organize meetups. We can’t wait to see the meetup ideas you may have!

Localization impact by the numbers

Snippets

Friends of the Lion

Image by Elio Qoshi

 

Know someone in your l10n community who’s been doing a great job and should appear here? Contact on of the l10n-drivers and we’ll make sure they get a shout-out (see list at the bottom)!

Useful Links

Questions? Want to get involved?

Did you enjoy reading this report? Let us know how we can improve by reaching out to any one of the l10n-drivers listed above.

May 09, 2018 09:55 PM

May 08, 2018

Mozilla L10n Blog

Making it hard to break Firefox from Pontoon

When submitting a translation, Pontoon runs automated quality checks. They identify issues with punctuation, capitalization, variables, etc. before translations are saved, which prevents localizers from submitting bad translations. We just launched several improvements to the way quality checks work, so let’s have a look.

Using compare-locales checks in Pontoon

In addition to internal Pontoon checks and Translate Toolkit checks, Pontoon now also runs compare-locales checks. They are used when building Firefox, so it’s much harder now for localizers to break Firefox builds by submitting a bad translation through Pontoon.

Errors and warnings

There are two types of quality check failures now: errors and warnings.

Errors cover issues that would cause the translation to be unused in product, for example removed from Firefox builds or mozilla.org. For this reason, errors cannot be bypassed by localizers – the button to submit a translation is removed and the error needs to be fixed before the translation can be saved.

Errors are denoted with a red circled X

Errors are denoted with a red circled X.

Warnings, unlike errors, are displayed when we’re not completely sure that the string contains critical issues (false positives). For that reason, warnings can be bypassed by localizers – they can save a translation anyway.

Note: since Translate Toolkit checks may result in many false positives in some scenarios, they can be turned off using the Translate Toolkit Checks toggle (previously called Quality Checks).

Next steps

Thumbs up to Jotes for turning this long awaited feature into a reality! He’s already busy with the next step, which is storing information about the failed checks in a database. Once that’s in place, we’ll start performing quality checks during sync and finally present the information about errors and warnings on dashboards.

May 08, 2018 07:35 PM

September 16, 2017

Mitchell Baker

Busting the myth that net neutrality hampers investment

This week I had the opportunity to share Mozilla’s vision for an Internet that is open and accessible to all with the audience at MWC Americas.

I took this opportunity because we are at a pivotal point in the debate between the FCC, companies, and users over the FCC’s proposal to roll back protections for net neutrality. Net neutrality is a key part of ensuring freedom of choice to access content and services for consumers.

Earlier this week Mozilla’s Heather West wrote a letter to FCC Chairman Ajit Pai highlighting how net neutrality has fueled innovation in Silicon Valley and can do so still across the United States.

The FCC claims these protections hamper investment and are bad for business. And they may vote to end them as early as October. Chairman Pai calls his rule rollback “restoring internet freedom” but that’s really the freedom of the 1% to make decisions that limit the rest of the population.

At Mozilla we believe the current rules provide vital protections to ensure that ISPs don’t act as gatekeepers for online content and services. Millions of people commented on the FCC docket, including those who commented through Mozilla’s portal that removing these core protections will hurt consumers and small businesses alike.

Mozilla is also very much focused on the issues preventing people coming online beyond the United States. Before addressing the situation in the U.S., journalist Rob Pegoraro asked me what we discovered in the research we recently funded in seven other countries into the impact of zero rating on Internet use:


(Video courtesy: GSMA)

If you happen to be in San Francisco on Monday 18th September please consider joining Mozilla and the Internet Archive for a special night: The Battle to Save Net Neutrality. Tickets are available here.

You’ll be able to watch a discussion featuring former FCC Chairman Tom Wheeler; Representative Ro Khanna; Mozilla Chief Legal and Business Officer Denelle Dixon; Amy Aniobi, Supervising Producer, Insecure (HBO); Luisa Leschin, Co-Executive Producer/Head Writer, Just Add Magic (Amazon); Malkia Cyril, Executive Director of the Center for Media Justice; and Dane Jasper, CEO and Co-Founder of Sonic. The panel will be moderated by Gigi Sohn, Mozilla Tech Policy Fellow and former Counselor to Chairman Wheeler. It will discuss how net neutrality promotes democratic values, social justice and economic opportunity, what the current threats are, and what the public can do to preserve it.

September 16, 2017 04:00 AM

August 18, 2017

Mitchell Baker

Resignation as co-chair of the Digital Economy Board of Advisors

For the past year and a half I have been serving as one of two co-chairs of the U.S. Commerce Department Digital Economy Board of Advisors. The Board was appointed in March 2016 by then-Secretary of Commerce Penny Pritzer to serve a two year term. On Thursday I sent the letter below to Secretary Ross.

Dear Secretary Ross,
I am resigning from my position as a member and co-chair of the Commerce Department’s Digital Economy Board of Advisors, effective immediately.
It is the responsibility of leaders to take action and lift up each and every American. Our leaders must unequivocally denounce bigotry, racism, sexism, hate, and violence.
The digital economy is fundamental to creating an economy that offers opportunity to all Americans. It has been an honor to serve as member and co-chair of this board and to work with the Commerce Department staff.
Sincerely,
Mitchell Baker
Executive Chairwoman
Mozilla

August 18, 2017 07:12 PM

April 28, 2017

Mitchell Baker

New Mozilla Foundation Board Members: Mohamed Nanabhay and Nicole Wong

Today, I’m thrilled to announce that Mohamed Nanabhay and Nicole Wong have joined the Mozilla Foundation Board of Directors.

Over the last few years, we’ve been working to expand the boards for both the Mozilla Foundation and the Mozilla Corporation. Our goals for the Foundation board roles were to grow Mozilla’s capacity to move our mission forward; expand the number and diversity of people on our boards, and; add specific skills in areas related to movement building and organizational excellence. Adding Mohamed and Nicole represents a significant move forward on these goals.

We met Mohamed about seven years ago through former board member and then Creative Commons CEO Joi Ito. Mohamed was at Al Jazeera at the time and hosted one of Mozilla’s first Open News fellows. Mohamed Nanabhay currently serves as the Deputy CEO of the Media Development Investment Fund (MDIF), which invests in independent media around the world providing the news, information and debate that people need to build free, thriving societies.

Nicole is an attorney specializing in Internet, media and intellectual property law. She served as President Obama’s deputy chief technology officer (CTO) and has also worked as the vice president and deputy general counsel at Google to arbitrate issues of censorship. Nicole has already been active in helping Mozilla set up a new fellows program gathering people who have worked in government on progressive tech policy. That program launches in June.

Talented and dedicated people are the key to building an Internet as a global public resource that is open and accessible to all. Nicole and Mohammad bring expertise, dedication and new perspectives to Mozilla. I am honored and proud to have them as our newest Board members.

Please join me in welcoming Mohamed and Nicole to the Board. You can read more about why Mohamed chose to join the Board here, and why Nicole joined us here.

Mitchell

April 28, 2017 08:29 PM

March 23, 2017

Axel Hecht

Can’t you graph that graph

I’m going to just recreate blame, he said. It’s going to be easy, he said.

We have a project to migrate the localization of Firefox to one repository for all channels, nick-named cross-channel, or x-channel in short. The plan is to create one repository that holds all the en-US strings we need for Firefox and friends on all channels. One repository to rule them all, if you wish. So you need to get the contents of mozilla-central, comm-central, *-aurora, *-beta, *-release, and also some of *-esr?? together in one repository, with, say, one toolkit/chrome/global/customizeToolbar.dtd file that has all the strings that are used by any of the apps on any branch.

We do have some experience with merging the content of localization files as part of l10n-merge which is run at Firefox build time. So this shouldn’t be too hard, right?

Enter version control, and the fact that quite a few of our localizers are actually following the development of Firefox upstream, patch by patch. That they’re trying to find the original bug if there’s an issue or a question. So, it’d be nice to have the history and blame in the resulting repository reflect what’s going on in mozilla-central and its dozen siblings.

Can’t we just hg convert and be done with it? Sadly, that only converts one DAG into another hg DAG, and we have a dozen. We have a dozen heads, and we want a single head in the resulting repository.

Thus, I’m working on creating that repository. One side of the task is to update that target repository as we see updates to our 12 original heads. I’m pretty close to that one.

The other task is to create a good starting point. Or, good enough. Maybe if we could just create a repo that had the same blame as we have right now? Like, not the hex or integer revisions, but annotate to the right commit message etc? That’s easy, right? Well, I thought it was, and now I’m learning.

To understand the challenges here, one needs to understand the data we’re throwing at any algorithm we write, and the mercurial code that creates the actual repository.

As of FIREFOX_AURORA_45_BASE, just the blame for the localized files for Firefox and Firefox for Android includes 2597 hg revisions. And that’s not even getting CVS history, but just what’s in our usual hg repository. Also, not including comm-central in that number. If that history was linear, things would probably be pretty easy. At least, I blame the problems I see in blame on things not being linear.

So, how non-linear is that history. The first attempt is to look at the revision set with hg log -G -r .... . That creates a graph where the maximum number of parents of a single changeset is at 1465. Yikes. We can’t replay that history in the target repository, as hg commits can only have 2 parents. Also, that’s clearly not real, we’ve never had that many parallel threads of development. Looking at the underlying mercurial code, it’s showing all reachable roots as parents of a changeset, if you have a sparse graph. That is, it gives you all possible connections through the underlying full graph to the nodes in your selection. But that’s not what we’re interested in. We’re interested in the graph of just our nodes, going just through our nodes.

In a first step, I wrote code that removes all grandchildren from our parents. That reduces the maximum number of parents to 26. Much better, but still bad. At least it’s at a size where I can start to use graphviz to create actual visuals to inspect and analyze. Yes, I can graph that graph.

The resulting graph has a few features that are actually already real. mozilla-central has multiple roots. One is the initial hg import of the Firefox code. Another is including Firefox for Android in mozilla-central, which used to be an independent repository. Yet another is the merge of services/sync. And then I have two heads, which isn’t much of a problem, it’s just that their merge commit didn’t create anything to blame for, and thus doesn’t show up in my graph. Easy to get to, too.

Looking at a subset of the current graph, it’s clear that there are more arcs to remove:

Anytime you have an arc that just leap-frogs to an ancestor, you can safely remove that. I indicated some in the graph above, and you’ll find more – I was just tired of annotating in Preview. As said before, I already did that for grandchildren. Writing this post I realize that it’s probably easy enough to do it for grandgrandchildren, too. But it’s also clear from the full graph, that that algorithm probably won’t scale up. Seems I need to find a good spot at which to write an explicit loop detection.

This endeavour sounds a bit academic at first, why would you care? There are two reasons:

Blame in mercurial depends on the diff that’s stored in the backend, and the diff depends on the previous content. So replaying the blame in some way out of band doesn’t actually create the same blame. My current algorithm is to just add the final lines one by one to the files, and commit. Whitespace and reoccurring lines get all confused by that algorithm, sadly.

Also, this isn’t a one-time effort. The set of files we need to expose in the target depends on the configuration, and often we fix the configuration of Firefox l10n way after the initial landing of the files to localize. So having a sound code-base to catch up on missed history is an important step to make the update algorithm robust. Which is really important to get it run in automation.

PS: The tune for this post is “That Smell” by Lynyrd Skynyrd.

March 23, 2017 02:01 PM

March 13, 2017

Mitchell Baker

The “Worldview” of Mozilla

There are a set of topics that are important to Mozilla and to what we stand for in the world — healthy communities, global communities, multiculturalism, diversity, tolerance, inclusion, empathy, collaboration, technology for shared good and social benefit.  I spoke about them at the Mozilla All Hands in December, if you want to (re)listen to the talk you can find it here.  The sections where I talk about these things are at the beginning, and also starting at about the 14:30 minute mark.

These topics are a key aspect of Mozilla’s worldview.  However, we have not set them out officially as part of who we are, what we stand for and how we describe ourselves publicly.   I’m feeling a deep need to do so.

My goal is to develop a small set of principles about these aspects of Mozilla’s worldview. We have clear principles that Mozilla stands for topics such as security and free and open source software (principles 4 and 7 of the Manifesto).  Similarly clear principles about topic such as global communities and multiculturalism will serve us well as we go forward.  They will also give us guidance as to the scope and public voice of Mozilla, spanning official communications from Mozilla, to the unofficial ways each of us describes Mozilla.

Currently, I’m working on a first draft of the principles.  We are working quickly, as quickly as we can have rich discussions and community-wide participation. If you would like to be involved and can potentially spend some hours reviewing and providing input please sign up here. Jascha and Jane are supporting me in managing this important project.  
I’ll provide updates as we go forward.  

March 13, 2017 06:28 PM

March 03, 2017

Axel Hecht

On updating the automation behind l10n.m.o

Or, how to change everything and nobody sees a difference.

Heads up: All I’m writing about here is running on non-web-facing VMs behind VPN.

tl;dr: I changed 5 VMs, landed 76 changesets in 7 repositories, resolving 12 bugs, got two issues in docker fixed, and took a couple of days of downtime. If automation is your cup of tea, I have some open questions at the end, too.

To set the stage: Behind the scenes of the elmo website, there’s a system that generates the data that it shows. That system consists of two additional VMs, which help with the automation.

One is nick-named a10n, and is responsible for polling all those mercurial repositories that we use for l10n, and to update the elmo database with information about these repositories as it comes in. elmo basically keeps a copy of the mercurial metadata for quicker access.

The other is running buildbot to do the actual data collection jobs about the l10n status in our source repositories. This machine runs both a master and one slave (the actual workhorse, not my naming).

This latter machine is an old VM, on old OS, old Python (2.6), never had real IT support, and is all around historic. And needed to go.

With the help of IT, I had a new VM, with a new shiny python 2.7.x, and a new storage. Something that can actually run current versions of compare-locales, too. So I had to create an update for

Python 2.6 Python 2.7.x
globally installed python modules virtualenv
Django 1.4.18 Django 1.8.x
Ubuntu CentOS
Mercurial 3.7.3 Mercurial 4.0.1 and hglib
individual local clones unified local clones
No working stage docker-compose up

At the same time, we also changed hg.m.o from http to https all over the place, which also required a handful of code changes.

One thing that I did not change is buildbot. I’m using a heavily customized version of buildbot 0.7.12, which is incompatible with later buildbot changes. So I’m tied to my branch of 0.7.12 for now, and with that to Twisted 8.2.0. That will change, but in a different blog post.

Unified Repositories

One thing I wanted and needed for a long time was to use unified clones of our mercurial repositories. Aside from the obvious win in terms of disk usage, it allows to use mercurial directly to create a diff from a revision that’s only on aurora against a revision that’s only on beta. Sadly, I did think otherwise when I wrote the first parts of elmo and the automation behind it, often falling back to default instead of an actual hash revision, if I didn’t know anything ad-hoc. So that had to go, and required a surprising amount of changes. I also changed the way that comparisons are triggered, making them fully reproducible. They also got more robust. I used to run hg id -ir . to get the revision, which worked OK, unless you had extension errors in stdout/stderr. Meh. Good that that’s gone.

As I noted, the unified repositories also benefit doing diffs, which is one of the features of elmo for reviewing localizations. Now that we can just use plain mercurial to get those diffs, I could remove a bunch of code that created diffs between aurora and beta by creating diffs between each head and some ancestor, and then sticking those diffs back together. Good that that’s gone.

Testing

Testing an automation with that many moving parts is hard. Some things can be tested via unit tests, but more often, you just need integration tests. I still have to find a way to write automated integration tests, but even manual integration tests require a ton of set-up:

Doing this manually is evil, and on Macs, it’s not even possible, because Twisted 8.2.0 doesn’t build anymore. I used to have a script that did many of these things, but that’s …. you guessed it. Good that that’s gone. Now I have a docker-compose test setup, that has most things running with just a docker-compose up. I’m still running elmo and MySQL on my host machine, fixing that is for another day. Also, I haven’t found a good way to do initial project setup like database creations. Anyway, after finding a couple of bugs in docker, this system now fires up quickly and let’s me do various changes and see how they pass through the system. One particularly nice artifact is that the output of docker-compose is actually all the logs together in one stream. So as you’re pushing things through the system, you just have one log to watch.

As part of this work, I also greatly simplified the code structure, and moved the buildbot integration from three repositories into one. Good that those are gone.

snafus

Sadly there were a few bits and pieces where my local testing didn’t help:

Changing the URL schemes for hg.m.o to https alongside this change triggered a couple of problems where Twisted 8.2 and modern Python/OpenSSL can’t get a connection up. Had to replace the requests to websites with synchronous urllib2.urlopen calls.

Installing mercurial in a virtualenv to be used via hglib is good, but WSGI doesn’t activate the virtualenv, and thus PATH isn’t set. My fix still needs some server-side changes to work.

I didn’t have enough local testing for the things that Thunderbird needs. That left that setup burning for longer than I anticipated. The fix wasn’t hard, just badly timed.

Every now and then, Django 1.8.x and MySQL decide that it’s a good idea to throw away the connection, and die badly. In the case of long-running automation jobs, that’s really hard to prevent, in particular because I still haven’t fully understood what change actually made that happen, and what the right fix is. I just plaster connection.close() into every other function, and see if it stops dying.

On Saturday morning I woke up, and the automation didn’t process Firefox for a locale on aurora. I freaked out, and added tons of logging. Good logging that is. Best logging. Found a different bug. Also found out that the locale was Belarus, and that wasn’t part of the build on Saturday. Hit my head against a wall or two.

Said logging made uncaught exceptions in some parts of the code actually show up in logs, and discovered that I hadn’t tested my work against bad configurations. And we have that, Thunderbird just builds everything on central, regardless of whether the repositories it should use for that exist or not. I’m not really happy yet with the way I fixed this.

Open Questions

March 03, 2017 08:01 PM

January 31, 2017

Mitchell Baker

A Thank You to Reid Hoffman

Today I want to say thank you to Reid Hoffman for 11 years as a Mozilla Corporation board member. Reid’s normal “tour of duty” on a board is much shorter. Reid joined Mozilla as an expression of his commitment to the Open Internet and the Mozilla mission, and he’s demonstrated that regularly. Almost five years ago I asked Reid if he would remain on the Mozilla board even though he had already been a member for six years. Reid agreed. When Chris Beard joined us Reid agreed to serve another two years in order to help Chris get settled and prime Mozilla for the new era.

Mozilla is in a radically better place today than we were two, three, or five years ago, and is poised for a next phase of growth and influence. Take a look at the Annual Report we published Dec 1, 2016 to get a picture of our financial and operational health. Or look at The Glass Room, or our first  Internet Health Report, or the successful launch of Firefox Focus (or Walt Mossberg’s article about Mozilla) to see what we’ve done the last few months.

And so after an extended “tour of duty” Reid is leaving the Mozilla Corporation board and becoming an Emeritus board member. He remains a close friend and champion of Mozilla and the Open Internet. He continues to help identify technologists, entrepreneurs, and allies who would be a good fit to join Mozilla, including at the board level.  He also continues to meet with and provide support to our key executives.

A heartfelt thank you to Reid.

January 31, 2017 05:06 PM

December 05, 2016

Mitchell Baker

Helen Turvey Joins the Mozilla Foundation Board of Directors

This post was originally posted on the Mozilla.org website.

Helen Turvey, new Mozilla Foundation Board member

Helen Turvey, new Mozilla Foundation Board member

Today, we’re welcoming Helen Turvey as a new member of the Mozilla Foundation Board of Directors. Helen is the CEO of the Shuttleworth Foundation. Her focus on philanthropy and openness throughout her career makes her a great addition to our Board.

Throughout 2016, we have been focused on board development for both the Mozilla Foundation and the Mozilla Corporation boards of directors. Our recruiting efforts for board members has been geared towards building a diverse group of people who embody the values and mission that bring Mozilla to life. After extensive conversations, it is clear that Helen brings the experience, expertise and approach that we seek for the Mozilla Foundation Board.

Helen has spent the past two decades working to make philanthropy better, over half of that time working with the Shuttleworth Foundation, an organization that provides funding for people engaged in social change and helping them have a sustained impact. During her time with the Shuttleworth Foundation, Helen has driven the evolution from traditional funder to the current co-investment Fellowship model.

Helen was educated in Europe, South America and the Middle East and has 15 years of experience working with international NGOs and agencies. She is driven by the belief that openness has benefits beyond the obvious. That openness offers huge value to education, economies and communities in both the developed and developing worlds.

Helen’s contribution to Mozilla has a long history: Helen chaired the digital literacy alliance that we ran in UK in 2013 and 2014; she’s played a key role in re-imagining MozFest; and she’s been an active advisor to the Mozilla Foundation executive team during the development of the Mozilla Foundation ‘Fuel the Movement’ 3 year plan.

Please join me in welcoming Helen Turvey to the Mozilla Foundation Board of Directors.

Mitchell

You can read Helen’s message about why she’s joining Mozilla here.

Background:

Twitter: @helenturvey

High-res photo

December 05, 2016 02:30 AM

December 01, 2016

Mitchell Baker

Julie Hanna Joins the Mozilla Corporation Board of Directors

This post was originally posted on the Mozilla.org website.

Julie Hanna, new Mozilla Corporation Board member

Julie Hanna, new Mozilla Corporation Board member

Today, we are very pleased to announce the latest addition to the Mozilla Corporation Board of Directors – Julie Hanna. Julie is the Executive Chairman for Kiva and a Presidential Ambassador for Global Entrepreneurship and we couldn’t be more excited to have her joining our Board.

Throughout this year, we have been focused on board development for both the Mozilla Foundation and the Mozilla Corporation boards of directors. We envisioned a diverse group who embodied the same values and mission that Mozilla stands for. We want each person to contribute a unique point of view. After extensive conversations, it was clear to the Mozilla Corporation leadership team that Julie brings exactly the type of perspective and approach that we seek.

Born in Egypt, Julie has lived in various countries including Jordan and Lebanon before finally immigrating to the United States. Julie graduated from the University of Alabama at Birmingham with a B.S. in Computer Science. She currently serves as Executive Chairman at Kiva, a peer-peer lending pioneer and the world’s largest crowdlending marketplace for underserved entrepreneurs. During her tenure, Kiva has scaled its reach to 190+ countries and facilitated nearly $1 billion dollars in loans to 2 million people with a 97% repayment rate. U.S. President Barack Obama appointed Julie as a Presidential Ambassador for Global Entrepreneurship to help develop the next generation of entrepreneurs. In that capacity, her signature initiative has delivered over $100M in capital to nearly 300,000 women and young entrepreneurs across 86 countries.

Julie is known as a serial entrepreneur with a focus on open source. She was a founder or founding executive at several innovative technology companies directly relevant to Mozilla’s world in browsers and open source. These include Scalix, a pioneering open source email/collaboration platform and developer of the most advanced AJAX application of its time, the first enterprise portal provider 2Bridge Software, and Portola Systems, which was acquired by Netscape Communications and become Netscape Mail.

She has also built a wealth of experience as an active investor and advisor to high-growth technology companies, including sharing economy pioneer Lyft, Lending Club and online retail innovator Bonobos. Julie also serves as an advisor to Idealab, Bill Gross’ highly regarded incubator which has launched dozens of IPO-destined companies.

Please join me in welcoming Julie Hanna to the Mozilla Board of Directors.

Mitchell

Background:

Twitter: @JulesHanna

High-res photo

 

December 01, 2016 07:18 PM

September 30, 2016

Mitchell Baker

Mozilla Hosting the U.S. Commerce Department Digital Economy Board of Advisors

Today Mozilla is hosting the second meeting of the Digital Economy Board of Advisors of the United States Department of Commerce, of which I am co-chair.

Support for the global open Internet is the heart of Mozilla’s identity and strategy. We build for the digital world. We see and understand the opportunities it offers, as well as the threats to its future. We live in a world where a free and open Internet is not available to all of the world’s citizens; where trust and security online cannot be taken for granted; and where independence and innovation are thwarted by powerful interests as often as they are protected by good public policy. As I noted in my original post on being named to the Board, these challenges are central to the “Digital Economy Agenda,” and a key reason why I agreed to participate.

Department of Commerce Secretary Pritzker noted earlier this year: “we are no longer moving toward the digital economy. We have arrived.” The purpose of the Board is to advise the Commerce Department in responding to today’s new status quo. Today technology provides platforms and opportunities that enable entrepreneurs with new opportunities. Yet not everyone shares the benefits. The changing nature of work must also be better understood. And we struggle to measure these gains, making it harder to design policies that maximize them, and harder still to defend the future of our digital economy against myopic and reactionary interests.

The Digital Economy Board of Advisors was convened to explore these challenges, and provide expert advice from a range of sectors of the digital economy to the Commerce Department as it develops future policies. At today’s meeting, working groups within the Board will present their initial findings. We don’t expect to agree on everything, of course. Our goal is to draw out the shared conclusions and direction to provide a balanced, sustainable, durable basis for future Commerce Department policy processes. I will follow up with another post on this topic shortly.

Today’s meeting is a public meeting. There will be two live streams: one for the 8:30 am-12:30 pm PT pre-lunch session and one for the afternoon post-lunch 1:30-3:00pm PT. We welcome you to join us.

Although the Board has many more months left in its tenure, I can see a trend towards healthy alignment between our mission and the outcomes of the Board’s activities. I’m proud to serve as co-chair of this esteemed group of individuals.

September 30, 2016 02:10 PM

September 28, 2016

Mitchell Baker

UN High Level Panel and UN Secretary General Ban Ki-moon issue report on Women’s Economic Empowerment

“Gender equality remains the greatest human rights challenge of our time.”  UN Secretary General Ban Ki-moon, September 22, 2016.

To address this challenge the Secretary General championed the 2010 creation of UN Women, the UN’s newest entity. To focus attention on concrete actions in the economic sphere he created the “High Level Panel on Women’s Economic Empowerment” of which I am a member.

The Panel presented its initial findings and commitments last week during the UN General Assembly Session in New York. Here is the Secretary General, with the the co-chairs, and the heads of the IMF and the World Bank, the Executive Director of the UN Women, and the moderator and founder of All Africa Media, each of whom is a panel member.

UN General Assembly Session in New York

Photo Credit: Anar Simpson

The findings are set out in the Panel’s initial report. Key to the report is the identification of drivers of change, which have been deemed by the panel to enhance women’s economic empowerment:

  1. Breaking stereotypes: Tackling adverse social norms and promoting positive role models
  2. Leveling the playing field for women: Ensuring legal protection and reforming discriminatory laws and regulations
  3. Investing in care: Recognizing, reducing and redistributing unpaid work and care
  4. Ensuring a fair share of assets: Building assets—Digital, financial and property
  5. Businesses creating opportunities: Changing business culture and practice
  6. Governments creating opportunities: Improving public sector practices in employment and procurement
  7. Enhancing women’s voices: Strengthening visibility, collective voice and representation
  8. Improving sex-disaggregated data and gender analysis

Chapter Four of the report describes a range of actions that are being undertaken by Panel Members for each of the above drivers. For example under the Building assets driver: DFID and the government of Tanzania are extending land rights to more than 150,000 Tanzanian women by the end of 2017. Tanzania will use media to educate people on women’s land rights and laws pertaining to property ownership. Clearly this is a concrete action that can serve as a precedent for others.

As a panel member, Mozilla is contributing to the working on Building Assets – Digital. Here is my statement during the session in New York:

“Mozilla is honored to be a part of this Panel. Our focus is digital inclusion. We know that access to the richness of the Internet can bring huge benefits to Women’s Economic Empowerment. We are working with technology companies in Silicon Valley and beyond to identify those activities which provide additional opportunity for women. Some of those companies are with us today.

Through our work on the Panel we have identified a significant interest among technology companies in finding ways to do more. We are building a working group with these companies and the governments of Costa Rica, Tanzania and the U.A. E. to address women’s economic empowerment through technology.

We expect the period from today’s report through the March meeting to be rich with activity. The possibilities are huge and the rewards great. We are committed to an internet that is open and accessible to all.”

You can watch a recording of the UN High Level Panel on Women’s Economic Empowerment here. For my statement, view starting at: 2.07.53.

There is an immense amount of work to be done to meet the greatest human rights challenge of our time. I left the Panel’s meeting hopeful that we are on the cusp of great progress.

September 28, 2016 09:54 PM

November 24, 2014

Staś Małolepszy

Meet.js talk on Clientside localization in Firefox OS

Firefox OS required a fast and lean localization method that could scale up to 70 languages, cater to the needs of hundreds of developers worldwide all speaking different languages and support a wide spectrum of devices with challenging hardware specs.

At the end of September, I went to Poznań to speak about localization technology in Firefox OS at Meet.js Summit. In my talk I discussed how we had been able to create a localization framework which embraces new Web technologies like Web components and mutation observers, how we'd come up with new developer tools to make localization work easier and what exciting challenges lay ahead of us.

November 24, 2014 11:10 AM

October 05, 2014

Axel Hecht

Update to the l10n team page

I’ve given the team pages on l10n.mozilla.org a good whack in the past few days. Time to share some details and get feedback before I roll it out.

The gist of it: More data in less screen space, I just folded things into rows, and made the rows slimmer. Better display of sign-off status, I separated status from progress and actions. Actions are now ordered chronologically, too.

The details? Well, see my recording where I walk you through:


View it on youtube.

Comments here or in bug 1077988.

October 05, 2014 02:02 PM

June 17, 2014

Axel Hecht

Create your own dashboard

We have a lot of data around localizations, but it’s hard to know what people might be looking for.

I just switched a new feature live, edit your own dashboard.

You can select branches of products, as well as the localizations you’re interested in, and get data you want.

Say you’re looking for mobile and India. You’d want Firefox OS and Firefox for Android aka Fennec. The latter is actively localized on aurora, so you’d want the gaia tree and fennec_aurora. You want Assamese, Bengali, Gujarati…. and 9 other languages. Select gu and pa, too, ’cause why not.

Or are you keen on Destop in Latin America? Again we’re looking at Aurora, so fx_aurora is our tree of choice this time. Locales are Spanish in its American Variants, and Brazilian Portuguese.

Select generously, you can always reduce your selection through the controls on the right side of the resulting dashboard.

Play around, and compare the Status and History columns. Try to find stories, and share them in the comments below.

A bit more details on fx-aurora vs Firefox 32. Right now, Firefox 32 is on the Aurora channel and repository branch. Thus, selecting either gives you the same data today. In six weeks, though, 32 is going to be on beta, so if you bookmark a link, it’d give you different data then. That’s why you can only select one or the other.

June 17, 2014 03:38 PM

May 27, 2014

Staś Małolepszy

Pseudolocales for Firefox OS

With bug 900182 fixed, it is now possible to enable pseudolocales in developer builds of Firefox OS. Pseudolocales are based on US English and are automatically generated according to strategies defined in build/l10n.js.
Presently, two pseudolocales can be enabled in custom Gaia builds:

A screenshot of Accented and Mirrored English

What to test

Pseudolocales make it easy to do localizability testing without the need of knowing a foreign language. The strategies used to autogenerate the pseudo-localized strings create text which is readable by an English speaker without much trouble.

How to build

Pseudolocales are enabled by default in custom builds of Gaia, so you can likely continue building it as you've always have:

make

For non-standard setups, use the following make flags:

GAIA_CONCAT_LOCALES=1 LOCALES_FILE=shared/resources/languages.json make

You can then go into Settings and change the language of the device in the Language panel. Or, you could set one of the pseudolocales to be the default locale from the get-go, with:

GAIA_DEFAULT_LOCALE=qps-ploc make

The pseudolocales are currently generated on build time as part of the webapps-optimize logic, and as such tey don't work in DEBUG=1 builds yet.

Next steps

An important piece of the initial pseudolocales work was the creation of a framework which allows adding new locales in the future. Accented and Mirrored English are the first two useful ones, but I'd like to add at least one more, and also improve the strategies for generating the existing ones.

I filed a few follow-up bugs. I think the following two are particularly interesting:

May 27, 2014 03:41 PM

May 22, 2014

Axel Hecht

Progress on l10n.mozilla.org

Today we’re launching an update to l10n.mozilla.org (elmo).

Team pages and the project overview tables now contain sparklines, indicating the progress over the past 50 days.

Want to see how a localization team is doing? Now with 100% more self-serve.

If the sparklines go up like so

good progress

the localization is making good progress. Each spark is an update (either en-US or the locale), so sparks going up frequently show that the team is actively working on this one.

If the sparklines are more like

not so much

then, well, not so much.

The sparklines always link to an interactive page, where you can get more details, and look at smaller or larger time windows for that project and that locale.

You should also look at the bugzilla section. A couple of bugs with recent activity is good. More bugs with no activity for a long time, not so much.

Known issues: We still have localizations showing status for central/nightly, even though those teams don’t work on central. Some teams do, but not all. Also, the sparklines start at some point in the past 50 days, that’s because we don’t figure out the status before. We could.

May 22, 2014 11:44 AM

April 08, 2014

Staś Małolepszy

Refactored l10n.js landed in Gaia

Today Zibi and I landed the refactored code of shared/js/l10n.js in Gaia master. A culmination of months of hard work, the refactor is also a first step of a larger initiative to innovate in the area of mobile localization and eventually, to implement L20n in Gaia.

Firefox OS faces a number of challenges related to localization. Growing the locale coverage beyond 20 supported locales; adapting to multiple screen sizes and form factors; ensuring the performance and memory consumption is good when the DOM is localized on the fly. In order to respond to these needs and many more that we will encounter in the future, we need a flexible and modular codebase, with a thought-out API designed to perform well in asynchronous scenarios.

We decided to re-write Gaia's l10n.js drawing from our experiences from developing L20n. The underlying concepts are similar: the code is organized into useful abstractions like the localization Context managing the fallback chain of Locale objects, or the Entity class which represents a single translation unit.

It's not L20n just yet. The library still uses the .properties file format.
There are no custom macros or arbitrary dicts which allow localizers greater flexibility in creating translations. The language fallback is limited to two locales. It still is a good first step towards empowering developers and localizers alike.

To minimize the risk of regressions, the refactored code was almost a drop-in replacement for the old l10n.js library. We decided to keep the exact same API of the navigator.mozL10n object. Everything should just work as it did before. We feel that there's room for improvement in the API design, and we'll soon start suggesting changes to it (e.g. bug 993188 has made many developers implement workarounds in their apps, which would break if we fixed it right away in our refactored drop-in patch).

Notable changes

Feedback and bugs

The top priority fot this refactor was to be 100% compatible with the old l10n.js library. We made sure all tests pass on Travis and on TBPL and put a lot of effort into writing additional tests. We will be monitoring the tree for any regressions. If you notice something weird going on with your device or your app, for instance related to language switching, please let us know.
File bugs in the newly created Gaia::L10n component in Bugzilla or find us in #gaia and #l20n.

Next steps

The list of tasks on our to-do list is long, but we couldn't be more exited about them. Ranging from UX improvements to developer-friendly APIs, to giving localizes more control over their translations, next months are going to be great for localization and Firefox OS. A sneak peek, in no particular order:

April 08, 2014 06:33 PM

November 07, 2013

Staś Małolepszy

L20n 1.0 Release Candidate now available

L20n 1.0 RC is now available for testing. We ask users of this release candidate to watch for and report any major bugs they encounter; if no release-blocking bugs are reported, L20n 1.0 will be released in approximately one week.

Following previous beta releases, this release adds the last big feature planned in the 1.0 scope: secure HTML localization which allows localizers to use some HTML in translations. Furthermore, two Context methods, ctx.get and ctx.getEntity, have been renamed to ctx.getSync and ctx.getEntitySync respectively.

You can download the RC from l20n.org (which also received an overhaul) or from the builds repository. To quickly get started, clone the demo repository, which contains an example HTML file localized with L20n (preview).

Changes since Beta 4

Following is a summary of the most notable changes since Beta 4. You can also consult the full changelog:

HTML localization

Secure HTML localization was one of the last big features on our 1.0 list. In Beta 4 and before, the developer could add the (now obsolete) data-l10n-overlay attribute to any HTML element to allow translations to have HTML content. The RC removes this attribute completely and allows a subset of HTML elements to be present in all translations, without any intervention from the developer.

In particular, text-level semantic elements like em and strong are now allowed in all translations, together with certain translatable attributes, like title and placeholder. This allows localizers to stay in control when it comes to deciding where to use some HTML to mark up the text (for instance, they can freely use the sup element in Mme). Similarly, HTML entities are also allowed and available at localizer's discretion (e.g.  ).

The documentation gives a few other examples demonstrating how the HTML in the translation is sanitized before it's inserted into the DOM. You may also be interested in learning the details of the implementation from the original thread on the mailing list, as well as from bug 921169 and bug 922576.

getSync and getEntitySync

We also take this opportunity to rename two methods available on Context instances. ctx.get is now ctx.getSync and can be used, as before, to synchronously get a translation string from the Context instance. ctx.getEntity becomes ctx.getEntitySync and can be used, also as before, to synchronously get a translation object consisting of the string value, attributes and the language code.

Nothing has changed in the behavior of these methods (they already were synchronous before), and the change to their names is cosmetic and keeps L20n forward-compatible. It allows us to focus more on our asynchronous API, which in turn makes it possible to continue our research into language packages for L20n and into running L20n (or parts of it) in worker threads.

In L20n 1.0, ctx.getSync doesn't have a direct asynchronous counterpart (bug 931707). Instead, we encourage everyone to use the ctx.localize API, which is asynchronous and supports retranslation when the language changes and when values of L20n's globals change (the concept known as responsive localization). Here's an example:

ctx.localize(['hello', 'about'], function(l10n) {
  var node = document.querySelector('[data-l10n-id=hello]');
  node.textContent = l10n.entities.hello.value;
  node.classList.remove('hidden');
});

The callback function will be invoked when the translations of hello and about are available. It will also be called again if the user changes the language, or when the values of L20n's globals like @screen change (but only if hello or about actually use them).

Testing the RC

If you take the Release Candidate for a spin (download from l20n.org or clone the demo repository), please report any bugs that you find (or things that look like bugs). Pending any new release-blocking bugs, we will release 1.0 final in approximately one week.

You can file bugs in Bugzilla, or find us in #l20n on irc.mozilla.org, or email the tools-l10n mailing list. Thank you!

November 07, 2013 01:37 PM

October 18, 2013

Staś Małolepszy

Prototype of Language Packages for Firefox OS

Making Firefox OS available in as many languages as possible is part of Mozilla's mission to give users the freedom of choice and let them be in control of their online experience.

To this end, Zibi and I have implemented a prototype of user-installable language packages, or langpacks, which are downloaded to the device when the user requests them. Watch the short video below to see them in action.

How the prototype works

We started with a simple language package server in node.js which is periodically pinged by Firefox OS and returns the list of languages that the server has available for a given set of apps. This list is used to build the language choice menu in the Settings app. When the user selects a language which is not installed on the device, the server is queried again and the corresponding language resources are downloaded. Currently, only very basic version-checking is being done and we'd like to explore versioning of langpacks further in the next iteration.

Ideally, this would happen independently of the user's running any apps, in a background service. The list of additional languages available on the server would be synchronized periodically, and stored locally on the device. The Data Store API might be useful for making this data available to other apps.

Since there are no background services in Firefox OS yet, we implemented the prototype in every app. We're looking into Service Workers as potentially being the right solution to extracting this logic into a single component.

Lastly, langpacks currently comprise of translation files only, and don't provide additional keyboard layouts (although the Keyboard IME API should make this possible), nor do they change the translated names of apps which are stored in webapp manifests.

Why langpacks are important

Langpacks have a number of advantages and scalability is only one of them. Instead of pre-installing all available languages on each device sold around the world, langpacks let the user decide which language they need (and which they don't). They make it possible to decouple localization updates from OS updates and updates of individual apps, which might be useful for testing and for releasing early versions of localizations to gather feedback.

Where I see langpacks shine is in the fact that they move the decision making from developers to localizers and users. If pre-installing languages on the device is 'push', then langpacks are 'pull'. With langpacks, the community can test new translations easily. We can cater to the needs of minority languages. And the users can decide which languages they want to have their devices available in.

October 18, 2013 10:21 AM

September 19, 2013

Axel Hecht

A tale of convert and convert

Or, how I made converting gaia to gaia-l10n suck less.

Background: For Firefox OS, we’re exposing a modified repository to localizers, so that it’s easier to find out what to work on, and to get support from the l10n dashboards. Files in the main gaia repository on github like

apps/browser/locales/browser.en-US.properties

should become

apps/browser/browser.properties

and the localizable sections in manifest.webapp,

{
…
  "locales": {
    "en-US": {
      "name": "Browser",
      "description": "Gaia Web Browser"
    }
…
}

are exposed in manifest.properties as

name: Browser
description: Gaia Web Browser

We’re also not supporting git on the l10n dashboard yet, so we need hg repositories.

I haven’t come across a competitor to hg convert on the git side yet, so I looked on the mercurial side of life. I started by glancing of the code in hgext/convert in the upstream mercurial code. That does a host of things to get parents and graphs right, and I didn’t feel like replicating that. It doesn’t offer hooks for dynamic file maps, though, let alone content rewriting. But it’s python, and it’s open-source. So I forked it.

With hg convert. Isn’t that meta? That gives me a good path to update the extension with future updates to upstream mercurial. I started out with a conversion of mercurial 2.7.1, then removed all the stuff I don’t need like bzr support etc. Then I made the mercurial code do what I need for gaia. I had to disable some checks that try to avoid commits that don’t actually change the contents, because I don’t mind that happening. And last but not least I added the filemap and the shamap of the initial conversion of hgext/convert, so that future updates don’t depend on my local disk.

Now I could just run hg gaiaconv and get what I want. Enter the legacy repositories for en-US. We only want fast-forward merges in hg, and in the conversion to git. No history editing allowed. But as you can probably guess, the new history is completely incompatible with the old, from changeset one. But I don’t mind, I hacked that.

I did run the regular hg gaiaconv up to the revision 21000 of the integration/gaia-central repository. That ended up with the graph for revision 4af36780fb5d.

gaia-central conversion

I pulled the old conversion for v1-train, which is the graph for revision ad14a618e815.

old en-US for v1-train

Then I did a no-op merge of the old graph into the new one.

merge

That’s all good, but now future conversions via gaiaconv would still pick up the non-merged revision. Well, unless one just edits the generated shamap, and replaces all references to 4af36780fb5d with cfb28f851111. And yes, that actually works.

more conversions post-merge

Tadaaa, a fully automated conversion process, and only forward merges.

Repositories involved in this post:

September 19, 2013 02:49 PM

February 15, 2013

Axel Hecht

Risk management for releases at scale

Let me share some recent revelations I had. It all started with the infamous Berlin airport. Not the nice one in Tegel, but the BBI desaster. The one we’ve thought we’d open last year, and now we don’t know which year.

Part of the newscoverage here in Germany was all about how they didn’t do any risk analysis, and are doomed, and how that other project for the Olympics in London did do risk analysis, and got in under budget, ahead of time.

So what’s good for the Olympics can’t be bad for Firefox, and I started figuring out the math behind our risk to ship Firefox, at a given time, with loads of localizations. How likely is it that we’ll make it?

Interestingly enough, the same algorithm can also be applied to a set of features that are scheduled for a particular Firefox release. Locales, features, blockers, product-managers, developers, all the same thing :-). Any bucket of N things trying to make a single deadline have similar risks. And the same cure. So bear with me. I’ll sprinkle graphs as we go to illustrate. They’ll link to a site that I’ve set up to play with the numbers, reproducing the shown graphs.

The setup is like this: Every single item (localization, for exampe) has a risk, and I’m assuming the same risk across the board. I’m trying to do that N times, and I’m interested in how likely I’ll get all of them. And then I evaluate the impact of different amounts of freeze cycles. If you’re like me, and don’t believe any statistics unless they’re done by throwing dices, check out the dices demo.

Anyway, let’s start with 20% risk per locale, no freeze, and up to 100 locales.

80%, no freeze, up to 100

Ouch. We’re crossing 50-50 at 3 items already, and anything at scale is a pretty flat zero-chance. Why’s that? What we’re seeing is an exponential decay, the base being 80%, and the power being how often we do that. This is revelation one I had this week.

How can we help this? If only our teams would fail less often? Feel free to play with the numbers, like setting the successrate from 80% to 90%. Better, but the system at large still doesn’t scale. To fight an exponential risk, we need a cure that’s exponential.

Turns out freezes are just that. And that’d be revelation two I had this week. Let’s add some 5 additional frozen development cycles.

80%, 5 freezes, up to 100

Oh hai. At small scales, even just one frozen cycle kills risks. Three features without freeze have a 50-50 chance, but with just one freeze cycle we’re already at 88%, which is better than the risk of each individual feature. At large scales like we’re having in l10n, 2 freezes control the risk to mostly linear, 3 freezes being pretty solid. If I’m less confident and go down to 70% per locale, 4 or 5 cycles create a winning strategy. In other words, for a base risk of 20-30%, 4-5 freeze cycles make the problem for a localized release scale.

It’s actually intuitive that freezes are (kinda) exponentially good. The math is a tad more complicated, but simplified, if your per-item success rate is 70%, you only have to solve your problem for 30% of your items in the next cycle, and for 9% in the second cycle. Thus, you’re fighting scale with scale. You can see this in action on the dices demo, which plays through this each time you “throw” the dices.

Now onwards to my third revelation while looking at this data. Features and blockers are just like localizations. Going in to the rapid release cycle with Firefox 5 etc, we’ve made two rules:

That worked fine for a while, but since then, mozilla has grown as an organization. We’ve also built out dependencies inside our organization that make us want particular features in particular releases. That’s actually a good situation to be in. It’s good that people care, and it’s good that we’re working on things that have organizational context.

But this changed the risks in our release cycle. We started off having a single risk of exponential scale after the migration date (l10n). Today, we have features going in to the cycle, and localizations thereof. At this point, having feature-freeze and string-freeze being the same thing becomes a risk for the release cycle at large. We should think about how to separate the two to mitigate the risk for each effectively, and ship awesome and localized software.

I learned quite a bit looking at our risks, I hope I could share some of that.

February 15, 2013 03:26 PM

September 27, 2012

Axel Hecht

Language packs are restartless now

Language packs are add-ons that you can install to add additional localizations to our desktop applications.

Starting with tomorrow’s nightly, and thus following the Firefox 18 train, language packs will be restartless. That was bug 677092, landed as 812d0ba83175.

To change your UI language, you just need to install a language pack, set your language (*), and open a new window. This also works for updates to an installed language pack. Opening a new window is the workaround for not having a reload button on the chrome window.

The actual patch turned out to be one line to make language packs restartless, and one line so that they don’t try to call in to bootstrap.js. I was optimistic that the chrome registry was already working, and rightfully so. There are no changes to the language packs themselves.

Tests were tricky, but Blair talked me through most of it, thanks for that.

(*) Language switching UI is bug 377881, which has a mock-up for those interested. Do not be scared, it only shows if you have language packs installed.

September 27, 2012 11:04 AM

July 20, 2012

Axel Hecht

Why l10n tools should be editors instead of serializers

If your tool serializes internal state instead of editing files, it’ll do surprising things if it encounters surprising content. Like, turn

errNotSemicolonTerminated=Named character reference was not terminated by a semicolon. (Or “&” should have been escaped as “&”.)

into

errNotSemicolonTerminated=Named character reference was not terminated by a semicolon. (Or “” should have been escaped as “amp;”.)

And that’s for a string the localizer never touched.

(likely narro issue 316)

July 20, 2012 04:18 PM