Planet Mozilla Automation

February 02, 2016

Henrik Skupin

Firefox Desktop automation goals Q1 2016

As promised in my last blog posts I don’t want to only blog about the goals from last quarters, but also about planned work and what’s currently in progress. So this post will be the first one which will shed some light into my active work.

First lets get started with my goals for this quarter.

Execute firefox-ui-tests in TaskCluster

Now that our tests are located in mozilla-central, mozilla-aurora, and mozilla-beta we want to see them run on a check-in basis including try. Usually you will setup Buildbot jobs to get your wanted tasks running. But given that the build system will be moved to Taskcluster in the next couple of months, we decided to start directly with the new CI infrastructure.

So how will this look like and how will mozmill-ci cope with that? For the latter I can say that we don’t want to run more tests as we do right now. This is mostly due to our limited infrastructure I have to maintain myself. Having the needs to run firefox-ui-tests for each check-in on all platforms and even for try pushes, would mean that we totally exceed the machine capacity. Therefore we continue to use mozmill-ci for now to test nightly and release builds for en-US but also a couple of other locales. This might change later this year when mozmill-ci can be replaced by running all the tasks in Taskcluster.

Anyway, for now my job is to get the firefox-ui-tests running in Taskcluster once a build task has been finished. Although that this can only be done for Linux right now it shouldn’t matter that much given that nothing in our firefox-puppeteer package is platform dependent so far. Expanding testing to other platforms should be trivial later on. For now the primary goal is to see test results of our tests in Treeherder and letting developers know what needs to be changed if e.g. UI changes are causing a regression for us.

If you are interested in more details have a look at bug 1237550.

Documentation of firefox-ui-tests and mozmill-ci

We are submitting our test results to Treeherder for a while and are pretty stable. But the jobs are still listed as Tier-3 and are not taking care of by sheriffs. To reach the Tier-2 level we definitely need proper documentation for our firefox-ui-tests, and especially mozmill-ci. In case of test failures or build bustage the sheriffs have to know what’s necessary to do.

Now that the dust caused by all the refactoring and moving the firefox-ui-tests to settles a bit, we want to start to work more with contributors again. To allow an easy contribution I will create various project documentation which will show how to get started, and how to submit patches. Ultimately I want to see a quarter of contribution project for our firefox-ui-tests around mid this year. Lets see how this goes…

More details about that can be found on bug 1237552.

February 02, 2016 04:39 PM

Armen Zambrano G. (@armenzg)

End of MozCI QoC term

We recently completed another edition of Quarter of Contribution and I had the privilege to work with MikeLingF3real & xenny.
I want to take a moment to thank all three of you for your hard work and contributions! It was a pleasure to work together with you during this period.

Some of the highlights of this term are:

You can see all other mozci contributions in here.

One of the things I learned from this QoC term:
  • Prepare sets of issues that are related which build towards a goal or a feature.
    • The better you think it through the easier it will be for you and the contributors
    • In GitHub you can create milestones of associated issues
  • Remind them to review their own code.
    • This is something I try to do for my own patches and saves me from my own embarrassment :)
  • Put it on the contributors to test their code before requesting formal review
    • It forces them to test that it does what they expect it to do
  • Set expectations for review turn around.
    • I could not be reviewing code every day since I had my own personal deliverables. I set Monday, Wednesday and Friday as code review days.
It was a good learning experience for me and I hope it was beneficial for them as well.

Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

February 02, 2016 02:48 PM

Byron Jones

happy bmo push day!

the following changes have been pushed to

discuss these changes on

Filed under: bmo, mozilla

February 02, 2016 07:23 AM

January 27, 2016

Geoff Brown


Bug 1233220 added a new Android-only mochitest-chrome test called test_awsy_lite.html. Inspired by, test_awsy_lite runs similar code and takes similar measurements to, but runs as a simple mochitest and reports results to Perfherder.

There are some interesting trade-offs to this approach to performance testing, compared to running a custom harness like or Talos.

+ Writing and adding a mochitest is very simple.

+ It is easy to report to Perfherder (see

+ Tests can be run locally to reproduce and debug test failures or irregularities.

+ There’s no special hardware to maintain. This is a big win compared to ad-hoc systems that might fail because someone kicks the phone hanging off the laptop that’s been tucked under their desk, or because of network changes, or failing hardware. was plagued by problems like this and hasn’t produced results in over a year.

? Your new mochitest is automatically run on every push…unless the test job is coalesced or optimized away by SETA.

? Results are tracked in Perfherder. I am a big fan of Perfherder and think it has a solid UI that works for a variety of data (APK sizes, build times, Talos results). I expect Perfherder will accommodate test_awsy_lite data too, but some comparisons may be less convenient to view in Perfherder compared to a custom UI, like

– For Android, mochitests are run only on Android emulators, running on aws. That may not be representative of performance on real phones — but I’m hoping memory use is similar on emulators.

– Tests cannot run for too long. Some Talos and other performance tests run many iterations or pause for long periods of time, resulting in run-times of 20 minutes or more. Generally, a mochitest should not run for that long and will probably cause some sort of timeout if it does.

For test_awsy_lite.html, I took a few short-cuts, worth noting:

Results are in Perfherder. Add data for “android-2-3-armv7-api9” or “android-4-3-armv7-api15” and you will see various tests named “Resident Memory …”, each corresponding to a traditional measurement.


January 27, 2016 01:39 PM

January 20, 2016

Byron Jones

happy bmo push day!

the following changes have been pushed to

discuss these changes on

Filed under: bmo, mozilla

January 20, 2016 07:33 AM

January 19, 2016

mozregression updates

Release 2.2.0 and GUI release 0.8.0

New mozregression releases, coming with great new features!

Changes for both GUI and command line:

GUI changes only:

Command line changes only:

January 19, 2016 12:00 AM

January 13, 2016

David Burns

Marionette Executable Release v0.6.0

I have just released a new version of the Marionette, well the executable that you need to download.

The main fixes in this release is the ability to speak to Firefox and get meaningful error messages. This was a slight oversight on our part to make sure that we don't run commands out of sync. We have also added in getPageSource. This "native" call runs in the browser instead of trying to do it in the JavaScript sandbox which is what a number of the drivers were attempting. This will be added to the specification very soon.

I have also landed the update to interactions to the specification. This reads much better and has prose that makes it implementable. I suspect as the likes of Google and Microsoft start looking to implement it there will be bugs that need fixing.

Since you are awesome early adopters it would be great if we could raise bugs.

I am not expecting everything to work but below is a quick list that I know doesn't work.

Switching of Frames needs to be done with either a WebElement or an index. Windows can only be switched by window handles. This is currently how it has been discussed in the specification.

If in doubt, raise bugs!

Thanks for being an early adopter and thanks for raising bugs as you find them!

January 13, 2016 02:48 PM

January 11, 2016

Byron Jones

happy bmo push day!

the following changes have been pushed to

discuss these changes on

Filed under: bmo, mozilla

January 11, 2016 03:13 PM

Julien Pagès

mozregression – Engineering Productivity Project of the Month

Hello from Engineering Productivity! Once a month we highlight one of our projects to help the Mozilla community discover a useful tool or an interesting contribution opportunity.

This month’s project is mozregression!

Why is mozregression useful ?

mozregression helps to find regressions in Mozilla projects like Firefox or Firefox on Android. It downloads and runs the builds between two dates (or changesets) known to be good and bad, and lets you test each build to finally find by bisection the smallest possible range of changesets where the regression appears.

It does not build locally the application under test, instead, it uses pre-built files, making it fast and easy for everyone to look for the origin of a regression.


# Search a Firefox regression in mozilla-central starting from 2016-01-01

mozregression -g 2016-01-01

# Firefox regression, on mozilla-aurora from 2015-09-01 to 2015-10-01

mozregression --repo aurora -g 2015-09-01 -b 2015-10-01

# Look for a regression in fennec (firefox for android)

mozregression --app fennec -g 2015-09-01 -b 2015-10-01

# regression on firefox in inbound, using debug builds and starting from known changesets

mozregression -b 6f4da397ac3c -g eb4dc9b5a928 -B debug --repo inbound

Note that a graphical interface exists also.

Where do I find mozregression ?

Start with:

What are we working on ?

Currently mozregression is improving in multiple areas, among them:


William Lachance (:wlach) and myself (:parkouss) are the current maintainers of mozregression.

We welcome contributors! Mike Ling is helping the project for quite some time now, adding useful features and fixing various bugs – he’s currently working on providing ready to use binaries for Mac OS X. A big thanks Mike Ling for your contributions!

Also thanks to Saurabh Singhal and Wasif Hider, who are recent contributors on the graphical user interface.

If you want to contribute as a developer or help us on the documentation, please say hi on the #ateam irc channel!

Reporting bugs / new ideas

You can also help a lot by reporting bugs or new ideas! Please file bugs on bugzilla with the mozregression component:

mozregression’s bug list:

For more information about all Engineering Productivity projects visit our wiki. If you’re interested in helping out, the A-Team bootcamp has resources for getting started.

January 11, 2016 02:47 PM

David Burns

Public Source vs Open Source

A few weeks ago I had an interesting conversation on twitter and then on instant messaging about a bit of software that was open sourced. Some thought, and might still do, the new piece of software might not be supported.

There was also recently a good blog post from James Long about how it can be hard to create open source code and then maintain it. Either life gets in the way or another project gets in the way.

I have always had a different view of code to most people. The idea is simple, at least in my head.

Open Source

The idea of Open Source has changed over the years and this has meant the original meaning is not quite right. Open Source has started to have a certain aspect of community involved, from people submitting patches (PRs on Github).

Projects that have been around for a very long time have organically grown some kind of community. These tend to be people who believe in the project or see other people working on it that they get to know. We see meet ups forming as more an more people get involved.

This is best part of be source code! The openness with people and code all wrapped up in one!

However, not all code out in the open will achieve this! (And this is fine, not all pieces of code need to have community. Imagine if every package on NPM had a meet up!?!?).

Public Source

Public source everything that open source has minus all community side of things. A really good example of Public Source is Android. You can see all the code, you derive your own work but want to submit a patch? Well Cryogen might take it but Google, seemingly, don't care.

Also, most projects on Github probably fall under this category. Especially if the person is a starter and not a maintainer, to use James' concept.

The thing to remember is that everyone wins when the code is in the public but before you start getting all hung up on "support" from people who have given up the code, and their time, to put it out there remember that open source needs to grow from public source!

January 11, 2016 10:46 AM

January 08, 2016

Henrik Skupin

Review of automation work – Q4 2015

The last quarter of 2015 is gone and its time to reflect what happened in Q4. In the following you will find a full overview again for the whole quarter. It will be the last time that I will do that. From now on I will post in shorter intervals to specific topics instead of covering everything. This was actually a wish from our latest automation survey which I want to implement now. I hope you will like it.

So during the last quarter my focus was completely on getting our firefox-ui-tests moved into mozilla-central, and to use mozharness to execute firefox-ui-tests in mozmill-ci via the test archive. As result I had lesser time for any other project. So lets give some details…

Firefox UI Tests / Mozharness

One thing you really want to have with tests located in the tree is that those are not failing. So I spent a good amount of time to fix our top failures and all those regressions as caused by UI changes (like the security center) in Firefox as preparation for the move. I got them all green and try my best to keep that state now while we are in the transition.

The next thing was to clean-up the repository and split apart all the different sub folders into their own package. With that others could e.g. depend on our firefox-puppeteer package for their own tests. The whole work of refactoring has been done on bug 1232967. If you wonder why this bug is not closed yet it’s because we still have to wait with the landing of the patch until mozmill-ci production uses the new mozharness code. This will hopefully happen soon and only wait of some other bugs to be fixed.

But based on those created packages we were able to use exactly that code to get our harness, puppeteer, and tests landed on We also package them into the archive for use in mozmill-ci. Details about all that can be found on bug 1212609. But please be aware that we still use the Github repository as integration repository. I regularly mirror the code to hg, which has to happen until we can also use the test package for localized builds and update tests.

Beside all that there were also a couple of mozharness fixes necessary. So I implemented a better fetching of the tooltool script, added the uninstall feature, and also setup the handling of crash symbols for firefox-ui-tests. Finally the addition of test package support finished up my work on mozharness for Q4 in 2015.

During all the time I was also sheriffing our test results on Treeherder (e.g. mozilla-central) because we are still Tier-3 level and sheriffs don’t care about it.

Mozmill CI

Our Jenkins based CI system is still called mozmill-ci even it doesn’t really run any mozmill tests anymore. We decided to not change its name given that it will only be around this year until we can run all of our tests in TaskCluster. But lots of changes have been landed, which I want to announce below:

Addons / Tools

I also had some time to work on supporting tools. Together with the help of contributors we got the following done:


Nightly Tester Tools


So all in all it was a productive quarter with lots of things accomplished. I’m glad that we got all of this done. Now in Q1 it will continue and more interesting work is in-front of me, which I’m excited about. I will announce that soon in my next blog post.

Until then I would like to give a little more insight into our current core team for Firefox automation. A picture taken during our all hands work week in Orlando early in December shows Syd, Maja, myself, and David:

Group Picture

Lets get started into 2016 with lots of ideas, discussions, and enough energy to get those things done.

January 08, 2016 10:25 PM

Dan Minor

Using masked writes with ARM NEON intrinsics

I recently fixed Bug 1105513 which was to provide an ARM NEON optimized version of the AudioBlockPanStereoToStereo for the case where the “OnTheLeft” is an array. This is used by the StereoPanner node when the value is set at a future time, for instance with code like the following:

panner = oac.createStereoPanner();
panner.pan.setValueAtTime(-0.1, 0.0);
panner.pan.setValueAtTime(0.2, 0.5);

The “OnTheLeft” values determine whether the sound is on the left or right of the listener at a given time, which controls the interpolation calculation performed when panning. If this changes with time, then this is passed as an array rather than as a constant.

The unoptimized version of this function checks each value of “OnTheLeft” and performs the appropriate calculation. This isn’t an option for NEON which lacks this kind of conditional execution.

The bright side is that NEON does provide masked writes where a variable controls which components of a vector are written. Unfortunately, the NEON documentation is spare at best, so it took a few tries to get things right.

The first trick is to convert from a bool to a suitable mask. What a bool is, is of course platform dependent, but in this case I had an array of eight bytes, each containing a zero or a one. The best solution I came up with was to load them as a vector of 8 unsigned bytes and then load each corresponding float value in the mask individually:

isOnTheLeft = vld1_u8((uint8_t *)&aIsOnTheLeft[i]);
voutL0 = vsetq_lane_f32(vget_lane_u8(isOnTheLeft, 0), voutL0, 0);
voutL1 = vsetq_lane_f32(vget_lane_u8(isOnTheLeft, 1), voutL0, 1);

Once loaded, they can be converted into a suitable mask by using the vcgtq function which sets all bits to 1 in the first argument if it is greater than the second argument:

voutL0 = (float32x4_t)vcgtq_f32(voutL0, zero);

After that, the appropriate calculations are done for both the case where “OnTheLeft” is true and where it is false. These are then written to the result using vbsql function, which treats the mask as the output, and selects from the second two arguments based upon the value in the mask:

voutL0 = vbslq_f32((uint32x4_t)voutL0, onleft0, notonleft0);

I evaluated these changes on a StereoPanner benchmark where I saw a small performance improvement.

January 08, 2016 01:53 PM

Mark Côté

BMO in 2015

It’s been a whole year since my last BMO update, partly because I’ve been busy with MozReview (and blogging a lot about it), and partly because the BMO team got distracted from our goals by a few sudden priority changes, which I’ll get to later in this post.

Plans from 2014

Even with some large interruptions, we fully achieved three of our five goals for the year and made good progress on a fourth.

Alternative Bug Views

Have you tried out the new modal UI? Although not completely finished (it lacks a few features that the standard UI has), it’s very usable. I don’t remember the last time I had to switch back, and I’ve been using it for at least 6 months. Bonus: gone is the intermediate page when you change a bug’s product, a gripe from time immemorial!

Even though there are still a large number of controls, the new UI is a lot more streamlined. glob gave a brief presentation at a Mozilla Project Meeting in November if you’d like to learn more.

The part we haven’t yet undertaken is building on this new framework to provide alternate views of bug data depending on what the user is trying to accomplish. We want to experiment with stripping down the presented data to only what is needed for a particular task, e.g. developing, triaging, approving, etc. The new UI is a lot more flexible than the old, so in 2016 we’ll build out at least one new task-centric view.

GitHub Authentication

If you haven’t noticed, you can log into BMO via GitHub. If you’ve never used BMO before, you’ll be prompted to set up an account after authenticating. As with Persona, only users with no special privileges (i.e. not admins nor people in security groups) can log in via GitHub.

Auth Delegation

Originally designed to smooth the process of logging into Review Board, auth delegation for API keys is actually a general-use feature that greatly improves the user experience, not to mention security, of third-party apps by allowing them to delegate authentication to BMO. There’s now no reason for apps to directly ask for your BMO credentials!

MozReview Details

There’s now a panel just above the attachments table that shows all the MozReview commits associated with the displayed bug along with a bit of other info:

We’re currently sorting out a single method to display other relevant information, notably, status of reviews, and then we’ll add that to this table.

Improved Searchability

This is the big item we haven’t made much progress on. We’ve got a plan to mirror some data to an Elasticsearch cluster and wire it into Quick Search. We’ve even started on the implementation, but it’s not going to be ready until mid-2016. It will increase search speeds, understandably one of the more common complaints about BMO.

Curve balls

We had two sets of surprises in 2015. One was work that ended up consuming more time than we had expected, and the other was important work that suddenly got a big priority boost.

BMO Backup in AWS

The first is that we moved the BMO failover out of a data center in Phoenix and into the cloud. IT did most of the work, but we had to make a series of changes to BMO to facilitate the move. We also had a lot of testing to do to. The upside is that our new failover system has had more testing than our old one had for quite some time!

Hardened Security

In August we found out that an attacker had compromised a privileged BMO account, using a combination of a weak, reused password and an exploit in another web site. In addition to a huge forensics effort from the great security folks at Mozilla, the BMO team implemented a number of security enhancements to BMO, most notably two-factor authentication. This work naturally took high priority and is the main reason for the slippage of our big 2015 goals. Here’s to a more secure 2016!

Other Stuff

As usual, the BMO team rolled out a pile of smaller fixes, enhancements, improvements, and new features. A few notable examples include

You can always find the exhaustive list of recent changes to BMO on the wiki or on the group/mailing list.

January 08, 2016 01:20 AM

January 05, 2016

Henrik Skupin

Automation Survey Follow-up

As promised in my last post about the automation survey results I wanted to come up with a follow-up to clarify our next steps in being more open for our activities, discussions, and also quarterly goals. Sorry, that it has been taken a bit longer but end of the quarter and especially the year is mostly packed with stuff to finish up. Also the all-hands work week in Orlando beginning of December hold me off from doing a lot real work.

So lets get started with the mailing list topic first. As we have seen most people kinda like to get our news via the automation mailing list. But given the low usage of that list in the last months it was a bit surprising. Nearly all the time I sent emails myself (not to count in Travis results). That means we want to implement a change here. From now on we won’t use the list but instead utilize the list. Also because this is the recommended list for the Engineering Productivity team we are all part of, and discussions will reach a larger audience. So please subscribe to this list via Google Groups or Email.

For status updates about our current activities we started to use last quarter. It seems to work pretty well for us and everyone else is welcome to also post updates to our automation project section. If you are interested in those updates then read through that list or simply subscribe the page in your RSS reader.

Please also note that from now on there will be no Firefox Automation reports anymore. Instead I will reduce the amount of different contents, and only write about projects I worked on. So keep an eye out to not miss those!

January 05, 2016 12:03 PM

January 04, 2016

Mark Côté

Review Board history

A few weeks ago, mdoglio found an article from six years ago comparing Review Board and Splinter in the context of GNOME engineering. This was a fascinating read because, without having read this article in advance, the MozReview team ended implementing almost everything the author talked about.

Firstly, I admit the comparison isn’t quite fair when you replace with GNOME doesn’t use attachment flags, which BMO relies heavily on. I haven’t ever submitted a patch to GNOME, but I suspect BMO’s use of review flags makes the review process at least a bit simpler.

The first problem with Review Board that he points out is that the “post-review command-line leaves a lot to be desired when compared to git-bz”. This was something we did early on in MozReview, all be it with Mercurial instead: the ability to push patches up to MozReview with the hg command. Admittedly, we need an extension, mainly because of interactions with BMO, but we’ve automated that setup with mach mercurial-setup to reduce the friction. Pushing commits is the area of MozReview that has seen the fewest complaints, so I think the team did a great job there in making it intuitive and easy to use.

Then we get to what the author describes as “a more fundamental problem”: “a review in Review Board is of a single diff”. As he continues, “More complex enhancements are almost always done as patchsets [emphasis his], with each patch in the set being kept as standalone as possible. … Trying to handle this explicitly in the Review Board user interface would require some substantial changes”. This was also an early feature of MozReview, implemented at the same time as hg push support. It’s a core philosophy baked into MozReview, the single biggest feature that distinguishes MozReview from pretty much every other code-review tool out there. It’s interesting to see that people were thinking about this years before we started down that road.

An interesting aside: he says that “a single diff … [is] not very natural to how people work with Git”. The article was written in 2009, as GitHub was just starting to gain popularity. GitHub users tend to push fix-up commits to address review points rather than editing the original commits. This is at least in part due to limitations present early on in GitHub: comments would be lost if the commit was updated. The MozReview team, in fact, has gotten some push back from people who like working this way, who want to make a bunch of follow-up commits and then squash them all down to a single commit before landing. People who strongly support splitting work into several logical commits and updating them in place actually tend to be Mercurial users now, especially those that use the evolve extension, which can even track bigger changes like commit reordering and insertion.

Back to Review Board. The author moves onto how they’d have to integrate Review Board with Bugzilla: “some sort of single-sign-on across Bugzilla and Review Board”, “a bugzilla extension to link to reviews”, and “a Review Board extension to update bugs in Bugzilla”. Those are some of the first features we developed, and then later improved on.

There are other points he lists that we don’t have, like an “automated process to keep the repository list in Review Board in sync with the 600+ GNOME repositories”. Luckily many people at Mozilla work on just one repo: mozilla-central. But it’s true that we have to add others manually.

Another is “reduc[ing] the amount of noise for bug reporters”, which you get if you confine all patch-specific discussion to the review tool. We don’t have this yet; to ease the transition to Review Board, we currently mirror pretty much everything to Bugzilla. I would really like to see us move more and more of code-related discussion to Review Board, however. Hopefully as more people transition to using MozReview full time, we can get there.

Lastly, I have to laugh a bit at “it has a very slick and well developed web interface for reviewing and commenting on patches”. Clearly we thought so as well, but there are those that prefer the simplicity of Splinter, even in 2015, although probably mostly from habit. Trying to reconcile these two views is very challenging.

January 04, 2016 10:19 PM

David Burns

The "power" of overworking

The other week I was in Orlando, Florida for a Mozilla All-Hands. It is a week where around 1200 Mozillians get together to spend time with each other planning, coding, or solving some hard problems.

One of the topics that came up was how someone always seemed to be online. This comment was a little more than "they never seem to go offline from IRC". It was "they seem to commenting on things around 20 hours a day". Overworking is a simple thing to do and when you love your job you can easily be pulled into this trap.

I use the word trap and I mean it!

If you are overworking you put yourself into this state where people come to expect that you will overwork. If you overwork, and have a manager who doesn't notice that you are overworking, when you do normal hours they begin to think that you are slacking. If you do have a manager who is telling you to stop overdoing it, you might then have colleagues who don't notice that you work all the hours. They then expect you to do be this machine, doing everything and more. And those colleagues that notice you doing too many hours start to think your manager is a poor manager for not helping you have a good work/life balance.

At this point, everyone is starting to lose. You are not being as productive as your could be. Studies have shown that working more than 40 hours a week only marginally increases productivity and this only lasts for a few weeks before productivity drops below the productivity you would have if you worked 40 hours a week.

The reasons for overworking can be numerous but the one that regularly stands out is imposter syndrome. "If I work 50 hours a week then people won't see me fail because I will hopefully have fixed it in time". This is a fallacy, people are happy to wait for problems to be fixed as long as it is in hand. Having one person be responsible for fixing things is a road to ruin. A good team is measured by how quickly they help colleagues. If you fall, there will be 2 people there to pick you up.

Before you start working more than 40 hours a week start thinking about the people this is going to impact. This is not only your colleagues, who start having to clean up technical debt, but your personal life. It is also your loved ones who are impacted. Missing an anniversary, a birthday, a dance/music recital. Work is never worth missing that!

If you are working more than 40 hours I suggest bringing this up in your next 1:1. Your manager will appreciate that you are doing some self care (if they are good managers) and work with you in making changes to your workload. They could be over promising their team and need to get this under control.

January 04, 2016 09:10 PM

December 31, 2015

Geoff Brown

Firefox for Android Performance Measures – Q4 Check-up



APK Size

This quarter we began tracking the size of the Firefox for Android APK, and some of its components. You can see the size of every build on treeherder using Perfherder.

Here’s how the APK size changed over the last 2 months, for mozilla-central Android 4.0 opt builds:


There are lots of increases and a few decreases here. The most significant decrease (almost half a megabyte) is on Nov 23, from mfinkle’s change for Bug 1223526. The most significant increase (~200K) is on Dec 20, from a Skia update, Bug 1082598.

It is worth noting that the sizes of over the same period were almost always increasing:



This section tracks Perfherder graphs for mozilla-central builds of Firefox for Android, for Talos tests run on Android 4.0 Opt. The test names shown are those used on treeherder. See for background on Talos.

We intend to retire the remaining Android Talos tests, migrating these tests to autophone in the very near future.


Measure of “checkerboarding” during simulation of real user interaction with page. Lower values are better.

This test is no longer running. It was noisy and needed to be rewritten for APZ. See discussion in bug 1213032 and bug 1230572.


An svg-only number that measures SVG rendering performance. About half of the tests are animations or iterations of rendering. This ASAP test (tsvgx) iterates in unlimited frame-rate mode thus reflecting the maximum rendering throughput of each test. The reported value is the page load time, or, for animations/iterations – overall duration the sequence/animation took to complete. Lower values are better.


730 (start of period) – 110 (end of period)

A small regression at the end of November corresponded with the introduction of APZ; it was investigated in bug 1229118. An extraordinary improvement on Dec 25 was the result of jchen’s refactoring.


Generic page load test. Lower values are better.


730 (start of period) – 680 (end of period)

Note the same regression and improvement as seen in tsvgx.


Throbber Start / Throbber Stop

These graphs are taken from  Browser startup performance is measured on real phones (a variety of popular devices).




Android tests are no longer run on Eideticker.


These graphs are taken from the mozbench dashboard at which includes some comparisons involving Firefox for Android. More info at


Sadly, the other mobile benchmarks have no data for most of November and December…I’m not sure why.

December 31, 2015 08:44 PM

December 30, 2015

Julien Pagès

Convert Firefox into Emacs

Firefox is a great browser. One of the reasons I really love it is because it is highly configurable: as an Emacs user, I wanted to use emacs key bindings inside Firefox – well, it’s easy to do that. And much more!

Most of the magic for me comes from the awesome keysnail addon. It basically convert Firefox into Emacs, is also highly configurable and have plugins.

For example, I now use C-x <left> and C-x <right> to switch tabs; C-x b to choose a specific tab (using the Tanything plugin) or C-x k to kill a tab. Tabs are now like Emacs buffers! Keysnail support the mark, incremental search (C-s and C-r), specific functions, … Even M-x is implemented, to search and run specific commands!

Also I use the Find As You Type Firefox feature, for links. It’s awesome: I just hit ‘, then start typing some letters in a link title that I want to follow – I can then use C-s or C-r to find next/previous matching links if needed, then I just press Return to follow the link.

I can browse the web more efficiently, I am less using the mouse and I can reuse the same key bindings in Emacs and Firefox! I keep my configuration files on github, feel free to look at it if you’re interested!

Happy browsing!

December 30, 2015 03:49 PM

December 24, 2015

Geoff Brown

Comparing Linux mochitest results across environments

A few weeks ago, I was trying to run Linux Debug mochitests in an unfamiliar environment and that got me to thinking about how well tests run on different computers. How much does the run-time environment – the hardware, the OS, system applications, UI, etc. – affect the reliability of tests?

At that time, Linux 64 Debug plain, non-e10s mochitests on treeherder – M(1) .. M(5) – were running well: Nearly all jobs were green. The most frequent intermittent failure was dom/html/test/test_fullscreen-api-race.html, but even that test failed only about 1 time in 10. I wondered, are those tests as reliable in other environments? Do intermittent failures reproduce with the same frequency on other computers?

Experiment: Borrow a test slave, run tests over VNC

I borrowed an aws test slave – see – and used VNC to access the slave and run tests. I downloaded builds and test packages from mozilla-central and invoked with the same arguments used for the automated tests shown on treeherder.   To save time, I restricted my tests to mochitest-1, but I repeated mochitest-1 10 times. All tests passed all 10 times. Additional runs produced intermittent failures, like test_fullscreen-api-race, with approximately the same frequency reported by Orange Factor for recent builds. tl;dr Treeherder results, including intermittent failures, for mochitests can be reliably reproduced on borrowed slaves accessed with VNC.

Experiment: Run tests on my laptop

Next I tried running tests on my laptop, a ThinkPad w540 running Ubuntu 14. I downloaded the same builds and test packages from mozilla-central and invoked with the same arguments used for the automated tests shown on treeherder. This time I noticed different results immediately: several tests in mochitest-1 failed consistently. I investigated and tracked down some failures to environmental causes: essential components like pulseaudio or gstreamer not installed or not configured correctly. Once I corrected those issues, I still had a few permanent test failures (like dom/base/test/test_applet_alternate_content.html, which has no bugs on file) and very frequent intermittents (like dom/base/test/test_bug704320_policyset.html, which is decidedly low-frequency in Orange Factor). I also could not reproduce the most frequent mochitest-1 intermittents I found on Orange Factor and reproduced earlier on the borrowed slave. An intermittent failure like test_fullscreen-api-race, which I could generally reproduce at least once in 10 to 20 runs on a borrowed slave, I could not reproduce at all in over 100 runs on my laptop. (That’s 100 runs of the entire mochitest-1 job. I also tried running specific tests or specific directories of tests up to 1000 times, but I still could not reproduce the most common intermittent failures seen on treeherder.) tl;dr Intermittent failures seen on treeherder are frequently impossible to reproduce on my laptop; some failures seen on my laptop have never been reported before.

Experiment: Run tests on a Digital Ocean instance

Digital Ocean offers virtual servers in the cloud, similar to AWS EC2. Digital Ocean is of interest because rr can be used on Digital Ocean but not on aws. I repeated my test runs, again with the same methodology, on a Digital Ocean instance set up earlier this year for Orange Hunter.

My experience on Digital Ocean was very similar to that on my own laptop. Most tests pass, but there are some failures seen on Digital Ocean that are not seen on treeherder and not seen on my laptop, and intermittent failures which occur with some frequency on treeherder could not be reproduced on Digital Ocean.

tl;dr Intermittent failures seen on treeherder are frequently impossible to reproduce on Digital Ocean; some failures seen on Digital Ocean have never been reported before; failures on Digital Ocean are also different (or of different frequency) from those seen on my laptop.


I found it relatively easy to run Linux Debug mochitests in various  environments in a manner similar to the test jobs we see on treeherder. Test results were similar to treeherder, in that most tests passed. That’s all good, and expected.

However, test results often differed in small but significant ways across environments and I could not reproduce most frequent intermittent failures seen on treeherder and tracked in Orange Factor. This is rather discouraging and the cause of the concern mentioned in my last post: While rr appears to be an excellent tool for recording and replaying intermittent test failures and seems to have minimal impact on the chances of reproducing an intermittent failure, rr cannot be run on the aws instances used to run Firefox tests in continuous integration, and it seems difficult to reproduce many intermittent test failures in different environments. (I don’t have a good sense of why this is: Timing differences, hardware, OS, system configuration?)

If rr could be run on aws, all would be grand: We could record test runs in aws with excellent chances of reproducing and recording intermittent test failures and could make those recordings available to developers interested in debugging the failures. But I don’t think that’s possible.

We had hoped that we could run tests in another environment (Digital Ocean) and observe the same failures seen on aws and reported in treeherder, but that doesn’t seem to be the case.

Another possibility is bug 1226676: We hope to start running Linux tests in a docker container soon. Once that’s working, if rr can be run in the container, perhaps intermittent failures will behave the same way and can be reproduced and recorded.

December 24, 2015 05:28 AM

December 22, 2015

Dave Hunt

Selenium tests with pytest

When you think of Mozilla you most likely first associate it with Firefox or our mission to build a better internet. You may not think we have many websites of our own, beyond perhaps the one where you can download our products. It’s only when you start listing them that you realise how many we actually have; addons repository, product support, app marketplace, build results, crash statistics, community directory, contributor tasks, technical documentation, and that’s just a few! Each of these have a suite of automated functional tests that simulate a user interacting with their browser. For most of these we’re using Python and the pytest harness. Our framework has evolved over time, and this year there have been a few exciting changes.

Over four years ago we developed and released a plugin for pytest that removed a lot of duplicate code from across our suites. This plugin did several things; it handled starting a Selenium browser, passing credentials for tests to use, and generating a HTML report. As it didn’t just do one job, it was rather difficult to name. In the end we picked pytest-mozwebqa because it was only specific in addressing the needs of the Web QA team at Mozilla. It really took us to a new level of consistency and quality across all our our web automation projects.

Enhanced HTML report generated by pytest-htmlThis year, when I officially joined the Web QA team, I started working on breaking the plugin up into smaller plugins, each with a single purpose. The first to be released was the HTML report generation (pytest-html), which generates a single file report as an alternative to the existing JUnit report or console output. The plugin was written such that the report can be enhanced by other plugins, which ultimately allows us to include screenshots and other useful things in the report.

Next up was the variables injection (pytest-variables). This was needed primarily because we have tests that require an existing user account in the application under test. We couldn’t simply hard-code these credentials into our tests, because our tests are open source, and if we exposed these credentials someone may be able to use them and adversely affect our test results. With this plugin we are able to store our credentials in a private JSON file that can be simply referenced from the command line.

The final plugin was for browser provisioning (pytest-selenium). This started as a fork of the original plugin because much of the code already existed. There were a number of improvements, such as providing direct access to the Selenium object in tests, and avoiding setting a default implicit wait. In addition to supporting Sauce Labs, we also added support for BrowserStack and TestingBot.

Now that pytest-selenium has been released, we have started to migrate our own projects away from pytest-mozwebqa. The migration is relatively painless, but does involve changes to tests. If you’re a user of pytest-mozwebqa you can check out a few examples of the migration. There will no longer be any releases of pytest-mozwebqa and I will soon be marking this project as deprecated.

The most rewarding consequence of breaking up the plugins is that we’ve already seen individual contributors adopting and submitting patches. If you’re using any of these plugins let us know – I always love hearing how and where our tools are used!

December 22, 2015 09:45 AM

mozregression updates

Release 2.1.0 and GUI release 0.7.0

This is a minor release of the command-line and GUI mozregression tools.

On the command line side:

And other minor features and fixes: bug 1195390, bug 1231745, bug 1232879, bug 1233649 and bug 1233905.

On the GUI side:

And other fixes: bug 1232660 and bug 1233657

Thanks to Saurabh Singhal and GopianiS! They are new mozregression contributors who helped me with this release.

December 22, 2015 12:00 AM

December 18, 2015

Geoff Brown

Recording and replaying mochitests with rr and mach

rr is a lightweight debugging tool that allows program execution to be recorded and subsequently replayed and debugged. gdb-based debugging of recordings is enhanced by reverse execution.

rr can be used to record and replay Firefox and Firefox tests on Linux. See If you have rr installed and have a Linux Debug build of Firefox handy, recording a mochitest is as simple as:

  ./mach mochitest --debugger=rr ...

For example, to record a single mochitest:

  ./mach mochitest testing/mochitest/tests/Harness_sanity/test_sanitySimpletest.html \
    --keep-open=false --debugger=rr

Even better, use –run-until-failure to repeat the mochitest until an intermittent failure occurs:

  ./mach mochitest testing/mochitest/tests/Harness_sanity/test_sanitySimpletest.html \
    --keep-open=false --run-until-failure --debugger=rr

To replay and debug the most recent recording:

  rr replay

Similar techniques can be applied to reftests, xpcshell tests, etc.

For a fun and simple experiment, you can update a test to fail randomly, maybe based on Math.random(). Run the test in a loop or with –run-until-failure to reproduce your failure, then replay: Your “random” failure should occur at exactly the same point in execution on replay.

In recent weeks, I have run many mochitests on my laptop in rr, hoping to improve my understanding of how well rr can record and replay intermittent test failures.

rr has some, but only a little, effect on test run-time. I can normally run mochitest-1 via mach on my laptop in about 17 minutes; with rr, that increases to about 22 minutes (130% of normal). That’s consistent with :roc’s observations at

I observed no difference in test results, when running on my laptop: the same tests passed and failed with or without rr, and intermittent failures occurred with approximately the same frequency with or without rr. (This may not be universal; others have noted differences:

So my experience with rr has been very encouraging: If I can reproduce an intermittent test failure on my laptop, I can record it with rr, then debug it at my leisure and benefit from rr “extras” like reverse execution. This seems great!

I still have a concern about the practical application of rr to recording intermittent failures reported on treeherder…I’ll try to write a follow-up post on that soon.

December 18, 2015 06:47 PM

December 05, 2015

Alice Scarpa

How I made Treeherder do something it was not meant to

For the past couple of months I have been working on integrating Try Extender with Treeherder. The goal was to add an “Add new jobs” button to Treeherder that would display every possible job for that push. Users would then be able to click on the jobs they want to trigger them.

It was a fun project in which I had a lot of help from the Treeherder team and I ended up learning a little about how TH works.

How Treeherder shows jobs

For every push, Treeherder makes a request to its API to obtain a JSON object with every job for that push and their respective symbols, status, types, platforms and whatever else is needed to correctly display them. Every single one of these jobs has an id and it’s in a row in Treeherder’s job database.

Buildbot jobs enter TH’s job database as part of the ETL layer. Celery tasks parse JSON files that are generated every minute by BuildAPI.

Runnable jobs database

Treeherder already knows how to get a list of jobs from an API endpoint and display them in the right places (if you are curious, mapResultSetJobs carries most of the weight). All I needed to do was add a new API endpoint with the list of every possible job (and the associated information).

To feed the information to the new endpoint, I created a table of runnable jobs. Jobs enter this new table through a daily task that downloads and processes allthethings.json.

Setting things up

With the database part ready, some things had to be done on the UI side. An (extremely reasonable) assumption made by Treeherder is that it will only show jobs that exist. Since runnable jobs don’t exist, I had to create a new type of job button that would not open the information panel and that would allow users to click on several jobs at the same time.

The triggering part was done by sending Pulse messages to Pulse Actions, which would then schedule jobs using mozci and releng’s amazing BuildBot Bridge (armenzg did a great job adding BBB support to mozci).

Possible improvements

The UX is not very intuitive.

Selecting several jobs is very annoying. One idea to fix that is to have a keyboard shortcut to “select all visible jobs”, so users could use the search box to filter only the jobs they wanted (e.g. “e10s”) and select everything that is showing.

Known problems

Since the triggering part happens in Pulse Actions and the selecting part happens in Treeherder, we don’t tell users what happened with their requests. Until bug 1032163 lands, only the push author and people with an “” email address will be able to extend pushes. Right now we have no way of telling users that their request was denied.

We can schedule test jobs when no build job exists, and we can trigger test jobs when the build job is already completed. But when the build job is currently running/pending, we don’t trigger anything. We could either trigger an additional build job or do nothing, and we choose to do nothing to avoid triggering costly unnecessary build jobs.

What about TaskCluster jobs?

Currently “Add new jobs” only supports triggering Buildbot jobs. What is needed to support TaskCluster jobs? 2 things:

If anyone is interested in working on this, please ping me (I’m adusca on IRC), or we can talk more about it in Mozlando ;)

December 05, 2015 12:00 AM

December 04, 2015

mozregression updates

GUI Release 0.6.0

After mozregression 2.0, it is time for the GUI to follow!

0.6.0 GUI release is based on the changes from mozregression 2.0: the bisection flow is now updated, and starting a bisection should be a lot easier since it does not ask anymore for a nightly or inbound bisection kind - simply choose an application, possibly a branch and some options (build type, bits) then choose the regression range based on dates, release numbers, build ids or raw changesets.

That’s all. :)

All in all, a great simplification of the interface and more power. Give it a try!

December 04, 2015 12:00 AM

December 03, 2015

Henrik Skupin

Results of the Firefox Automation Survey

November 23rd I blogged about the active survey covering the information flow inside our Firefox Automation team. This survey was open until November 30th and I thank everyone of the participants which have taken the time to get it filled out. In the following you can find the results:

Most of the contributors who are following our activities are with Mozilla for the last 3 years. Whereby half of them joined less than a year ago. There is also a 1:1 split between volunteers and paid staff members. This is most likely because of the low number of responses, but anyway increasing the number of volunteers is certainly something we want to follow-up on in the next months.

The question about which communication channel is preferred to get the latest news got answered with 78% for the automation mailing list. I feel that this is a strange result given that we haven’t really used that list for active discussions or similar in the past months. But that means we should put more focus on the list. Beside that also 55% listening our activities on Bugzilla via component watchers. I would assume that those people are mostly our paid staff who kinda have to follow each others work regarding reviews, needinfo requests, and process updates. 44% of all read our blog posts on the Mozilla A-Team Planet. So we will put more focus in the future to both blog posts and discussions on the mailing list.

More than half of our followers check for updates at least once a day. So when we get started with interesting discussions I would expect good activity throughout the day.

44% of all feel less informed about our current activities. Another 33% answered this question with ‘Mostly’. So it’s a clear indication what I already thought and which clearly needs action on our side to be more communicative. Doing this might also bring more people into our active projects, so mentoring would be much more valuable and time-effective as handling any drive-by projects which we cannot fully support.

A request for the type of news we should do more is definitely for latest changes and code landings from contributors. This will ensure people feel recognized and contributors will also know each others work, and see the effectiveness in regards of our project goals. But also discussions about various automation related topics (as mentioned already above) are highly wanted. Other topics like quarterly goals and current status updates are also wanted and we will see how we can do that. We might be able to fold those general updates into the Engineering Productivity updates which are pushed out twice a month via the A-Team Planet.

Also there is a bit of confusion about the Firefox Automation team and how it relates to the Engineering Productivity team (formerly A-Team). Effectively we are all part of the latter, and the “virtual” Automation team has only been created when we got shifted between the A-Team and QA-Team forth and back. This will not happen anymore, so we agreed on to get rid of this name.

All in all there are some topics which will need further discussions. I will follow-up with another blog post soon which will show off our plans for improvements and how we want to work to make it happen.

December 03, 2015 11:59 AM

December 01, 2015

mozregression updates

Release 2.0.0

2.0.0 is a major release of mozregression, as we changed the bisection flow based on ideas in the post I wrote a couple of weeks ago.

Now mozregression will automatically detect a merge commit, and switch to bisect in the branch where the merged commits comes from. So mozilla-inbound is no longer the default for Firefox when bisecting by date is done, since there is no default now.

Based on that, we have been able to simplify the overall usage of mozregression:

Those changes adds some new possibilities to bisect which were not available before, like bisecting using changesets on mozilla-central, only specify a good changeset (the bad changeset will be implied, and will be the most recent one).

Some examples:

# bisect using dates
mozregression -g 2015-11-20 -b 2015-11-25  # implied branch is m-c
mozregression -g 2015-11-20 -b 2015-11-25 --repo inbound
# bisect using changesets
mozregression -g dcd5230c4ce1 -b 931721112d8e  # implied branch is m-i
mozregression -g 1b2e15608f34 -b abbd213422a5 --repo m-c
# use debug builds
mozregression -g 2015-11-20 -b 2015-11-25 -B debug
mozregression -g dcd5230c4ce1 -b 931721112d8e -B debug
# of course, --launch works the same way now
mozregression --launch abbd213422a5 --repo m-c
mozregression --launch 2015-11-25 --repo m-i -B debug

Just keep in mind that when you use a changeset, the default branch will be the default integration branch for the application instead of the release branch. For firefox, mozilla-inbound will be the default when you use a changeset, and mozilla-central will be used otherwise. This is historical and we may change that in the future - for now just keep that in mind, or always specify a branch to be sure.

See Bugs 1095058, 1210013, 1225544, 1228951, 1225541 and 1228857 for a description of technical implementation.

December 01, 2015 12:00 AM

November 26, 2015

Armen Zambrano G. (@armenzg)

Mozhginfo/Pushlog client released

If you've ever spent time trying to query metadata from hg with regards to revisions, you can now use a Python library we've released to do so.

In bug 1203621 [1], our community contributor @MikeLing has helped us release the module we had written for Mozilla CI tools.

You can find the pushlog_client package in here [3] and you can find the code in here [4]

Thanks MikeLing!


Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

November 26, 2015 03:17 PM

November 24, 2015

Armen Zambrano G. (@armenzg)

Welcome F3real, xenny and MikeLing!

As described by jmaher, we started this week our first week of mozci's quarter of contribution.

I want to personally welcome Stefan, Vaibhav and Mike to mozci. We hope you get to learn and we thank you for helping Mozilla move forward in this corner of our automation systems.

I also want to give thanks to Alice for committing at mentoring. This could not be possible without her help.

Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

November 24, 2015 05:58 PM

Mozilla CI tools meet up

In order to help the contributors' of mozci's quarter of contribution, we have set up a Mozci meet up this Friday.

If you're interested on learning about Mozilla's CI, how to contribute or how to build your own scheduling with mozci come and join us!

9am ET -> other time zones
Vidyo room:

Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

November 24, 2015 05:52 PM

mozregression updates

Release 1.2.0 and GUI release 0.5.0

Minor releases of the command line and GUI mozregression tools.

On the command line side:

mozregression --launch 20151102030241

On the GUI side:

November 24, 2015 12:00 AM

November 23, 2015

Henrik Skupin

Survey about sharing information inside the Firefox Automation team

Within the Firefox Automation team we were suffering a bit in sharing information about our work over the last couple of months. That mainly happened because I was alone and not able to blog more often than once in a quarter. The same applies to our dev-automation mailing list which mostly only received emails from Travis CI with testing results.

Given that the team has been increased to 4 people now (beside me this is Maja Frydrychowicz, Syd Polk, and David Burns, we want to be more open again and also trying to get more people involved into our projects. To ensure that we do not make use of the wrong communication channels – depending where most of our readers are – I have setup a little survey. It will only take you a minute to go through but it will help us a lot to know more about the preferences of our automation geeks. So please take that little time and help us.

The survey can be found here and is open until end of November 2015:

Thank you a lot!

November 23, 2015 09:40 PM

November 20, 2015

Geoff Brown

Running and debugging Firefox for Android with mach

Recent updates to mach provide support for running and debugging Firefox for Android.

When run from a Firefox for Android context, ‘mach run’ starts Firefox on a connected Android device. As with other Android mach commands, if no device is found, mach offers to start an emulator, and if Firefox is not installed, mach offers to install it.

gbrown@mozpad:~/src$ ./mach run
No Android devices connected. Start an emulator? (Y/n) y 
Starting emulator running Android 4.3...
It looks like Firefox is not installed on this device.
Install Firefox? (Y/n) y
Installing Firefox. This may take a while...
 1:22.97 /usr/bin/make -C . -j8 -s -w install
 1:32.04 make: Entering directory `/home/gbrown/objdirs/droid'
 1:47.48 2729 KB/s (42924584 bytes in 15.358s)
 1:48.22     pkg: /data/local/tmp/
 2:05.97 Success
 2:06.34 make: Leaving directory `/home/gbrown/objdirs/droid'
Starting: Intent { act=android.activity.MAIN cmp=org.mozilla.fennec_gbrown/.App }

Parameters can be passed to Firefox on the command line. For example, ‘mach run –guest’ starts Firefox in guest mode.

mach also supports gdb-based debugging with JimDB, :jchen’s celebrated fork of gdb for Firefox for Android. ‘mach run –debug’ starts JimDB. If necessary, mach will even fetch, install, and configure JimDB for you.

  $ ./mach run --debug
  JimDB (arm) not found: /home/gbrown/.mozbuild/android-device/jimdb-arm does not exist
  Download and setup JimDB (arm)? (Y/n) y
  Installing JimDB (linux64/arm). This may take a while...
   * [new branch]      master     -> origin/master
   * [new tag]         gdbutils-2 -> gdbutils-2
   * [new tag]         initial-release -> initial-release
   1:45.57 /home/gbrown/.mozbuild/android-device/jimdb-arm/bin/gdb -q --args 

  Fennec GDB utilities
    (see utils/gdbinit and utils/gdbinit.local on how to configure settings)
  1. Debug Fennec (default)
  2. Debug Fennec with env vars and args
  3. Debug using jdb
  4. Debug content Mochitest
  5. Debug compiled-code unit test
  6. Debug Fennec with pid
  Enter option from above: 1

  New ADB device is "emulator-5554"
  Using device emulator-5554
  Using object directory: /home/gbrown/objdirs/droid
  Set sysroot to "/home/gbrown/.mozbuild/android-device/jimdb-arm/lib/emulator-5554".
  Updated solib-search-path.
  Ignoring BHM signal.
  Using package org.mozilla.fennec_gbrown.
  Launching org.mozilla.fennec_gbrown... Done
  Attaching to pid 674... Done
  Setting up remote debugging... Done

  Ready. Use "continue" to resume execution.
  : No such file or directory.

See for more info on JimDB.

November 20, 2015 04:05 PM

Joel Maher

Introducing the contributors for the MozCI Project

As I previously announced who will be working on Pulse Guardian, the Web Platform Tests Results Explorer, and the  Web Driver Infrastructure projects, I would like to introduce the contributors for the 4th project this quarter, Mozilla CI Tools – Polish and Packaging:

* MikeLing (:mikeling on IRC) –

What interests you in this specific project?

As its document described, Mozilla CI Tools is designed to allow interacting with the various components which compose Mozilla’s Continuous Integration. So, I think get involved into it can help me know more about how Treeherder and Mozci works and give me better understanding of A-team.

What do you plan to get out of this after 8 weeks?

Keep try my best to contribute! Hope I can push forward this project with Armen, Alice and other contributors in the furture :)

Are there any interesting facts/hobbies that you would like to share so others can enjoy reading about you?

I’m a guy who would like to keep challenge myself and try new stuff.

* Stefan (:F3real on IRC) –

What interests you in this specific project?

I thought it would be good starting project and help me learn new things.

What do you plan to get out of this after 8 weeks?

Expand my knowledge and meet new people.

Are there any interesting facts/hobbies that you would like to share so others can enjoy reading about you?

I play guitar but I don’ t think that’s really interesting.

* Vaibhav Tulsyan (:xenny on IRC) –

What interests you in this specific project?

Continuous Integration, in general, is interesting for me.

What do you plan to get out of this after 8 weeks?

I want to learn how to work efficiently in a team in spite of working remotely, learn how to explore a new code base and some new things about Python, git, hg and Mozilla. Apart from learning, I want to be useful to the community in some way. I hope to contribute to Mozilla for a long term, and I hope that this helps me build a solid foundation.

Are there any interesting facts/hobbies that you would like to share so others can enjoy reading about you?

One of my hobbies is to create algorithmic problems from real-world situations. I like to think a lot about the purpose of existence, how people think about things/events and what affects their thinking. I like teaching and gaining satisfaction from others’ understanding.


Please join me in welcoming all the contributors to this project and the previously mentioned ones as they have committed to work on a larger project with their free time!

November 20, 2015 03:37 PM

Introducing a contributor for the WebDriver Infrastructure project

As I previously announced who will be working on Pulse Guardian and the Web Platform Tests Results Explorer, let me introduce who will be working on Web Platform Tests – WebDriver Infrastructure:

* Ravi Shankar (:waffles on IRC) –

What interests you in this specific project?

There are several. Though I love coding, I’m usually more inclined to Python & Rust (so, a “Python project” is what excited me at first). Then, my recently-developed interest in networking code (ever since my work on a network-related issue in Servo), and finally, I’m very curious about how we’re establishing the Python-JS communication and emulate user inputs.

What do you plan to get out of this after 8 weeks?

Over the past few months of my (fractional) contributions to Mozilla, I’ve always learned something useful whenever I finish working on a bug/issue. Since this is a somewhat “giant” implementation that requires more time and commitment, I think I’ll learn some great deal of stuff in relatively less time (which is what excites me).

Are there any interesting facts/hobbies that you would like to share so others can enjoy reading about you?

Well, I juggle, or I (try to) reproduce some random music in my flute (actually, a Bansuri – Indian flute) when I’m away from my keyboard.


We look forward to working with Ravi over the next 8 weeks.  Please say hi in irc when you see :waffles in channel :)

November 20, 2015 03:25 PM

Introducing 2 contributors for the Web Platform Tests project

As I previously announced who will be working on Pulse Guardian, let me introduce who will be working on Web Platform Tests – Results Explorer:

* Kalpesh Krishna (:martianwars on irc) –

What interests you in this specific project?

I have been contributing to Mozilla for a couple of months now and was keen on taking up a project on a slightly larger scale. This particular project was recommended to me by Manish Goregaokar. I had worked out a few issues in Servo prior to this and all involved Web Platform Tests in some form. That was the initial motivation. I find this project really interesting as it gives me a chance to help build an interface that will simplify browser comparison so much! This project seems to have more of planning rather than execution, and that’s another reason that I’m so excited! Besides, I think this would be a good chance to try out some statistics / data visualization ideas I have, though they might be a bit irrelevant to the goal.

What do you plan to get out of this after 8 weeks?

I plan to learn as much as I can, make some great friends, and most importantly make a real sizeable contribution to open source :)

Are there any interesting facts/hobbies that you would like to share so others can enjoy reading about you?

I love to star gaze. Constellations and Messier objects fascinate me. Given a chance, I would love to let my imagination run wild and draw my own set of constellations! I have an unusual ambition in life. Though a student of Electrical Engineering, I have always wanted to own a chocolate factory (too much Roald Dahl as a child) and have done some research regarding the same. Fingers crossed! I also love to collect Rubiks Cube style puzzles. I make it a point to increase my collection by 3-4 puzzles every semester and learn how to solve them. I’m not fast at any of them, but love solving them!

* Daniel Deutsch

What interests you in this specific project?

I am really interested in getting involved in Web Standards. Also, I am excited to be involved in a project that is bigger than itself–something that spans the Internet and makes it better for everyone (web authors and users).

What do you plan to get out of this after 8 weeks?

As primarily a Rails developer, I am hoping to expand my skill-set. Specifically, I am looking forward to writing some Python and learning more about JavaScript. Also, I am excited to dig deeper into automated testing. Lastly, I think Mozilla does a lot of great work and am excited to help in the effort to drive the web forward with open source contribution.

Are there any interesting facts/hobbies that you would like to share so others can enjoy reading about you?

I live in Brooklyn, NY and have terrible taste in music. I like writing long emails, running, and Vim.


We look forward to working with these great 2 hackers over the next 8 weeks.

November 20, 2015 03:20 PM

Introducing a contributor for the Pulse Guardian project

3 weeks ago we announced the new Quarter of Contribution, today I would like to introduce the participants.  Personally I really enjoy meeting new contributors and learning about them. It is exciting to see interest in all 4 projects.  Let me introduce who will be working on Pulse Guardian – Core Hacker:

Mike Yao

What interests you in this specific project?

Python, infrastructure

What do you plan to get out of this after 8 weeks?

Continue to contribute to Mozilla

Are there any interesting facts/hobbies that you would like to share so others can enjoy reading about you?

Cooking/food lover, I was chef long time ago. Free software/Open source and Linux changed my mind and career.


I do recall one other eager contributor who might join in late when exams are completed, meanwhile, enjoy learning a bit about Mike Yao (who was introduced to Mozilla by Mike Ling who did our first every Quarter of Contribution).

November 20, 2015 03:14 PM

November 14, 2015

Julien Pagès

mozregression – new way for handling merges

I am currently investigating how we can make mozregression smarter to handle merges, and I will explain how in this post.


While bisecting builds with mozregression on mozilla-central, we often end up with a merge commit. These commits often incorporate many individual changes, consider for example this url for a merge commit. A regression will be hard to find inside such a large range of commits.

How mozregression currently works

Once we reach a one day range by bisecting mozilla-central or a release branch, we keep the most recent commit tested, and we use that for the end of a new range to bisect mozilla-inbound (or another integration branch, depending on the application) The beginning of that mozilla-inbound range is determined by one commit found 4 days preceding the date of the push of the commit (date pushed on mozilla-central) to be sure we won’t miss any commit in mozilla-central.

But there are multiple problems. First, it is not always the case that the offending commit really comes from m-i. It could be from any other integration branch (fx-team, b2g-inbound, etc). Second, bisecting over a 4 days range in mozilla-inbound may involve testing a lot of builds, with some that are useless to test.

Another approach

How can we improve this ? As just stated, there are two points that can be improved:

So, how can this be achieved ? Here is my current approach (technical):

  1. Once we are done with the nightlies (one build per day) from a bisection from m-c or any release branch, switch to use taskcluster to download possible builds between. This way we reduce the range to two pushes (one good, one bad) instead of a full day. But since we tested them both, only the commits in the most recent push may contain the regression.
  2. Read the commit message of the top most commit in the most recent push. If it does not looks like a merge commit, then we can’t do anything (maybe this is not a merge, then we are done).
  3. We have a merge push. So now we try to find the exact commits around, on the branch where the merged commits come from.
  4. Bisect this new push range using the changesets and the branch found above, reduce that range and go to 2.

Let’s take an example:

mozregression -g 2015-09-20 -b 2015-10-10

We are bisecting firefox, on mozilla-central. Let’s say we end up with a range 2015-10-01 – 2015-10-02. This is how the pushlog will looks like at the end, 4 pushes and more than 250 changesets.

Now mozregression will automatically reduce the range (still on mozilla-central) by asking you good/bad for those remaining pushes. So, we would end up with two pushes – one we know is good because we tested the top most commit, and the other we know is bad for the same reason. Look at the following pushlog, showing what is still untested (except for the merge commit itself) – 96 commits, coming from m-i.

And then mozregression will detect that it is a merge push from m-i, so automatically it will let you bisect this range of pushes from m-i. That is, our 96 changesets from m-c now converted to testable pushes in m-i. And we will end with a smaller range, for example this one where it will be easy to find our regression because this is one push without any merge.


Note that both methods for the example above would have worked. Mainly because we are ending in commits originated from m-i. I tried with another bisection, this time trying to find a commit in fx-team – in that case, current mozregression is simply out – but with the new method it was handled well.

Also using the current method, it would have required around 7 steps after reducing to the one day range for the example above. The new approach can achieve the same with around 5 steps.

Last but not least, this new flow is much more cleaner:

  1. start to bisect from a given branch. Reduce the range to one push on that branch.
  2. if we found a merge, find the branch, the new pushes, and go to 1 to bisect some more with this new data. Else we are done.

Is this applicable ?

Well, it relies on two things. The first one (and we already rely on that a bit currently) is that a merged commit can be found in the branch where it comes from, using the changeset. I have to ask vcs gurus to know if that is reliable, but from my tests this is working well.

Second thing it that we need to detect a merge commit – and from which branch commits comes from. Thanks to the consistency of the sheriffs in their commit messages, this is easy.

Even if it is not applicable everywhere for some reason, it appears that it often works. Using this technique would result in a more accurate and helpful bisection, with speed gain and increased chances to find the root cause of a regression.

This need some more thinking and testing, to determine the limits (what if this doesn’t work ? Should we/can we use the old method in that case ?) but this is definitely something I will explore more to improve the usefulness of mozregression.

November 14, 2015 09:42 AM

November 11, 2015

Joel Maher

Adventures in Task Cluster – running a custom Docker image

I needed to get compiz on the machine (bug 1223123), and I thought it should be on the base image.  So to take the path of most resistance, I dove deep into the internals of taskcluster/docker and figured this out.  To be fair, :ahal gave me a lot of pointers, in fact if I would have taken better notes this might have been a single pass to success vs. 3 attempts.

First let me explain a bit about how the docker image is defined and how it is referenced/built up.  We define the image to use in-tree.  In this case we are using taskcluster/docker-test:0.4.4 for the automation jobs.  If you look carefully at the definition in-tree, we have a Dockerfile which outlines who we inherit the base image from:

FROM          taskcluster/ubuntu1204-test-upd:

This means there is another image called ubuntu1204-test-upd, and this is also defined in tree which then references a 3rd image, ubuntu1204-test.  These all build upon each other creating the final image we use for testing.  If you look in each of the directories, there is a REGISTRY and a VERSION file, these are used to determine the image to pull, so in the case of wanting:

docker pull taskcluster/desktop-test:0.4.4

we would effectively be using:

docker pull {REGISTRY}/desktop-test:{VERSION}

For our use case, taskcluster/desktop-test is defined on  This means that you could create a new version of the container ‘desktop-test’ and use that while pushing to try.  In fact that is all that is needed.

First lets talk about how to create an image.  I found that I needed to create both a desktop-test and an ubuntu1204-test image on Docker Hub.  Luckily in tree there is a script which will take a currently running container and make a convenient package ready to upload, some steps would be:

  • docker pull taskcluster/desktop-test:0.4.4
  • docker run taskcluster/desktop-test:0.4.4
  • apt-get install compiz; # inside of docker, make modifications
  • # now on the host we prepare the new image (using elvis314 as the docker hub account)
  • echo elvis314 > testing/docker/docker-test/REGISTRY
  • echo 0.4.5 > testing/docker/docker-test/VERSION  # NOTE: I incremented the version
  • cd testing/docker
  • docker-test # go run a 5K
  • docker push elvis314/docker-test # go run a 10K

those are the simple steps to update an image, what we want to do is actually verify this image has what we need.  While I am not an expert in docker, I do like to keep my containers under control, so I do a |docker ps -a| and then |docker rm <cid>| for any containers that are old.  Now to verify I do this:

  • docker pull elvis314/desktop-test:0.4.5
  • docker run elvis314/desktop-test:0.4.5
  • compiz # this verifies my change is there, I should get an error about display not found!

I will continue on here assuming things are working.  As you saw earlier, I had modifed filed in testing/docker/desktop-test, these should be part of a patch to push to try.  In fact that is all the magic.  to actually use compiz successfully, I needed to add this to to launch |compiz &| after initializing Xvfb.

Now when you push to try with your patch, any tasks that used taskcluster/desktop-test before will use the new image (i.e. elvis314/desktop-test).  In this case I was able to see the test cases that opened dialogs and windows pass on try!

November 11, 2015 03:03 PM

Dan Minor

Autoland Update

Today I made the first successful autolanding to mozilla-inbound from MozReview. We’ve also been dogfooding autolanding to version-control-tools for the past few weeks without running into any problems, although the volume of commits to mozilla-inbound will give the automatic rebase and retry code more exercise than it receives when landing to version-control-tools.

Bug 1220214 tracks the workflow and user interface improvements we want to make before we enable autolanding to mozilla-inbound for everyone. The largest (and riskiest) change we want to make is to enable automatic rewriting of commit summaries to reflect who actually granted a “ship-it” in MozReview.

Without this work, people would have to amend their commits prior to landing to replace any r? with a r=, which makes autolanding much less useful. I recently fixed Bug 1160479 which was the Autoland service portion of the rewriting. Glob is nearly done with Bug 1220232 which is the MozReview portion which determines the new commit summary and provide a confirmation dialog to the user.

November 11, 2015 01:27 PM

November 09, 2015

Joel Maher

Adventures in Task Cluster – Running tests locally

There is a lot of promise around Taskcluster (the replacement for BuildBot in our CI system at Mozilla) to be the best thing since sliced bread.  One of the deliverables on the Engineering Productivity team this quarter is to stand up the Linux debug tests on Taskcluster in parallel to running them normally via Buildbot.  Of course next quarter it would be logical to turn off the BuildBot tests and run tests via Taskcluster.

This post will outline some of the things I did to run the tests locally.  What is neat is that we run the taskcluster jobs inside a Docker image (yes this is Linux only), and we can download the exact OS container and configuration that runs the tests.

I started out with a try server push which generated some data and a lot of failed tests.  Sadly I found that the treeherder integration was not really there for results.  We have a fancy popup in treeherder when you click on a job, but for taskcluster jobs, all you need is to find the link to inspect task.  When you inspect a task, it takes you to a task cluster specific page that has information about the task.  In fact you can watch a test run live (at least from the log output point of view).  In this case, my test job is completed and I want to see the errors in the log, so I can click on the link for live.log and search away.  The other piece of critical information is the ‘Task‘ tab at the top of the inspect task page.  Here you can see the details about the docker image used, what binaries and other files were used, and the golden nugget at the bottom of the page, the “Run Locally” script!  You can cut and paste this script into a bash shell and theoretically reproduce the exact same failures!

As you can imagine this is exactly what I did and it didn’t work!  Luckily in the #taskcluster channel, there were a lot of folks to help me get going.  The problem I had was I didn’t have a v4l2loopback device available.  This is interesting because we need this in many of our unittests and it means that our host operating system running docker needs to provide video/audio devices for the docker container to use.  Now is time to hack this up a bit, let me start:

first lets pull down the docker image used (from the run locally script):

docker pull 'taskcluster/desktop-test:0.4.4'

next lets prepare my local host machine to run by installing/setting up v4l2loopback:

sudo apt-get install v4l2loopback-dkms

sudo modprobe v4l2loopback devices=2

Now we can try to run docker again, this time adding the –device command:

docker run -ti \
  --name "${NAME}" \
  --device=/dev/video1:/dev/video1 \
  -e MOZHARNESS_SCRIPT='mozharness/scripts/' \
  -e MOZHARNESS_CONFIG='mozharness/configs/unittests/ mozharness/configs/
' \
  -e GECKO_HEAD_REV='5e76c816870fdfd46701fd22eccb70258dfb3b0c' \

Now when I run the test command, I don’t get v4l2loopback failures!

bash /home/worker/bin/ --no-read-buildbot-config '--installer-url=' '--test-packages-url=' '--download-symbols=ondemand' '--mochitest-suite=browser-chrome-chunked' '--total-chunk=7' '--this-chunk=1'

In fact, I get the same failures as I did when the job originally ran :)  This is great, except for the fact that I don’t have an easy way to run the test by itself, debug, or watch the screen- let me go into a few details on that.

Given a failure in browser/components/search/test/browser_searchbar_keyboard_navigation.js, how do we get more information on that?  Locally I would do:

./mach test browser/components/search/test/browser_searchbar_keyboard_navigation.js

Then at least see if anything looks odd in the console, on the screen, etc.  I might look at the test and see where we are failing at to give me more clues.  How do I do this in a docker container?  The command above to run the tests, calls, which then calls as the user ‘worker’ (not as user root).  This is important that we use the ‘worker’ user as the pactl program to find audio devices will fail as root.  Now what happens is we setup the box for testing, including running pulseaudio, Xfvb, compiz (after bug 1223123), and bootstraps mozharness.  Finally we call the mozharness script to run the job we care about, in this case it is ‘mochitest-browser-chrome-chunked’, chunk 1.  It is important to follow these details because mozharness downloads all python packages, tools, firefox binaries, other binaries, test harnesses, and tests.  Then we create a python virtualenv to setup the python environment to run the tests while putting all the files and unpacking them in the proper places.  Now mozharness can call the test harness (python –browser-chrome …)  Given this overview of what happens, it seems as though we should be able to run: <params> –test-path browser/components/search/test

Why this doesn’t work is that mozharness has no method for passing in a directory or single test, let along doing other simple things that |./mach test| allows.  In fact, in order to run this single test, we need to:

Of course most of this is scripted, how can we take advantage of our scripts to set things up for us?  What I did was hack the locally to not run mozharness and instead echo the command.  Likewise with the mozharness script to echo the test harness call instead of calling it.  Here is the commands I ended up using:

  • bash /home/worker/bin/ --no-read-buildbot-config '--installer-url=' '--test-packages-url=' '--download-symbols=ondemand' '--mochitest-suite=browser-chrome-chunked' '--total-chunk=7' --this-chunk=1
  • #now that it failed, we can do:
  • cd workspace/build
  • . venv/bin/activate
  • cd ../build/tests/mochitest
  • python –app ../../application/firefox/firefox –utility-path ../bin –extra-profile-file ../bin/plugins –certificate-path ../certs –browser-chrome browser/browser/components/search/test/
  • # NOTE: you might not want –browser-chrome or the specific directory, but you can adjust the parameters used

This is how I was able to run a single directory, and then a single test.  Unfortunately that just proved that I could hack around the test case a bit and look at the output.  In docker there is no simple way to view the screen.   To solve this I had to install x11vnc:

apt-get install x11vnc

Assuming the Xvfb server is running, you can then do:

x11vnc &

This allows you to connect with vnc to the docker container!  The problem is you need the ipaddress.  I then need to get the ip address from the host by doing:

docker ps #find the container id (cid) from the list

docker inspect <cid> | grep IPAddress

for me this is and now from my host I can do:


This is great as I can now see what is going on with the machine while the test is running!

This is it for now.  I suspect in the future we will make this simpler by doing:

Stay tuned for my next post on how to update your own custom TaskCluster image- yes it is possible if you are patient.

November 09, 2015 08:49 PM

November 05, 2015

Jonathan Griffin

Engineering Productivity Update, November 5, 2015

It’s the first week of November, and because of the December all-hands and the end-of-year holidays, this essentially means the quarter is half over. You can see what the team is up to and how we’re tracking against our deliverables with this spreadsheet.

Highlights gps did some interesting work investigating ways to increase cloning performance on Windows; it turns out closing files which have been appended is a very expensive process there. He also helped roll out bundle-related cloning improvements in Mercurial 3.6.

Community: jmaher has posted details about our newest Quarter of Contribution. One of our former Outreachy interns, adusca, has blogged about what she gets out of contributing to open source software.

MozReview and Autoland: dminor blogged about the most recent MozReview work week in Toronto. Meanwhile, mcote is busy trying to design a more intuitive way to deal with parent-child review requests. And glob, who is jumping in to help out with MozReview, has created a high-level diagram sketching out MozReview’s primary components and dependencies.

Autoland has been enabled for the version-control-tools repo and is being dogfooded by the team. We hope to have it turned on for landings to mozilla-inbound within a couple of weeks.

Treeherder: the team is in London this week working on the automatic starring project. They should be rolling out an experimental UI soon for feedback from sheriffs and others. armenzg has fixed several issues with automatic backfilling so it should be more useful.

Perfherder: wlach has blogged about recent improvements to Perfherder, including the ability to track the size of the Firefox installer.

Developer Workflows: gbrown has enabled |mach run| to work with Android.

TaskCluster Support: the mochitest-gl job on linux64-debug is now running in TaskCluster side-by-side with buildbot. Work is ongoing to green up other suites in TaskCluster. A few other problems (like failure to upload structured logs) need to be fixed before we can turn off the corresponding buildbot jobs and make the TaskCluster jobs “official”.

e10s Support: we are planning to turn on e10s tests on Windows 7 as they are greened up; the first job which will be added is the e10s version of mochitest-gl, and the next is likely mochitest-devtools-chrome. To help mitigate capacity impacts, we’ve turned off Windows XP tests by default on try in order to allow us to move some machines from the Windows XP pool to the Windows 7 pool, and some machines have already been moved from the Linux 64 pool (which only runs Talos and Android x86 tests) to the Windows 7 pool. Combined with some changes recently made by Releng, Windows wait times are currently not problematic.

WebDriver: ato, jgraham and dburns recently went to Japan to attend W3C TPAC to discuss the WebDriver specification. They will be extending the charter of the working group to get it through to CR. This will mean certain parts of the specification need to finished as soon as possible to start getting feedback.

The Details


Mobile Automation

Firefox and Media Automation


Perfherder/Performance Testing

TaskCluster Support

General Automation



November 05, 2015 01:59 PM

November 04, 2015

William Lachance

Perfherder: Onward!

In addition to the database refactoring I mentioned a few weeks ago, some cool stuff has been going into Perfherder lately.

Tracking installer size

Perfherder is now tracking the size of the Firefox installer for the various platforms we support (bug 1149164). I originally only intended to track Android .APK size (on request from the mobile team), but installer sizes for other platforms came along for the ride. I don’t think anyone will complain. :)

Screen Shot 2015-11-03 at 5.28.48 PM


Just as exciting to me as the feature itself is how it’s implemented: I added a log parser to treeherder which just picks up a line called “PERFHERDER_DATA” in the logs with specially formatted JSON data, and then automatically stores whatever metrics are in there in the database (platform, options, etc. are automatically determined). For example, on Linux:

PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"subtests": [{"name": "", "value": 99030741}], "name": "installer size", "value": 55555785}]}

This should make it super easy for people to add their own metrics to Perfherder for build and test jobs. We’ll have to be somewhat careful about how we do this (we don’t want to add thousands of new series with irrelevant / inconsistent data) but I think there’s lots of potential here to be able to track things we care about on a per-commit basis. Maybe build times (?).

More compare view improvements

I added filtering to the Perfherder compare view and added back links to the graphs view. Filtering should make it easier to highlight particular problematic tests in bug reports, etc. The graphs links shouldn’t really be necessary, but unfortunately are due to the unreliability of our data — sometimes you can only see if a particular difference between two revisions is worth paying attention to in the context of the numbers over the last several weeks.

Screen Shot 2015-11-03 at 5.37.02 PM


Even after the summer of contribution has ended, Mike Ling continues to do great work. Looking at the commit log over the past few weeks, he’s been responsible for the following fixes and improvements:

Next up

My main goal for this quarter is to create a fully functional interface for actually sheriffing performance regressions, to replace alertmanager. Work on this has been going well. More soon.

Screen Shot 2015-11-04 at 10.41.26 AM

November 04, 2015 03:45 PM

Alice Scarpa

What I got from contributing to OSS

There are a lot of good altruistic reasons to contribute to Open Source Software, but this post focuses on my selfish reasons.

Learning Projects

I’m OK at reading books, implementing examples and doing exercises, but when it comes to thinking about good projects to get my hands dirty and implement stuff, I had a lot of trouble thinking of stuff to do. OSS provides an endless supply of bugs, projects and features to work on.

Code Reviews

Before I got started on OSS, the only person who ever really read my code was myself. Every patch I submitted to Mozilla was reviewed by at least one person, and that really improved my code. From running a Python linter plugin in Emacs to learning idiomatic ways of writing expressions, I learned a lot of good habits.


Whenever I was working on a bug, I could ask for help and someone would always answer, no matter if it was a problem specific to a bug or a general language/module/tool question. This way I was able to accomplish things that were unimaginable to me before.


Knowing someone is using a feature/tool I wrote is an amazing feeling. Even bug reports make me happy! I cherish every IRC mention of my projects.


Before I got started with OSS, all of my programming experience came from books and small projects. Contributing to OSS I got a chance to work on larger codebases, work with other people and play with technologies that I wouldn’t get to play by myself.


I’m now part of a very friendly community, full of people that I respect, like and trust. They help me a lot, and sometimes I even get to help back!


I used to be very afraid of not being good enough to contribute to OSS. I was not sure I was a real programmer. There were several bugs that I was completely sure I would not be able to fix, until I fixed them. Now I look back at what I did and I feel proud. I feel like maybe I really am a programmer.

If you are interested in long-term contributing, the A-team has some pretty cool contribution opportunities on the next quarter of contribution. Check it out!

November 04, 2015 12:00 AM

November 03, 2015

Joel Maher

Lost in Data – Episode 3 – digging into alerts from an uplift

Yesterday I recorded a session where I looked at alerts from an uplift.  I did a lot of rambling and not a lot of content, but there are a few interesting differences between uplift alerts and normal alerts:

If you want to take a look at this, the link is on


I do plan to do more episodes soon, a few topics of interest:

November 03, 2015 01:24 PM

October 31, 2015

Julien Pagès

mozregression 1.1.0 release

New release of mozregression, with some goodies!

See for more details and the full changelog.

October 31, 2015 10:04 AM

mozregression updates

Release 1.1.0

This new release of mozregression includes some new features:

And a few bugfixes also:

Thanks to Mikeling for being really active on some bugs here!

There is also a basic support for firefox os builds (flame, aries, simulator). Lots of work still needs to be done to make it really useful, (see bug 1205560) but it is now possible to bisect between dates or changesets on a given branch: mozregression will download the builds and ask you to flash it on the device.

# Regression finding by date range with aries-opt builds (defaults to b2g-inbound)
mozregression --app b2g-aries --bad 2015-09-10 --good 2015-09-07
# Regression finding on mozilla-inbound for debug builds
mozregression --app b2g-aries --build-type debug --bad 2015-09-10 --good 2015-09-07 \
              --inbound-branch mozilla-inbound
# Flame builds with a good and bad revision
mozregression --app b2g-flame --bad-rev c4bf8c0c2044 --good-rev b93dd434b3cd
# find more information
mozregression --help
mozregression --list-build-types

Thanks to Michael Shal, Naoki Hirata and others for helping me on this.

October 31, 2015 12:00 AM

October 29, 2015

Joel Maher

Looking for hackers interested in hacking for 6-8 weeks on a Quarter of Contribution project

Today I am happy to announce the second iteration of the Quarter of Contribution.  This will take place between November 23 and run until January 18th.

We are looking for contributors who want to tackle more bugs or a larger project and who are looking to prove existing skills or work on learning new skills.

There are 4 great projects that we have:

There are no requirements to be an accomplished developer.  Instead we are looking for folks who know the basics and want to improve.  If you are interested, please read about the program and the projects and ask questions to the mentors or in the #ateam channel on

Happy hacking!

October 29, 2015 08:26 PM

October 23, 2015

William Lachance

The new old Perfherder data model

I spent a good chunk of time last quarter redesigning how Perfherder stores its data internally. Here are some notes on this change, for posterity.

Perfherder’s data model is based around two concepts:

  1. Series signatures: A unique set of properties (platform, test name, suite name, options) that identifies a performance test.
  2. Series data: A set of measurements for a series signature, indexed by treeherder push and job information.

When it was first written, Perfherder stored the second type of data as a JSON-encoded series in a relational (MySQL) database. That is, instead of storing each datum as a row in the database, we would store sequences of them. The assumption was that for the common case (getting a bunch of data to plot on a graph), this would be faster than fetching a bunch of rows and then encoding them as JSON. Unfortunately this wasn’t really true, and it had some serious drawbacks besides.

First, the approach’s performance was awful when it came time to add new data. To avoid needing to decode or download the full stored series when you wanted to render only a small subset of it, we stored the same series multiple times over various time intervals. For example, we stored the series data for one day, one week… all the way up to one year. You can probably see the problem already: you have to decode and re-encode the same data structure many times for each time interval for every new performance datum you were inserting into the database. The pseudo code looked something like this for each push:

for each platform we're testing talos on:
  for each talos job for the platform:
    for each test suite in the talos job:
      for each subtest in the test suite:
        for each time interval in one year, 90 days, 60 days, ...:
           fetch and decode json series for that time interval from db
           add datapoint to end of series
           re-encode series as json and store in db

Consider that we have some 6 platforms (android, linux64, osx, winxp, win7, win8), 20ish test suites with potentially dozens of subtests… and you can see where the problems begin.

In addition to being slow to write, this was also a pig in terms of disk space consumption. The overhead of JSON (“{, }” characters, object properties) really starts to add up when you’re storing millions of performance measurements. We got around this (sort of) by gzipping the contents of these series, but that still left us with gigantic mysql replay logs as we stored the complete “transaction” of replacing each of these series rows thousands of times per day. At one point, we completely ran out of disk space on the treeherder staging instance due to this issue.

Read performance was also often terrible for many common use cases. The original assumption I mentioned above was wrong: rendering points on a graph is only one use case a system like Perfherder has to handle. We also want to be able to get the set of series values associated with two result sets (to render comparison views) or to look up the data associated with a particular job. We were essentially indexing the performance data only on one single dimension (time) which made these other types of operations unnecessarily complex and slow — especially as the data you want to look up ages. For example, to look up a two week old comparison between two pushes, you’d also have to fetch the data for every subsequent push. That’s a lot of unnecessary overhead when you’re rendering a comparison view with 100 or so different performance tests:

Screen Shot 2015-08-07 at 1.57.39 PM

So what’s the alternative? It’s actually the most obvious thing: just encode one database row per performance series value and create indexes on each of the properties that we might want to search on (repository, timestamp, job id, push id). Yes, this is a lot of rows (the new database stands at 48 million rows of performance data, and counting) but you know what? MySQL is designed to handle that sort of load. The current performance data table looks like this:

| Field          | Type             |
| id             | int(11)          |
| job_id         | int(10) unsigned |
| result_set_id  | int(10) unsigned |
| value          | double           |
| push_timestamp | datetime(6)      |
| repository_id  | int(11)          | 
| signature_id   | int(11)          | 

MySQL can store each of these structures very efficiently, I haven’t done the exact calculations, but this is well under 50 bytes per row. Including indexes, the complete set of performance data going back to last year clocks in at 15 gigs. Not bad. And we can examine this data structure across any combination of dimensions we like (push, job, timestamp, repository) making common queries to perfherder very fast.

What about the initial assumption, that it would be faster to get a series out of the database if it’s already pre-encoded? Nope, not really. If you have a good index and you’re only fetching the data you need, the overhead of encoding a bunch of database rows to JSON is pretty minor. From my (remote) location in Toronto, I can fetch 30 days of tcheck2 data in 250 ms. Almost certainly most of that is network latency. If the original implementation was faster, it’s not by a significant amount.

Screen Shot 2015-10-23 at 1.55.09 PM

Lesson: Sometimes using ancient technologies (SQL) in the most obvious way is the right thing to do. DoTheSimplestThingThatCouldPossiblyWork

October 23, 2015 06:28 PM

Dan Minor

MozReview Toronto Work Week

We’re just wrapping up another MozReview work week, this time in Toronto. Our main goal was to indoctrinate Glob into MozReview development as he is joining us for at least a few quarters. Since we reserve our fortnightly “Engineering Productivity Updates” for significant contributions, here is a list of my insignificant contributions from this week instead:

October 23, 2015 01:26 PM

October 22, 2015

Byron Jones

moving from bugzilla to mozreview

for the next couple of quarters (at least) i’ll be shifting my attention full time from bugzilla to mozreview. this switch involves a change of language, frameworks, and of course teams. i’m looking forward to new challenges.

one of the first things i’ve done is sketch out a high level architectural diagram of mozreview and its prime dependencies:

MozReview Architectural Diagram

mozreview exists as an extension to reviewboard, using bugzilla for user authentication, ldap to check commit levels, with autoland pushing commits automatically to try (and to mozilla-central soon).  there’s mecurial extensions on both the client and server to make pushing things easer, and there are plans to perform static analysis with bots.

Filed under: mozilla, mozreview

October 22, 2015 07:53 PM

Mark Côté

MozReview's Parental issues

As mentioned in my previous post on MozReview, one of the biggest sources of confusion is the way we present the “squashed” diffs, that is, the diff that show all of the changes in a commit series, the sum of all the proposed changes. We also refer to these as “parent” review requests, since they function as something to hold all the commits together. They are stored in MozReview as separate review requests, similar to the individual commits.

The confusion results from several things:

There are a few simple things we can do to fix these problems: use better link names, put a big “This is an overview of the commit series” message, and/or put a warning “You must review individual commits” on the review dialog. But really, we need to step back and think about the way we present the squashed diffs, and if they even make sense as a concept in MozReview.

To reiterate, squashed diffs provide a complete view of a whole commit series. The concept of a commit series doesn’t exist in core Review Board (nor does it exist in many other code-review tools), but it’s central to the idea of the repository-centric approach (like in GitHub pull requests). We added this concept by storing metadata resulting from pushes to tie commit series together with a parent, and we added UI elements like the commits table.

There are three broad ways we can deal with squashed diffs going forward. We need to settle on one and make the associated UI changes to make our model clear to users.

  1. Remove squashed diffs altogether.

    This is the simplest option. Squashed diffs aren’t actually technically necessary, and they can distract reviewers from the individual commits, which is where they should be spending most of their time, since, in most cases, this is how the code will be landing in the main repository. Some other repository-centric review tools, like Critic, don’t have the concept of an overview diff, so there are precedents. However, it might be a bit heavy handed to tell reviewers that they can’t view all the commits as a single diff (at least, without pulling them down locally).

  2. Continue to allow reviews, of some sort, on squashed diffs.

    This is what we have now: reviewers can leave reviews (at the moment, comments only) on squashed diffs. If we decide we want to continue to allow users to leave reviews on squashed diffs, we’ll need to both figure out a better UI to distinguish them from the individual commits and also settle several open questions:

    • Should reviewers be able to grant ship its (i.e. r+s) on squashed diffs? This would imply that the commits probably haven’t been reviewed individually, which would defeat the purpose of a commit-centric system. That said, reviewer time is very important, so we could have a trade off to support more work flows.

    • Conversely, should reviewers be able to leave comments on the parent diff? For simplicity, we could allow reviewers to leave a “ship it” review on a squashed diff that would apply to all commits but force them to leave any comments on diffs on the commits themselves. This would essentially remove the ability to review squashed diffs themselves but would leave the convenience of saying “this is all good”.

    • If we do want to allow review comments on squashed diffs, how should they be consolidated with the reviews on individual commits? Right now, reviews (general comments and comments on diffs) for the squashed diff and all commits are all on separate pages/views. Giving one view into all activity on a commit series would be ideal if we want to support squashed-diff reviews. Arguably, this would be valuable even if we didn’t have reviews on squashed diffs.

    For comparison, GitHub pull requests support this model. There are three tabs in a pull request: “Files changed”, which is the squashed diff; “Commits”, which is a list of commits with links to the individual commit diffs; and “Conversation”, which shows comments on the commits and on the squashed diff (along with other events like updates to the commits). The way they are presented is a little confusing (comments on the squashed diff are just labelled “<user> commented on the diff”, whereas comments on the diffs are of the form “<user> commented on <file> in <commit hash>”), but it is a useful single view. However, note that pull requests do not have the concept of a “ship it” or “r+”, which makes the GitHub interface simpler.

    This approach would support multiple reviewer work flows, but it is also the most complicated, both in terms of UX and technical implementation, and it waters down the philosophy behind MozReview.

  3. Provide read-only overview diffs.

    The third approach is to keep squashed diffs but make them read only. They could be used as reference, to get a big picture of the whole series, but since they are read only, they would be easily distinguishable from commits and would force reviewers to look at the individual commits. This is really just option 1 above, with a reference view of the whole series. It would be more work than option 1 but less than option 2, and would preserve the philosophy.

The MozReview team has been leaning towards option 3. We have a mock-up that strips away a lot of the UI that would be useless in this scenario and makes the intention clear. It’s not the prettiest, but it wouldn’t take too much work to get here:

However, we’d like to hear user feedback before making any decisions. Whichever option we go with, we’ll come up with a plan to get there that ideally will have incremental improvements, depending on the complexity of the full solution, so that we can start to fix things right away.

October 22, 2015 06:29 PM

October 21, 2015

Jonathan Griffin

Engineering Productivity Update, Oct 21, 2015

It’s Q4, and at Mozilla that means it’s planning season. There’s a lot of work happening to define a Vision, Strategy and Roadmap for all of the projects that Engineering Productivity is working on; I’ll share progress on that over the next couple of updates.


Build System: Work is starting on a comprehensive revamp of the build system, which should make it modern, fast, and flexible. A few bits of this are underway (like migration of remaining Makefiles to; more substantial progress is being planned for Q1 and the rest of 2016.

Bugzilla: Duo 2FA support is coming soon! The necessary Bugzilla changes has landed, we’re just waiting for some licensing details to be sorted out.

Treeherder: Improvements have been made to the way that sheriffs can backfill jobs in order to bisect a regression. Meanwhile, lots of work continues on backend and frontend support for automatic starring.

Perfherder and Performance Testing: Some optimizations were made to Perfherder which has made it more performant – no one wants a slow performance monitoring dashboard! jmaher and bc are getting close to being able to run Talos on real devices via Autophone; some experimental runs are already showing up on Treeherder.

MozReview and Autoland: It’s no longer necessary to have an LDAP account in order to push commits to MozReview; all that’s needed is a Bugzilla account. This opens the door to contributors using the system. Testing of Autoland is underway on MozReview’s dev instance – expect it to be available in production soon.

TaskCluster Migration: OSX cross-compiled builds are now running in TaskCluster and appearing in Treeherder as Tier-2 jobs, for debug and static checking. The TC static checking build with likely become the official build soon (and the buildbot build retired); the debug build won’t become official until work is done to enable existing test jobs to consume the TC build.

Work is progressing on enabling TaskCluster test jobs for linux64-debug; our goal is to have these all running side-by-side the buildbot jobs this quarter, so we can compare failure rates before turning off the corresponding buildbot jobs in Q1. Moving these jobs to TaskCluster enables us to chunk them to a much greater degree, which will offer some additional flexibility in automation and improve end-to-end times for these tests significantly.

Mobile Automation: All Android test suites that show in Treeherder can now be run easily using mach.

Dev Workflow: It’s now easier to create new web-platform-tests, thanks to a new |mach web-platform-tests-create| command.

e10s Support: web-platform-tests are now running in e10s mode on linux and OSX platforms. We want to turn these and other tests in e10s mode on for Windows, but have hardware capacity problems. Discussions are underway on how to resolve this in the short-term; longer-term plans include an increase in hardware capacity.

Test Harnesses: run-by-dir is now applied to all mochitest jobs on desktop. This improves test isolation and paves the way for chunking changes which we will use to improve end-to-end times and make bisection turnaround faster. Structured logging has been rolled out to Android reftests; Firefox OS reftests still to come.

ActiveData: Work is in progress to build out a model of our test jobs running in CI, so that we can identify pieces of job setup and teardown which are too slow and targets of possible optimization, and so that we can begin to predict the effects of changes to jobs and hardware capacities. Mercurial 3.6 will have built-in support for seeding clones from pre-generated bundle files, and will have improved performance for cloning, especially on Windows.

Marionette and WebDriver: Message sequencing is being added to Marionette; this will help prevent synchronization issues where the client mixes up responses. Client-side work is being done in both Python and node.js. ato wrote an article making a case against visibility checks in WebDriver.



Perfherder/Performance Testing


TaskCluster Support

Mobile Automation

Dev Workflow

Firefox and Media Automation

General Automation




October 21, 2015 05:58 PM

October 20, 2015

Henrik Skupin

Firefox Automation report – Q3 2015

It’s time for another Firefox Automation report! It’s incredible how fast a quarter passes by without that I have time to write reports more often. Hopefully it will change soon – news will be posted in a follow-up blog post.

Ok, so what happened last quarter for our projects.


One of my deliverables in Q3 was to create mozharness scripts for our various tests in the firefox-ui-tests repository, so that our custom runner scripts can be replaced. This gives us a way more stable system and additional features like crash report handling, which are necessary to reach the tier 2 level in Treeherder.

After some refactoring of the firefox ui tests, scripts for the functional and update tests were needed. But before those could be implemented I had to spent some time in refactoring some modules of mozharness to make them better configurable for non-buildbot jobs. All that worked pretty fine and finally the entry scripts have been written. Something special for them is that they even have different customers, so extra configuration files had to be placed. In detail it’s us who run the tests in Jenkins for nightly builds, and partly for release builds. On the other side Release Engineering want to run our update tests on their own hardware when releases have to be tested.

By the end of September all work has been finished. If you are interested in more details feel free to check the tracking bug 1192369.


Our Jenkins instance got lots of updates for various new features and necessary changes. All in all I pushed 27 commits which affected 53 files.

Here a list of the major changes:

October 20, 2015 07:42 PM

October 19, 2015

Armen Zambrano G. (@armenzg)

Fixes for Sheriffs' Treeherder backfilling and scheduling support

Sheriffs have the ability to backfill jobs on Treeherder (to determine the push where are regression starts) and the ability to run missing jobs on a push (due to coalescing/SETA).

Few weeks ago we added alerts to pulse_actions:

 and we've fixed a couple of bugs since then:

Other changes:

Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

October 19, 2015 04:46 PM

October 16, 2015

Geoff Brown

More enhancements to mach test commands for Android

As I wrote in my last post, using mach to test Firefox for Android in an emulator simplifies the testing process and removes the need to connect a physical phone or tablet. Similarly, mach now looks out for and offers to “fix” some other common Android-specific complications.

The first complication is Firefox itself. “Browser tests” like mochitests and reftests run inside Firefox. On Android, that means that Firefox for Android must be installed on your device. When using a phone or tablet, you can connect it by usb, and use “mach install” to install Firefox. But you might forget — I know I forget all the time and then wonder, why didn’t my tests run?! Also, if you are running an emulator automatically from a mach test command, you may not have a chance to install Firefox. So now mach test commands that require Firefox for Android check to see if it is installed; if it isn’t, mach prompts you to install Firefox from your local build.

Another complication is the “host utilities” required for most test types on Android. Many tests make requests from Firefox (running on the Android device) back to a web server running on the local computer – the test “host”. The test harnesses automatically start that web server for you, but they need to run executables like xpcshell and ssltunnel to do so. These host utilities must run on your computer (the host driving the tests via mach and the test harnesses) rather than on Android. Your Android build probably has xpcshell and ssltunnel, but they are Android executables and will not run on the Linux or OSX that’s probably running on your host. You can set the MOZ_HOST_BIN environment variable to point to utilities suitable for your host (a desktop Firefox build will do), but if you neglect to set MOZ_HOST_BIN, mach will notice and prompt you to set up ready-made utilities that can be downloaded (for Linux or OSX only).

Putting it all together, if nothing is set up and all these components are needed, you might see something like this:

gbrown@mozpad:~/src$ ./mach robocop testLoad
No Android devices connected. Start an emulator? (Y/n) y
Fetching AVD. This may take a while...
Starting emulator running Android 4.3...
It looks like Firefox is not installed on this device.
Install Firefox? (Y/n) y
Installing Firefox. This may take a while...
Host utilities not found: environment variable MOZ_HOST_BIN is not set to a directory containing host xpcshell
Download and setup your host utilities? (Y/n) y
Installing host utilities. This may take a while...

…and then your tests will run!

Some people are concerned about all this prompting; they suggest just going ahead and doing the necessary steps rather than waiting for these Y/n questions to be answered. I see the appeal, but there are consequences. For example, you may have simply forgotten to connect your physical device and have no desire to download an AVD and run an emulator. Overall, I think it is best to prompt and it is easy to avoid most prompts if you wish:

mach android-emulator && mach install && mach <your test>

Happy testing!

October 16, 2015 08:39 PM

Running Firefox for Android tests in the emulator

Recent enhancements to mach test commands make it easier than ever to run tests of all sorts – mochitests, reftests, robocop, etc – against Firefox for Android.

One of the barriers to running tests on an Android device is the device itself. You can run tests on an Android phone or tablet as long as it meets certain requirements:

In my experience, most Android devices work just fine: Connect a phone or tablet by usb, check that adb can see it, check that the phone is connected to wifi, and all is well. But there are several places where things can go wrong. And of course, not everyone has an Android phone available for testing. Running tests in an Android emulator resolves several of these concerns.

Now when you run a mach test command from a Firefox for Android build environment and there is no adb-visible device connected, mach will offer to start an emulator for you:

gbrown@mozpad:~/src$ ./mach robocop testLoad
No Android devices connected. Start an emulator? (Y/n) y
Fetching AVD. This may take a while...
Starting emulator running Android 4.3...

mach will search for the emulator first in the build config location, then in the Android SDK (as found by the ANDROID_SDK_ROOT environment variable), and finally in the “mach bootstrap” cache (under .mozbuild).

Once the emulator is found, mach will then download an Android image (an “AVD”, an Android Virtual Device, including kernel and system images) from Mozilla tooltool servers. The image is selected from those we use for tests on treeherder:

The AVD download can take a few minutes — those are big files. Don’t worry, they are cached (in your .mozbuild directory) so that subsequent runs are much faster. Once the AVD is downloaded, the emulator is launched and as soon as Android has booted, mach will start the requested tests.

You can also start the emulator directly with “mach android-emulator” and even select an image version; for example:

mach android-emulator --version 2.3

See “mach help android-emulator” for more details.

Once an emulator is started, all mach test commands will recognize that device and use it automatically.

Happy testing!

October 16, 2015 07:57 PM

October 15, 2015

Geoff Brown

More Android mach test commands

Many of your favourite desktop mach test commands now work with Firefox for Android.

These commands now explicitly support Firefox for Android, running tests against a connected Android device or emulator when run from a Firefox for Android build environment:

 mach robocop
 mach mochitest
 mach reftest
 mach crashtest
 mach jstestbrowser
 mach xpcshell-test
 mach cppunittest

Note that most commands allow running the entire suite (rarely a good idea!), a directory or manifest, or a single test.

As usual, detailed information on testing Firefox for Android can be found at

October 15, 2015 10:33 PM

Firefox for Android Performance Measures – Q3 Check-up

I am a little late with the Q3 check-up – sorry!

This review of Android performance measurements covers the period July 1 – September 30: the third quarter of 2015.


– tcheck2 noisy – bug 1213032

– most test results are fairly steady over the quarter


This section tracks Perfomatic graphs from for mozilla-central builds of Firefox for Android, for Talos tests run on Android 4.0 Opt. The test names shown are those used on treeherder. See for background on Talos.

We intend to retire the remaining Android Talos tests, migrating these tests to autophone in the near future.


Measure of “checkerboarding” during simulation of real user interaction with page. Lower values are better.


20 (start of period) – 4 (end of period)

There is significant noise in recent results – bug 1213032 – but overall there is much less checkerboarding.


An svg-only number that measures SVG rendering performance. About half of the tests are animations or iterations of rendering. This ASAP test (tsvgx) iterates in unlimited frame-rate mode thus reflecting the maximum rendering throughput of each test. The reported value is the page load time, or, for animations/iterations – overall duration the sequence/animation took to complete. Lower values are better.


680 (start of period) – 730 (end of period)

Minor regression but fairly consistent results over the period.


Generic page load test. Lower values are better.


670 (start of period) – 670 (end of period)

Very steady results over the period.

Throbber Start / Throbber Stop

These graphs are taken from  Browser startup performance is measured on real phones (a variety of popular devices).


Slight improvement in time to throbber start.


Slight regression in time to throbber stop, especially for nexus-s-4.


Android tests are no longer run on Eideticker.


These graphs are taken from the mozbench dashboard at which includes some comparisons involving Firefox for Android. More info at





October 15, 2015 10:22 PM

October 13, 2015

Armen Zambrano G. (@armenzg)

mozci 0.15.1: Buildbot Bridge support + bug fixes

It's been a while since our last announced released and I would like to highlight some of the significant changes since then.
A major highlight for the latest release is the support to schedule Buildbot jobs through TaskCluster making use of the Buildbot Bridge. For more information about this please read in here.


@nikkisquared is our latest contributor who landed more formalized exceptions and error handling. Thanks!
As always, many thanks to @adusca and @vaibhavmagarwal for their endless contributions.

How to update

Run "pip install -U mozci" to update

New Features

  • Query Treeherder for hidden jobs
  • BuildbotBridge support
    • It allows submitting a graph of Buildbot jobs with dependencies
    • The graph is submitted to TaskCluster

Minor changes

  • Fix backfilling issue
  • Password prompt improvements
  • More specific exceptions
  • Skip Windows 10 jobs for talos

All changes

You can see all changes in here:

Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

October 13, 2015 07:55 PM

Mozilla CI tools contribution opportunities

A lot of feature development has happened around Mozilla CI tools this year and it has been great to have so many contributors to help us move forward! We would not be here without all the many contributions we've had.

For now, MozCI needs to wait a bit before we can do full progress around TaskCluster support (blocked on big-graph scheduling work )

Some of the work where we could get some help until then is the following:
There are more issues, however, I believe these are easier to contribute to.

Feel free to ping "armenzg" on IRC at the #ateam channel (Read more on how to contact us in here)

Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

October 13, 2015 07:41 PM

Andreas Tolfsen

Making Mercurial log make sense

If you’re into Mercurial bookmarks, hailed as the Mercurial equivalent to git branches, my excellent colleague Andrew Halberstadt has written an extension that returns commits associated with a particular bookmark called bookbinder.

In his own words, “[a] book binder is a person or machine [that] binds pages to books. In a similar vein, bookbinder binds commits to bookmarks”. It does this by implementing a feature revset allowing bookbinder to subvert all commands that have a REV argument and replace any detected bookmark names with the feature revset.

This normally works by passing the name of your bookmark to any command that supports bookmarks, e.g. hg log -r BOOKMARK, and it will then replace BOOKMARK with a revset containing all changesets “within” this bookmark.

Because the stock hg log doesn’t make much sense to me, especially without passing --graph to graphically display the repository’s DAG, I highly recommend having a look at it to give hg log back some sanity.

Since I never get around to clean up my bookmarks, it would be quite convenient to instruct bookbinder to show me the commits associated with the bookmark of my latest commit.

For this I’ve added an alias to my .hgrc that tells it to pick the bookmark from my last commit, or tip:

blog = log --rev 'feature(tip)'

If I have three commits on my bookmark, this will return something akin to what you would expect from git log:

% hg blog
changeset:   267448:66fae38273f2
bookmark:    bug_1202663_super
tag:         tip
user:        Andreas Tolfsen
date:        Tue Oct 13 16:52:26 2015 +0100
summary:     Bug 1213797: Refactor screen capture and SVG document support

changeset:   267447:3cc8741649af
user:        Andreas Tolfsen
date:        Fri Oct 09 12:02:42 2015 +0100
summary:     Bug 1202663: Use dispatcher for screen capture command in listener

changeset:   267446:880e2bdb24ff
parent:      267426:607a236c2299
user:        Andreas Tolfsen
date:        Tue Oct 13 16:58:40 2015 +0100
summary:     Bug 1213800: More exhaustive Marionette screen capture tests

October 13, 2015 07:14 PM

Byron Jones

happy bmo push day!

the following changes have been pushed to : user fields are cleared when navigating back to a page (eg. after an error) : help docs not available for User Preferences : gravatar container for comment 0 can be too wide : Button edges are hard to see at most zoom-levels, with bmo “experimental user interface” : Buttons besides ‘User Story’ overflow if the user story is a one-liner : linkification isn’t applied to the user-story field : Make login UX mobile friendly to assist mobile authentication workflow : BMO login being reset for every browser session : make the TOTP MFA code field type=number on mobile : Release Tracking Report doesn’t return bugs with tracking flags set to — when searching for “not fixed” values : Update crash signatures to match new Socorro signature generation : cannot remove other people from the cc list : Remember expanded/collapsed sections : Always show “never email me about this bug” checkbox : needinfo requests where the current user is the requestee shouldn’t be editable : add support for duo-security

Filed under: bmo, mozilla

October 13, 2015 06:03 AM

October 08, 2015

Andreas Tolfsen

The case against visibility checks in WebDriver

The WebDriver specification came to life as a blueprint description of Selenium’s behaviour to turn what is the de facto browser automation solution, into a specification that would become a de jure standard. Along the way we have rectified and corrected quirks and oddities in the existing work to make commands cohesive units that form part of a larger, more consistent protocol.

Now that almost all of the formal remote end steps are defined, we are looking closer at the relationship among different commands. A part of this is questioning the command primitives, and a current burning issue is the approximation to visibility.

For looking up and interacting with elements, a series of precondition steps must be met, one of which is that the element is deemed visible. The visibility of an element is said to be guided by what is “perceptually visible to the human eye”. This is a tough mark to hit since other web standards we rely on are refusing to go anywhere near this topic. Defining what it means to be visible to the user, it turns out, is extremely difficult to give an exhaustive definition of.

Tree traversal and relationships

From Selenium the specification has inherited a long and complex algorithm to give a crude approximation about the element’s nature and its relationships in the tree. This was further developed to take into account more things that we knew Selenium was missing.

The specification gives a highly non-normative summary of what it means by element visibility:

An element is in general to be considered visible if any part of it is drawn on the canvas within the [boundaries] of the viewport.

Through the special meaning of many HTML features and how the ancestral relationship between different elements have effect on elements’ visibility, it goes on to describe the steps of an algorithm that traverses the tree up and down, starting from the element it is trying to determine the visibility of.

Practically it looks at certain computed (resolved) styling properties that are generally known to make an element invisible, such as display: none, width and height &c. The visibility of certain elements, such as an <option> element, depend upon the characteristics of its parent: For these it traverses up the document until it finds the containing <select> element, then applies the same checks there.

Because many HTML elements need special treatment, each has separate rules defined for them. Some of these rules include <map>, <area>, textual nodes, <img>, but also many more.

Following explicit hiding rules, if the element has more than a single direct descendant element its own visibility will depend upon the visibility of its children. The same is true if any of its direct ancestral elements in tree order fail the visibility test, in which case the visibility will cascade or trickle down to all child elements.

What we arrive at is a recursive algorithm that, although extremely ineffective, tells if a node is visible within the constraints of the tree it is part of. But looking solely at the part of the tree an element is in, overlapping elements from other trees are not considered.

The tree-traversal approach also entirely avoids addressing issues around absolute positioning, custom elements, overflow, and device concepts such as initial- and actual viewport. Owing to some of the provisions it makes around how border widths influence a parent element’s effectual dimensions, or preconceived ideas on how input widgets are styled by user agents, it furthermore ascribes meaning, or interpretation, to areas where the web platform is bad at exposing the primitives.


A suggested alternative to tree traversal is a form of hit-testing that involves testing which element is under the cursor, and then do this for each coordinate of the element’s bounding box that is inside the viewport. This has the upside of avoiding the problems associated with tree traversal altogether, but the downside of being extremely inefficient, and can in the worst scenario be as bad as O(n).

It is also complicated by the fact is that the element inside the bounding box does not necessarily fill the entire rectangle. This is true if the element is clipped by border-radius, has a degree of rotation, or similar. The primitives offered to us give little guidance for tracking the exact path of an element’s shape.

SVG, canvas, Shadow DOM, and transforms

The tree traversal algorithm is also limited to HTML. Other document types such as SVG and canvas have entirely different visibility primitives, and the idea of implicit ancestor visibility simply makes no sense in these contexts.

Shadow DOM is similarly problematic because it introduces the concept of multiple DOMs. Elements that have a Shadow DOM attached do not expose the same standard DOM traversal and query APIs (such as Node, ParentNode, Element) as regular elements.

They also have the idea of scoped stylesheets, whereby it’s possible to style implementation details with a <style> element that just applies to the local scope. Matching rules are also constrained to the same DOM, meaning style selectors in the host document do not match inside a Shadow DOM.

CSS Transforms in itself isn’t an issue, but since the tree traversal algorithm is not a single atomic action, it does not take into account that the state of the document may change whilst it is executing. It should be possible to work around this issue by evaluating the entire algorithm on a sandboxed snapshot of the tree, but again, this exposes us to non-standardised realms of the web platform.


By this point it should almost go without saying that providing a consistent, future-proof method of ascertaining visibility is futile. Whilst WebDriver does have access to the browser internals required to solve this problem, the solution likely lies outside the scope of specifying a remote control protocol.

As we cannot guarantee that new platform features will be taken into account, the tree-traversal approach offered by Selenium is not a building block for the future. It’s a hacky approach that may work reasonably well given the naïve narritive of the simple and undynamic web document world of 10 years ago.

To provide a future-proof notion of naked-eye visibility, there’s a need to push for separate, foundational platform APIs that other standards, in turn, must relate to. WebDriver’s primary role is exposing a privileged remote control interface that enables introspection and control of user agents; not to define element- or textual visibility. When such primitives are made available WebDriver should be an obvious consumer of these.

The good news is that the current Selenium behaviour, given the set of restrictions pointed out, is possible to replicate using existing, public APIs. This will allow consumers to inject automation atoms (chunks of browser-independent JavaScript implementations) to reproduce the behaviour using WebDriver primitives.

October 08, 2015 01:39 PM

October 07, 2015

David Burns

A new Marionette version available for Selenium Users with Java, .NET and Ruby support

If you have been wanting to use Marionette but couldn't because you don't work in Python now is your chance to do so! Well, if you are a Java User, .NET and Ruby you can use it too!! All the latest downloads of the Marionette executable are available from our development github repository releases page. We will be moving this to the Mozilla organization the closer we get to a full release.

There is also a new page on MDN that walks you through the process of setting up Marionette and using it. There are examples for all the language bindings currently supported.

Since you are awesome early adopters it would be great if we could raise bugs.

I am not expecting everything to work but below is a quick list that I know doesn't work.

Switching of Frames needs to be done with either a WebElement or an index. Windows can only be switched by window handles. This is currently how it has been discussed in the specification.

If in doubt, raise bugs!

Thanks for being an early adopter and thanks for raising bugs as you find them!

October 07, 2015 04:15 PM

October 06, 2015

Byron Jones

happy bmo push day!

the following changes have been pushed to

discuss these changes on

Filed under: bmo, mozilla

October 06, 2015 07:28 AM

October 02, 2015

Joel Maher

Hacking on a defined length contribution program

Contribution takes many forms where each person has different reasons to contribute or help people contribute.  One problem we saw a need to fix was when a new contributor came to Mozilla and picked up a “good first bug”, then upon completion was left not knowing what to do next and picking up other random bugs.  The essential problem is that we had no clear path defined for someone to start making more substantial improvements to our projects.   This can easily lead toward no clear mentorship as well as a lot of wasted time setting up and learning new things.  In response to this, we decided to launch the Summer of Contribution program.

Back in May we announced two projects to pilot this new program: perfherder and developer experience.  In the announcement we asked that interested hackers commit to dedicating 5-10 hours/week for 8 weeks to one of these projects. In return, we would act as a dedicated mentor and do our best to ensure success.

I want to outline how the program was structured, what worked well, and what we want to do differently next time.

Program Structure

The program worked well enough, with some improvising, here is what we started with:

That was it, we improvised a little by doing:

What worked well

A lot worked very well, specifically advertising by blog post and newsgroup post and then setting the expectation of a longer contribution cycle rather than a couple weeks.  Both :wlach and myself have had a good history of onboarding contributors, and feel that being patient, responding quickly, communicating effectively and regularly, and treating contributors as team members goes a long way.  Onboarding is easier if you spend the time to create docs for setup (we have the ateam bootcamp).  Without mentors being ready to onboard, there is no chance for making a program like this work.

Setting aside a pile of bugs to work on was successful.  The first contribution is hard as there is so much time required for setup, so many tools and terms to get familiar with, and a lot of process to learn.  After the first bug is completed, what comes next?  Assuming it was enjoyable, one of two paths usually take place:

Both of these are OK models, but there is a trap where you could end up with a bug that is hard to fix, not well defined, outdated/irrelevant, or requires a lot of new learning/setup.  This trap is something to avoid where we can build on the experience of the first bug and work on the same feature but on a bug that is a bit more challenging.

A few more thoughts on the predefined set of bugs to get started:

  • These should not be easily discoverable as “good first bugs“, because we want people who are committed to this program to work on them, rather than people just looking for an easy way to get involved.
  • They should all have a tracking bug, tag, or other method for easily seeing the entire pool of bugs
  • All bugs should be important to have fixed, but they are not urgent- think about “we would like to fix this later this quarter or next quarter”.  If we do not have some form of urgency around getting the bugs fixed, our eagerness to help out in mentoring and reviewing will be low.  A lot of  times while working on a feature there are followup bugs, those are good candidates!
  • There should be an equal amount (5-10) of starter bugs, next bugs, and other bugs
  • Keep in mind this is a starter list, imagine 2-3 contributors hacking on this for a month, they will be able to complete them all.
  • This list can grow as the project continues

Another thing that worked is we tried to work in public channels (irc, bugzilla) as much as possible, instead of always private messaging or communicating by email. Also communicating to other team members and users of the tools that there are new team members for the next few months. This really helped the contributors see the value of the work they are doing while introducing them to a larger software team.

Blog posts were successful at communicating and helping keep things public while giving more exposure to the newer members on the team.  One thing I like to do is ensure a contributor has a Mozillians profile as well as links to other discoverable things (bugzilla id, irc nick, github id, twitter, etc.) and some information about why they are participating.  In addition to this, we also highlighted achievements in the fortnightly Engineering Productivity meeting and any other newsgroup postings we were doing.

Lastly I would like to point out a dedicated mentor was successful.  As a contributor it is not always comfortable to ask questions, or deal with reviews from a lot of new people.  Having someone to chat with every day you are hacking on the project is nice.  Being a mentor doesn’t mean reviewing every line of code, but it does mean checking in on contributors regularly, ensuring bugs are not stuck waiting for needinfo/reviews, and helping set expectations of how work is to be done.  In an ideal world after working on a project like this a contributor would continue on and try to work with a new mentor to grow their skills in working with others as well as different code bases.

What we can do differently next time?

A few small things are worth improving on for our next cycle, here is a few things we will plan on doing differently:

  • Advertising 4-5 weeks prior and having a defined start/end date (e.g. November 20th – January 15th)
  • Really limiting this to a specific number of contributors, ideally 2-3 per mentor.
  • Setting acceptance criteria up front.  This could be solving 2 easy bugs prior to the start date.
  • Posting an announcement welcoming the new team members, posting another announcement at the halfway mark, and posting a completion announcement highlighting the great work.
  • Setting up a weekly meeting schedule that includes status per person, great achievements, problems, and some kind of learning (guest speaker, Q&A, etc.).  This meeting should be unique per project.
  • Have a simple process for helping folks transition out of they have less time than they thought- this will happen, we need to account for it so the remaining contributors get the most out of the program.

In summary we found this to be a great experience and are looking to do another program in the near future.  We named this Summer of Contribution for our first time around, but that is limiting to when it can take place and doesn’t respect the fact that the southern hemisphere is experiencing Winter during that time.  With that :maja_zf suggested calling it Quarter of Contribution which we plan to announce our next iteration in the coming weeks!

October 02, 2015 06:18 PM

October 01, 2015

Jonathan Griffin

Engineering Productivity Update, Oct 1, 2015

We’ve said good-bye to Q3, and are moving on to Q4. Planning for Q4 goals and deliverables is well underway; I’ll post a link to the final versions next update.

Last week, a group of 8-10 people from Engineering Productivity gathered in Toronto to discuss approaches to several aspects of developer workflow. You can look at the notes we took; next up is articulating a formal Vision and Roadmap for 2016, which incorporates both this work as well as other planning which is ongoing separately for things like MozReview and Treeherder.


Bugzilla: Support for 2FA has been enhanced.


Perfherder and Performance Automation:

MozReview and Autoland: mcote posted a blog post detailing some of the rough edges in MozReview, and explaining how the team intends on tackling these. dminor blogged about the state of autoland; in short, we’re getting close to rolling out an initial implementation which will work similarly to the current “checkin-needed” mechanism, except, of course, it will be entirely automated. May you never have to worry about closed trees again!

Mobile Automation: gbrown made some additional improvements to mach commands on Android; bc has been busy with a lot of Autophone fixes and enhancements.

Firefox Automation: maja_zf has enabled MSE playback tests on trunk, running per-commit. They will go live at the next buildbot reconfig.

Developer Workflow: numerous enhancements have been made to |mach try|; see list below in the Details section.  run-by-dir has been applied to mochitest-plain on most platforms, and to mochitest-chrome-opt, by kaustabh93, one of team’s contributors. This reduces test bleedthrough, a source of intermittent failures, as well as improves our ability to change job chunking without breaking tests.

Build System: gps has improved test package generation, which results in significantly faster builds – a savings of about 5 minutes per build on OSX and Windows in automation; about 90s on linux.

TaskCluster Migration: linux64 debug builds are now running, so ahal is unblocked on getting linux64 debug tests running in TaskCluster.  armenzg has landed mozharness code to support running buildbot jobs via TaskCluster scheduling, via buildbot bridge.

The Details


Perfherder/Performance Testing

TaskCluster Support

Mobile Automation

Firefox and Media Automation

General Automation


WebDriver (highlights)




Firefox build system

October 01, 2015 10:50 PM

Dan Minor

Autoland to Inbound

We’re currently putting the finishing touches on Autoland to Inbound from MozReview. We have a relatively humble goal for the initial version of this: we’d like to replace the current manual “checkin-needed” process done by a sheriff. Basically, if a patch is reviewed and a try run has been done, the patch author can flag “checkin-needed” in Bugzilla and at some point in the future, a sheriff will land the changes.

In the near future, if a “ship-it” has been granted and a try run has been made, a button to land to Inbound will appear in the Automation menu in MozReview. At the moment, the main benefit of using Autoland is that it will retry landings in the event that the tree is closed. Enforcing a try run should cut down on tree closures, analysis of closure data in the past has shown a not insignificant number of patches which end up breaking the tree have had no try run performed.

A number of workflow improvements should be easy to implement once this is in place, for instance automatically closing MozReview requests to keep people’s incoming review requests sane and rewriting any r? in the commit summary to be r=.

October 01, 2015 07:15 AM

September 30, 2015

Mark Côté

Fixing MozReview's sore spots

MozReview was intentionally released early, with a fairly minimal feature set, and some ugly things bolted onto a packaged code-review tool. The code-review process at Mozilla hasn’t changed much since the project began—Splinter, a slightly fancier UI than dealing with raw diffs, notwithstanding. We knew this would be a controversial subject, with a variety of (invariably strong) opinions. But we also knew that we couldn’t make meaningful progress on a number of long-desired features, like autolanding commits and automatic code analysis, without moving to a modern repository-based review system. We also knew that, with the incredible popularity of GitHub, many developers expect a workflow that involves them pushing up commits for review in a rich web UI, not exporting discrete patches and looking at almost raw diffs.

Rather than spending a couple years off in isolation developing a polished system that might not hit our goals of both appealing to future Mozillians and increasing productivity overall, we released a very early product—the basic set of features required by a push-based, repository-centric code-review system. Ironically, perhaps, this has decreased the productivity of some people, since the new system is somewhat raw and most definitely a big change from the old. It’s our sincere belief that the pain currently experienced by some people, while definitely regrettable and in some cases unexpected, will be balanced, in the long run, by the opportunities to regularly correct our course and reach the goal of a world-class code review-and-submission system that much faster.

And so, as expected, we’ve received quite a bit of feedback. I’ve been noticing a pattern, which is great, because it gives us insight into classes of problems and needs. I’ve identified four categories, which interestingly correspond to levels of usage, from basic to advanced.

Understanding MozReview’s (and Review Board’s) models

Some users find MozReview very opaque. They aren’t sure what many of the buttons and widgets do, and, in general, are confused by the interface. This caught us a little off-guard but, in retrospect, is understandable. Review Board is a big change from Splinter and much more complex. I believe one of the sources of most confusion is the overall review model, with its various states, views, entry points, and exit points. Splinter has the concept of a review in progress, but it is a lot simpler.

We also had to add the concept of a series of related commits to Review Board, which on its own has essentially a patch-based model, similar to Splinter’s, that’s too limited to build on. The relationship between a parent review request and the individual “child” commits is the source of a lot of bewilderment.

Improving the overall user experience of performing a review is a top priority for the next quarter. I’ll explore the combination of the complexity of Review Board and the commit-series model we added in a follow-up post.

Inconveniences and lack of clarity around some features

For users who are generally satisfied by MozReview, at least, enough to use it without getting too frustrated, there are a number of paper cuts and limitations that can be worked around but generate some annoyance. This is an area we knew we were going to have to improve. We don’t yet have parity with Splinter/Bugzilla attachments, e.g. reviewers can’t delegate review requests, nor can they mark specific files as reviewed. There are other areas that we can go beyond Bugzilla, such as being able to land parts of a commit series (this is technically possible in Bugzilla by having separate patches, but it’s difficult to track). And there are specific things that Review Board has that aren’t as useful for us as they could be, like the dashboard.

This will also be a big part of the work in the next quarter (at least).

Inability to use MozReview at all due to technological limitations

The single biggest item here is lack of support for git, particularly a git interface for hg repos like mozilla-central. There are many people interested in using MozReview, but their work flows are based around git using git-cinnabar. gps and kanru did some initial work around this in bug 1153053; fleshing this out into a proper solution isn’t a small task, but it seems clear that we’ll have to finish it regardless before too long, if we want MozReview to be the central code-review tool at Mozilla. We’re still trying to decide how this fits into the above priorities; more users is good, but making existing users happier is as well.

Big-ticket items

As mentioned at the beginning of this post, the main reason we’re building a new review tool is to make it repository-centric, that is, based around commits, not isolated patches. This makes a lot of long-desired tools and features much more feasible, including autoland, automatic static analysis, commit rewriting to automatically include metadata like reviewers, and a bunch of other things.

This has been a big focus for the last few months. We’ve had autoland-to-try for a little while now, and autoland-to-inbound is nearly complete. We have a generic library for static analysis with which we’ll be able to build various review bots. And, of course, the one big feature we started with, the ability push commits to MozReview instead of exporting standalone patches, which by itself is both more convenient and preserves more information.

After autoland-to-inbound we’ll be putting aside other big features for a little while to concentrate on general user experience so that people enjoy using MozReview, but rest assured we’ll be back here to build more powerful workflows for everyone.

September 30, 2015 06:15 PM

Joel Maher

Say hi to Gabriel Machado- a newer contributor on the Perfherder project

Earlier this summer, I got an email from Gabriel asking how he could get involved in Automation and Tools projects at Mozilla.  This was really cool and I was excited to see Gabriel join the Mozilla Community.  Gabriel is knows as :goma on IRC and based on his interests and projects with mentors  available, hacking on TreeHerder was a good fit.  Gabriel also worked on some test cases for the Firefox-UI-Tests.  You can see the list of bugs he has been involved with and check him out on Github.

While it is great to see a contributor become more comfortable in their programming skills, it is even better to get to know the people you are working with.  As I have done before, I would like to take a moment to introduce Gabriel and let him do the talking:

Tell us about where you live –

I lived in Durham-UK since last year, where I was doing an exchange program. About Durham I can say that is a lovely place, it has a nice weather, kind people, beautiful castles and a stunning “medieval ambient”.  Besides it is a student city, with several parties and cultural activities.

I moved back to Brazil 3 days ago, and next week I’ll move to the city of Ouro Preto to finish my undergrad course. Ouro Preto is another beautiful historical city, very similar to Durham in some sense. It is a small town with a good university and stunning landmarks. It’s a really great place, designated a world heritage site by UNESCO.

Tell us about your school –

In 2 weeks I’ll begin my third year in Computer Science at UFOP(Federal University of Ouro Preto). It is a really good place to study computer science, with several different research groups. In my second year I earned a scholarship from the Brazilian Government to study in the UK. So, I studied my second year at Durham University. Durham is a really great university, very traditional and it has a great infra-structure. Besides, they filmed several Harry Potter scenes there :P

Tell us about getting involved with Mozilla –

In 2014 I was looking for some open source project to contribute with when I found the Mozilla Contributing Guide. It is a really nice guide and helped me a lot. I worked on some minors bugs during the year. In July of 2015, as part of my scholarship to study in the UK, I was supposed to do a small final project and I decided to work with some open source project, instead of an academic research. I contacted jmaher by email and asked him about it. He answered me really kindly and guided me to contribute with the Treeherder. Since then, I’ve been working with the A-Team folks, working with Treeherder and Firefox-Ui-Tests.

I think Mozilla does a really nice job helping new contributors, even the new ones without experience like me. I used to think that I should be a great hacker, with tons of programming knowledge to contribute with an open source project. Now, I think that contributing with an open source project is a nice way to become a great hacker with tons of programming knowledge

Tell us what you enjoy doing –

I really enjoy computers. Usually I spent my spare time testing new operating systems, window managers or improving my Vim. Apart from that, I love music. Specially instrumental.  I play guitar, bass, harmonica and drums and I really love composing songs. You can listen some of my instrumental musics here:

Besides, I love travelling and meeting people from different cultures. I really like talking about small particularities of different languages.

Where do you see yourself in 5 years?

I hope be a software engineer, working with great and interesting problems and contributing for a better (and free) Internet.

If somebody asked you for advice about life, what would you say?

Peace and happiness comes from within, do not seek it without.

Please say hi to :goma on irc in #treeherder or #ateam.

September 30, 2015 09:52 AM

Byron Jones

happy bmo push day!

the following changes have been pushed to

discuss these changes on

Filed under: bmo, mozilla

September 30, 2015 07:42 AM

September 29, 2015

William Lachance

Perfherder summer of contribution thoughts

A few months ago, Joel Maher announced the Perfherder summer of contribution. We wrapped things up there a few weeks ago, so I guess it’s about time I wrote up a bit about how things went.

As a reminder, the idea of summer of contribution was to give a set of contributors the opportunity to make a substantial contribution to a project we were working on (in this case, the Perfherder performance sheriffing system). We would ask that they sign up to do 5-10 hours of work a week for at least 8 weeks. In return, Joel and myself would make ourselves available as mentors to answer questions about the project whenever they ran into trouble.

To get things rolling, I split off a bunch of work that we felt would be reasonable to do by a contributor into bugs of varying difficulty levels (assigning them the bugzilla whiteboard tag ateam-summer-of-contribution). When someone first expressed interest in working on the project, I’d assign them a relatively easy front end one, just to cover the basics of working with the project (checking out code, making a change, submitting a PR to github). If they made it through that, I’d go on to assign them slightly harder or more complex tasks which dealt with other parts of the codebase, the nature of which depended on what they wanted to learn more about. Perfherder essentially has two components: a data storage and analysis backend written in Python and Django, and a web-based frontend written in JS and Angular. There was (still is) lots to do on both, which gave contributors lots of choice.

This system worked pretty well for attracting people. I think we got at least 5 people interested and contributing useful patches within the first couple of weeks. In general I think onboarding went well. Having good documentation for Perfherder / Treeherder on the wiki certainly helped. We had lots of the usual problems getting people familiar with git and submitting proper pull requests: we use a somewhat clumsy combination of bugzilla and github to manage treeherder issues (we “attach” PRs to bugs as plaintext), which can be a bit offputting to newcomers. But once they got past these issues, things went relatively smoothly.

A few weeks in, I set up a fortnightly skype call for people to join and update status and ask questions. This proved to be quite useful: it let me and Joel articulate the higher-level vision for the project to people (which can be difficult to summarize in text) but more importantly it was also a great opportunity for people to ask questions and raise concerns about the project in a free-form, high-bandwidth environment. In general I’m not a big fan of meetings (especially status report meetings) but I think these were pretty useful. Being able to hear someone else’s voice definitely goes a long way to establishing trust that you just can’t get in the same way over email and irc.

I think our biggest challenge was retention. Due to (understandable) time commitments and constraints only one person (Mike Ling) was really able to stick with it until the end. Still, I’m pretty happy with that success rate: if you stop and think about it, even a 10-hour a week time investment is a fair bit to ask. Some of the people who didn’t quite make it were quite awesome, I hope they come back some day. :)

On that note, a special thanks to Mike Ling for sticking with us this long (he’s still around and doing useful things long after the program ended). He’s done some really fantastic work inside Perfherder and the project is much better for it. I think my two favorite features that he wrote up are the improved test chooser which I talked about a few months ago and a get related platform / branch feature which is a big time saver when trying to determine when a performance regression was first introduced.

I took the time to do a short email interview with him last week. Here’s what he had to say:

1. Tell us a little bit about yourself. Where do you live? What is it you do when not contributing to Perfherder?

I’m a postgraduate student of NanChang HangKong university in China whose major is Internet of things. Actually,there are a lot of things I would like to do when I am AFK, play basketball, video game, reading books and listening music, just name it ; )

2. How did you find out about the ateam summer of contribution program?

well, I remember when I still a new comer of treeherder, I totally don’t know how to start my contribution. So, I just go to treeherder irc and ask for advice. As I recall, emorley and jfrench talk with me and give me a lot of hits. Then Will (wlach) send me an Email about ateam summer of contribution and perfherder. He told me it’s a good opportunity to learn more about treeherder and how to work like a team! I almost jump out of bed (I receive that email just before get asleep) and reply with YES. Thank you Will!

3. What did you find most challenging in the summer of contribution?

I think the most challenging thing is I not only need to know how to code but also need to know how treeherder actually work. It’s a awesome project and there are a ton of things I haven’t heard before (i.e T-test, regression). So I still have a long way to go before I familiar with it.

4. What advice would give you to future ateam contributors?

The only thing you need to do is bring your question to irc and ask. Do not hesitate to ask for help if you need it! All the people in here are nice and willing to help. Enjoy it!

September 29, 2015 03:37 PM

September 23, 2015

Dan Minor

Beyond the Code 2015

Mozilla was kind enough to send me to the Beyond the Code 2015 in Ottawa today. When I went to PyCon back in April, the sessions I felt I got the most out of were the non-technical ones so this seemed like a fun and useful way to spend the day.

My favourite talk of the day was given by Kat Bautista on Drawing as a Way of Thinking which was a funny and insightful talk on using drawings as a means of visual note-taking both for yourself and in front of a whiteboard as a way of facilitating meetings. Not “draw me like one of your french girls” drawing but rather a visual, cartoony shorthand for capturing ideas.

Having suffered through overzealous project management from time to time (e.g. two project managers for five developers) I could relate to her description of Gantt Charts as a way of to avoid having meaningful discussions (“Look at the spreadsheets I can make!”) but I think sketches can also be used to obscure issues, people love pictures and can easily overlook a lack of substance behind them.

draw autoland

I had read a number of books and papers on sketching for design as part of a Software Usability course I took a few years ago so I had already been exposed to some of the ideas before but I still got a lot out of the talk. That usability course was for the most part disappointing, the focus was on doing studies to evaluate existing user interfaces with basically nothing at all on how one would go about creating such a thing in the first place.

As I mentioned, Mozilla also sent me to PyCon 2015 back in April. I was remiss in capturing my thoughts back then. After nearly six months what has stayed most with me was Pete Fein’s talk Free Software, Free People on his experiences with Telecomix, in particular, one slide, where he urged people to be active and work for causes they care about (whatever they may be.)


September 23, 2015 02:37 PM

Byron Jones

happy bmo push day!

the following changes have been pushed to

discuss these changes on

Filed under: bmo, mozilla

September 23, 2015 05:24 AM

September 22, 2015

Joel Maher

DX Contribution Program – Wrap up

This summer went by fast.  Back in May we announced the opportunity to participate in the DX project.  In June we announced the team.  August ended and the project officially came to a halt.  First off, I would really like to thank all the participants who took interest in this program.  With 5 eager hackers, there is a lot to do!  As the summer wore on, some folks got side tracked, others worked when they could and some kept moving forward.  No matter what the outcome, I do applaud the 5 great contributors who spent their free time getting setup and started on a project.

Now that we are at the end, I would like to outline some of the work that was done and things that were learned.

A few great things got started like mach usage statistics, and Marionette harness cleanup.  These are not easy to get started on and figure out all the quirks with our full scale ecosystem.  It is wonderful to see ideas and progress being made.   Thanks BYK and sehgalvibhor.

Two other projects didn’t get much traction.  While I like to blame myself as a mentor (yes, I could have done a bit more or scoped the projects differently), we ended up not making progress on a few projects.  This is OK, and there is always stuff to learn.  On the projects that didn’t get going, I saw gma_fav struggle a bit, but persist.  I know the right project is out there and she will succeed next time, ideally at Mozilla!  I enjoyed learning about wunderlist from stanley, and will look forward to more of his enthusiasm.

The last project is run-by-dir.  kaustabh93 really worked consistently on this and learned a lot about working in the mozilla-central tree, using try, and understanding our infrastructure.  A lot of work was done for mochitest-plain and mochitest-chrome.  He is still hacking away and making fast progress.  As Kaustabh was the only contributor who was consistent throughout the entire program, I wanted to hear his perspective on the program:

How did you hear about the program, why were you interested?
I had been a contributor at Mozilla for quite sometime working mainly on AlertManager and familiarizing myself with Mochitests by means of solving bugs and reading code. It was at this time, that my curriculum needed me to do an industrial project (internships/training/other programs for which the company would acknowledge mt work by means of certificates or letters or both). I brought this up with my mentor and a few days later, he informed me about this program.
It had been an awesome experience coding for Mozilla so far & hence I got interested instantly.
What did you find challenging?  How did you overcome these challenges?
The existing volume of codes were huge and getting to know what is what was challenging. I overcame this by solving smaller bugs first and reading codes and of course a lot of help, advice & explanation from my mentor, jmaher. He is an awesome person and one of the best mentors, I’ve known.

What advice would you give for anybody looking to contribute in this program?

People looking to contribute to this program should get involved in coding for Mozilla some time prior to commencement of this program. That way they can be familiarized with the existing codes and can better understand their work and this way, they can make the maximum impact with their work. And they should continue contributing to Mozilla even after this program closes.
Other thoughts : One more thing that I should advice people regarding this program is that if they contribute to Mozilla just for this programs (contribution start and end dates coincides with that of the program), then they’re probably doing it wrong.

While I appreciate the callout as a great mentor, there are many other great Mozillian mentors, it is my pleasure to work with folks who contribute their free time to the project,

Without making this too long, I did learn a lot from this great group of participants.  A few tips for becoming a better mentor (you get better with experience and learn from mistakes) is to communicate regularly with whom you are mentoring, this should be at least twice a week.  While mentoring involves a lot of encouragement and gentle guidance, it is important to listen more to what roadblocks are in place (schedules, technical, etc.) and sometimes we need to move to a smaller or simpler project until the timing is right for the big project.  One other big callout was that gma_fav was helpful in making me really understand how our use of master/slave is offensive to many, I am glad I listened to her and worked with her for a couple months!

I look forward to watching these contributors grow outside of this program and meeting more great folks!

September 22, 2015 01:39 AM