Henrik SkupinMemchaser 0.6 has been released

The Firefox Automation team would like to announce the release of memchaser 0.6. After nearly a year of no real feature updates, but also some weeks of not being able to run Memchaser in Firefox Aurora (34.0a2) at all (due to a regression), we decided to release the current state of development as a public release. We are aware that we still do not fully support the default Australis theme since Firefox 29.0, but that’s an issue, which takes some more time to finish up.

Changes in 0.6

  • Upgrade to Add-on SDK 1.17 (#201)
  • Fix test_start_stop_logging for ‘File not found’ error (#199)
  • contentURL expects a string in Panel() and Widget() constructors (#198)
  • Bring the minimizeMemory() implementation up to date (#193)
  • Use ‘residentFast’ instead of ‘resident’ for memory reporter (#192)
  • Use postMessage for widget and panel communication instead port object (#185)
  • Support incremental cycle collection statistics (#187)
  • Added a usage section to the README (#178)
  • Change require statements and update the SDK to 1.14 (#174)
  • Simplify .travis.yml (#172)
  • Use . (dot) to include files while invoking /bin/sh (#169)
  • Use failonerror attribute in exec tasks (#168)

For all the details about the 0.6 release please check our issue tracker on Github.

Matt BrubeckLet's build a browser engine! Part 6: Block layout

Welcome back to my series on building a toy HTML rendering engine:

This article will continue the layout module that we started previously. This time, we’ll add the ability to lay out block boxes. These are boxes that are stack vertically, such as headings and paragraphs.

To keep things simple, this code implements only normal flow: no floats, no absolute positioning, and no fixed positioning.

Traversing the Layout Tree

The entry point to this code is the layout function, which takes a takes a LayoutBox and calculates its dimensions. We’ll break this function into three cases, and implement only one of them for now:

impl LayoutBox {
    /// Lay out a box and its descendants.
    fn layout(&mut self, containing_block: Dimensions) {
        match self.box_type {
            BlockNode(_) => self.layout_block(containing_block),
            InlineNode(_) => {} // TODO
            AnonymousBlock => {} // TODO
        }
    }

    // ...
}

A block’s layout depends on the dimensions of its containing block. For block boxes in normal flow, this is just the box’s parent. For the root element, it’s the size of the browser window (or “viewport”).

You may remember from the previous article that a block’s width depends on its parent, while its height depends on its children. This means that our code needs to traverse the tree top-down while calculating widths, so it can lay out the children after their parent’s width is known, and traverse bottom-up to calculate heights, so that a parent’s height is calculated after its children’s.

fn layout_block(&mut self, containing_block: Dimensions) {
    // Child width can depend on parent width, so we need to calculate
    // this box's width before laying out its children.
    self.calculate_block_width(containing_block);

    // Determine where the box is located within its container.
    self.calculate_block_position(containing_block);

    // Recursively lay out the children of this box.
    self.layout_block_children();

    // Parent height can depend on child height, so `calculate_height`
    // must be called *after* the children are laid out.
    self.calculate_block_height();
}

This function performs a single traversal of the layout tree, doing width calculations on the way down and height calculations on the way back up. A real layout engine might perform several tree traversals, some top-down and some bottom-up.

Calculating the Width

The width calculation is the first step in the block layout function, and also the most complicated. I’ll walk through it step by step.

To start, we need to know the values of the CSS width property and all the left and right edge size properties:

fn calculate_block_width(&mut self, containing_block: Dimensions) {
    let style = self.get_style_node();

    // `width` has initial value `auto`.
    let auto = Keyword("auto".to_string());
    let mut width = style.value("width").unwrap_or(auto.clone());

    // margin, border, and padding have initial value 0.
    let zero = Length(0.0, Px);

    let mut margin_left = style.lookup("margin-left", "margin", &zero);
    let mut margin_right = style.lookup("margin-right", "margin", &zero);

    let border_left = style.lookup("border-left-width", "border-width", &zero);
    let border_right = style.lookup("border-right-width", "border-width", &zero);

    let padding_left = style.lookup("padding-left", "padding", &zero);
    let padding_right = style.lookup("padding-right", "padding", &zero);

    // ...
}

This uses a helper function called lookup, which just tries a series of values in sequence. If the first property isn’t set, it tries the second one. If that’s not set either, it returns the given default value. This provides an incomplete (but simple) implementation of shorthand properties and initial values.

Note: This is similar to the following code in, say, JavaScript or Ruby:

margin_left = style["margin-left"] || style["margin"] || zero;

Since a child can’t change its parent’s width, it needs to make sure its own width fits the parent’s. The CSS spec expresses this as a set of constraints and an algorithm for solving them. The following code implements that algorithm.

First we add up the margin, padding, border, and content widths. The to_px helper method converts lengths to their numerical values. If a property is set to 'auto', it returns 0 so it doesn’t affect the sum.

let total = [&margin_left, &margin_right, &border_left, &border_right,
             &padding_left, &padding_right, &width].iter().map(|v| v.to_px()).sum();

This is the minimum horizontal space needed for the box. If this isn’t equal to the container width, we’ll need to adjust something to make it equal.

If the width or margins are set to 'auto', they can expand or contract to fit the available space. Following the spec, we first check if the box is too big. If so, we set any expandable margins to zero.

// If width is not auto and the total is wider than the container, treat auto margins as 0.
if width != auto && total > containing_block.width {
    if margin_left == auto {
        margin_left = Length(0.0, Px);
    }
    if margin_right == auto {
        margin_right = Length(0.0, Px);
    }
}

If the box is too large for its container, it overflows the container. If it’s too small, it will underflow, leaving extra space. We’ll calculate the underflow—the amount of extra space left in the container. (If this number is negative, it is actually an overflow.)

let underflow = containing_block.width - total;

We now follow the spec’s algorithm for eliminating any overflow or underflow by adjusting the expandable dimensions. If there are no 'auto' dimensions, we adjust the right margin. (Yes, this means the margin may be negative in the case of an overflow!)

match (width == auto, margin_left == auto, margin_right == auto) {
    // If the values are overconstrained, calculate margin_right.
    (false, false, false) => {
        margin_right = Length(margin_right.to_px() + underflow, Px);
    }

    // If exactly one size is auto, its used value follows from the equality.
    (false, false, true) => { margin_right = Length(underflow, Px); }
    (false, true, false) => { margin_left  = Length(underflow, Px); }

    // If width is set to auto, any other auto values become 0.
    (true, _, _) => {
        if margin_left == auto { margin_left = Length(0.0, Px); }
        if margin_right == auto { margin_right = Length(0.0, Px); }

        if underflow >= 0.0 {
            // Expand width to fill the underflow.
            width = Length(underflow, Px);
        } else {
            // Width can't be negative. Adjust the right margin instead.
            width = Length(0.0, Px);
            margin_right = Length(margin_right.to_px() + underflow, Px);
        }
    }

    // If margin-left and margin-right are both auto, their used values are equal.
    (false, true, true) => {
        margin_left = Length(underflow / 2.0, Px);
        margin_right = Length(underflow / 2.0, Px);
    }
}

At this point, the constraints are met and any 'auto' values have been converted to lengths. The results are the the used values for the horizontal box dimensions, which we will store in the layout tree. You can see the final code in layout.rs.

Positioning

The next step is simpler. This function looks up the remanining margin/padding/border styles, and uses these along with the containing block dimensions to determine this block’s position on the page.

fn calculate_block_position(&mut self, containing_block: Dimensions) {
    let style = self.get_style_node();
    let d = &mut self.dimensions;

    // margin, border, and padding have initial value 0.
    let zero = Length(0.0, Px);

    // If margin-top or margin-bottom is `auto`, the used value is zero.
    d.margin.top = style.lookup("margin-top", "margin", &zero).to_px();
    d.margin.bottom = style.lookup("margin-bottom", "margin", &zero).to_px();

    d.border.top = style.lookup("border-top-width", "border-width", &zero).to_px();
    d.border.bottom = style.lookup("border-bottom-width", "border-width", &zero).to_px();

    d.padding.top = style.lookup("padding-top", "padding", &zero).to_px();
    d.padding.bottom = style.lookup("padding-bottom", "padding", &zero).to_px();

    // Position the box below all the previous boxes in the container.
    d.x = containing_block.x +
          d.margin.left + d.border.left + d.padding.left;
    d.y = containing_block.y + containing_block.height +
          d.margin.top + d.border.top + d.padding.top;
}

Take a close look at that last statement, which sets the y position. This is what gives block layout its distinctive vertical stacking behavior. For this to work, we’ll need to make sure the parent’s height is updated after laying out each child.

Children

Here’s the code that recursively lays out the box’s contents. As it loops through the child boxes, it keeps track of the total content height. This is used by the positioning code (above) to find the vertical position of the next child.

fn layout_block_children(&mut self) {
    let d = &mut self.dimensions;
    for child in self.children.mut_iter() {
        child.layout(*d);
        // Track the height so each child is laid out below the previous content.
        d.height = d.height + child.dimensions.margin_box_height();
    }
}

The total vertical space taken up by each child is the height of its margin box, which we calculate just by adding all up the vertical dimensions.

impl Dimensions {
    /// Total height of a box including its margins, border, and padding.
    fn margin_box_height(&self) -> f32 {
        self.height + self.padding.top + self.padding.bottom
                    + self.border.top + self.border.bottom
                    + self.margin.top + self.margin.bottom
    }
}

For simplicity, this does not implement margin collapsing. A real layout engine would allow the bottom margin of one box to overlap the top margin of the next box, rather than placing each margin box completely below the previous one.

The ‘height’ Property

By default, the box’s height is equal to the height of its contents. But if the 'height' property is set to an explicit length, we’ll use that instead:

fn calculate_block_height(&mut self) {
    // If the height is set to an explicit length, use that exact length.
    match self.get_style_node().value("height") {
        Some(Length(h, Px)) => { self.dimensions.height = h; }
        _ => {}
    }
}

And that concludes the block layout algorithm. You can now call layout() on a styled HTML document, and it will spit out a bunch of rectangles with widths, heights, margins, etc. Cool, right?

Exercises

Some extra ideas for the ambitious implementer:

  1. Collapsing vertical margins.

  2. Relative positioning.

  3. Parallelize the layout process, and measure the effect on performance.

If you try the parallelization project, you may want to separate the width calculation and the height calculation into two distinct passes. The top-down traversal for width is easy to parallelize just by spawning a separate task for each child. The height calculation is a little trickier, since you need to go back and adjust the y position of each child after its siblings are laid out.

To Be Continued…

Thank you to everyone who’s followed along this far!

These articles are taking longer and longer to write, as I journey further into unfamiliar areas of layout and rendering. There will be a longer hiatus before the next part as I experiment with font and graphics code, but I’ll resume the series as soon as I can.

David BoswellLearning more about the Mozilla community

We’ve learned a lot this year as we’ve been working on enabling communities that have impact. We discovered there is a high contributor churn rate, scaling the size of the community doesn’t meet the needs of all teams and there is a need to make community building more reliable and predictable.

There are still many more questions than answers right now though and there is much more to learn. A contributor audit was done in 2011 that had useful findings and recommendations, but it has now been 3 years since that research was completed. It is time to do more.

400px-Audit_presentation

Research has provided other community-driven organizations with lessons that help them be more successful. For example, Lego has an active community and research has helped them develop a set of principles that promote successful interactions that provide value for both community members and the Lego organization.

We’ll be kicking off a new research project soon and we’d love to get your help. This will involve creating a survey to send out to Mozillians and will also dive into the contributor data we’ve started collecting. This won’t answer all the questions we have, but this will give us some insight and can provide a starting point for other research projects.

areweamillion_coding

Some specific asks for helping with the survey include thinking about what questions we want to ask and thinking about the audience of people we want to reach out to. For the data analysis part, please comment here or contact me if you’re interested in helping.


Henrik Skupinmozdownload 1.12 has been released

The Firefox Automation team would like to announce the release of mozdownload 1.12. Without any other release of our universal download tool in the last 7 months, a couple of nice new features and bug fixes will make this release even more useful. You can upgrade your installation easily via pip, or by downloading it from PyPI.

Changes in 1.12

  • Display selected build when downloading (#149)
  • Add support for downloading B2G desktop builds (#104)
  • Download candidate builds from candidates/ and not nightly/ (#218)
  • Add Travis CI build status and PyPI version badges to README (#220)
  • Add Python 2.6 to test matrix (#210)
  • Fix broken download of mac64 tinderbox builds (#144)
  • Allow download even if content-length header is missing (#194)
  • Convert run_tests script to Python (#168)
  • Ensure that –date option is a valid date (#196)

Henrik SkupinMozmill 2.0.7 and 2.0.8 have been released

The Firefox Automation team would like to announce the release of Mozmill 2.0.7 and Mozmill 2.0.8. Both versions had to be released in such a short time frame to ensure continuing support for Firefox. Some latest changes done for Firefox Nightly broke Mozmill, or at least made it misbehaving. If you run tests with Mozmill ensure to upgrade to the latest version. You can do this via PyPI, or simply download the already pre-configured environment.

Changes in 2.0.7

  • Bug 1066295 – testMultipleLoads.js is failing due to new HTTP Cache v2
  • Bug 1000098 – Fix testPageLoad.js test for invalid cert page
  • Bug 1065436 – Disable e10s until full support landed
  • Bug 1062773 – Disconnect errors invalidate the report
  • Bug 999393 – Expose assert and expect by default in sub modules
  • Bug 970820 – Mozmill aborts with socket.timeout when trying to send the report

Changes in 2.0.8

  • Bug 1067939 – JSBridge and Mozmill broken due to ‘let’ changes in bug 1001090

Please keep in mind that Mozmill 2.0 does not support electrolysis (e10s) builds of Firefox yet. We are working hard to get full support for e10s added, and hope it will be done until the next version bump mid of October.

Thanks everyone who was helping with those releases!

Sean MartellMozilla ID Project: Palette Explorations

Churning along with out explorations into the Mozilla brand, we’ve been looking at nailing down an expanded base color palette to bring more choice and a bolder feel. We’ve injected the colors used in the Firefox color palette – derived from the logo itself – and added a few more previously missed hues, such as greens and pinks.

colors-and-shapes

Along with the straight expansion of a single set, we’ve picked a vibrant set to compliment the base. With the two sets we then chose two stepped-down variants for each. This makes 6 variants for each of the named colors within the set.

All this is still in the works and not fully locked down, but we thought we’d share the thinking around this and offer a glimpse at where we’re going. We’re also updating the names of the base colors and adding a few fun labels in the process. Again, the labels aren’t finalized yet either, but thought we’d share. More to come on color and usage scenarios later!

Byron Joneswe’re changing the default search settings for advanced search

at the end of this week we will change the default search settings on bugzilla.mozilla.org’s advanced search page:

resolution-duplicate

the resolution DUPLICATE will no longer be selected by default – only open bugs will be searched if the resolution field is left unmodified.

this change will not impact any existing saved searches or queries.

 

DUPLICATE has been a part of our default query for as long as i can remember, and was included to accommodate using that form to search for existing bugs.

since the addition of “possible duplicates” to the bug creation workflow, the importance of searching duplicates has lessened, and returning duplicates by default to advanced users is more of a hindrance than a help.  the data reflects this – the logs indicate that over august less than 4% of advanced queries included DUPLICATE as a resolution.

 

this change is being tracked in bug 1068648.


Filed under: bmo

Kim MoirMozilla Releng: The ice cream

A week or so ago, I was commenting in IRC that I was really impressed that our interns had such amazing communication and presentation skills.  One of the interns, John Zeller said something like "The cream rises to the top", to which I replied "Releng: the ice cream of CS".  From there, the conversation went on to discuss what would be the best ice cream flavour to make that would capture the spirit of Mozilla releng.  The consensus at the end was was that Irish Coffee (coffee with whisky) with cookie dough chunks was the favourite.  Because a lot of people like on the team like coffee, whisky makes it better and who doesn't like cookie dough?

I made this recipe over the weekend with some modifications.  I used the coffee recipe from the Perfect Scoop.  After it was done churning in the ice cream maker,  instead of whisky, which I didn't have on hand, I added Kahlua for more coffee flavour.  I don't really like cookie dough in ice cream but cooked chocolate chip cookies cut up with a liberal sprinkling of Kahlua are tasty.

Diced cookies sprinkled with Kahlua

Ice cream ready to put in freezer

Finished product
I have to say, it's quite delicious :-) If I open source ever stops being fun, I'm going to start a dairy empire.  Not really. Now back to bugzilla...

Mozilla Release Management TeamFirefox 33 beta3 to beta4

  • 43 changesets
  • 114 files changed
  • 6806 insertions
  • 3156 deletions

ExtensionOccurrences
cpp49
h28
js12
cc9
build4
xul2
java2
xml1
sh1
dat1

ModuleOccurrences
security50
gfx12
media11
js9
ipc7
mobile3
dom3
content3
browser3
image2
caps2
accessible2
netwerk1
modules1

List of changesets:

Robert O'CallahanBug 1063052. NS_RUNTIMEABORT if a builtin stylesheet fails to load. r=heycam,a=lmandel - c43d3d833973
EKRBug 1063730 - Require HTTPS for Screen/window sharing. r=mt,sstamm a=lmandel - a706b85f6d4d
Jim MathiesBug 1066242 - Use a 'ui' chromium message loop/pump for the Windows compositor thread so that it can process native windowing events. r=Bas a=sylvestre - e3fe616ef9a2
Alexander SurkovBug 1020039 - Fix intermittent failures in relations/test_embeds.xul. a=test-only - 5007a59d2d92
Nick AlexanderBug 1041770 - Update missed reference. r=mrbkap, a=lmandel - 40044a225ae7
Chia-hung TaiBug 1057174 - [WebRTC] |DesktopDeviceInfoImpl::initializ| in desktop_device_info.cc use wrong argument while calling snprintf. r=rjesup, a=sledru - 645d232705b3
Tim AbraldesBug 1027906 - Set delayed token level for GMP plugin processes to USER_RESTRICTED. Whitelist certain files and registry keys that are required for EME plugins to successfully load. r=bobowen. r=jesup, r=bent, a=lmandel - 0af2575571f3
Jim MathiesBug 1066242 - Use a 'ui' chromium message loop/pump for the Windows compositor thread so that it can process native windowing events. r=Bas a=sylvestre - a128f3f1ce1f
Jim MathiesBug 1060738 - Add support for webrtc ThreadWindowsUI for use by webrtc desktop capture thread. r=jesup a=sylvestre - 2c6a2069023a
Jim MathiesBug 1060738 - Implement MessagePumpForNonMainUIThreads for Windows, a xpcom compatible subclass of chromium's MessagePumpForUI. r=tabraldes a=sylvestre - b6a5a3973477
Jim MathiesBug 1060738 - Switch to using chromium's Thread/tasks in MediaManager. On Windows, use MessagePumpForNonMainUIThreads for the background media thread. r=jesup a=sylvestre - 1355bb2a2765
Jim MathiesBug 1060738 - Add IsGUIThread asserts in various webrtc capture related methods. r=jesup a=sylvestre - 84daded3719c
Gervase MarkhamBug 1065977 - Uplift recent PSL changes to the release branches. a=lmandel - 5b7a15b4fee2
Gian-Carlo PascuttoBug 1063547 - Return no available devices where not supported, disable on Android. r=jesup, a=lmandel - abbbaa040046
Michael WuBug 1063733 - Optimize DataSourceSurface allocation. r=bas, r=seth, a=sledru - 5982da7a1215
Dão GottwaldBug 1061947 - Avoid flushing layout and making it dirty repeatedly in ToolbarIconColor.inferFromText. r=gijs, a=lmandel - 2938d6cea847
Lucas RochaBug 1041448 - Fix crash when double-tapping on empty top site spot. r=bnicholson, a=sledru - e0c49c71cc55
Margaret LeibovicBug 1063518 - Hide MLS "Learn More" link when MLS is disabled. r=liuche, a=lmandel - 275330447f6d
Mark FinkleBug 887755 - Lightweight theme preview is broken. r=margaret, a=lmandel - c21b3ccb9c19
Nicolas SilvaBug 1041744 - Don't crash if tile allocation fails. r=Bas, a=sledru - 9148cd599e9f
Nicolas SilvaBug 1061696 - Don't crash release builds when failing to allocate a surface in AutoRestoreClippedOut::save. r=Bas, a=sledru - ff9cef7b2f9d
Richard NewmanBug 1065531 - Crash in java.lang.NoSuchMethodError: android.os.Bundle.getString at org.mozilla.gecko.preferences.GeckoPreferences.setupPreferences. r=nalexander, a=lmandel - 430b3512f177
David MajorBug 1058131 - Avoid getting a crashy hook from Avast 10 Beta. r=bzbarsky, a=sledru - cd04e5bf0fec
Bobby HolleyBug 1061136 - Assume both http:// and https:// for schemeless URIs in CAPS prefs. r=bz, a=sledru - e608db37bafb
Bobby HolleyBug 1053725 - When one domain is whitelisted for file:// URI access, whitelist all subdomains. r=bz, a=sledru - a91c79c7e64e
Bobby HolleyBug 1008481 - Switch to the root dir instead of the profile dir. a=test-only - f58da8f6f47e
Michael ComellaBug 1062338 - Remove useless ic_menu_back drawable xml. r=lucasr, a=sledru - 1c636d0e8ec1
Brian SmithBug 1039064: Use strongly-typed enum instead of NSPR-style error handling, r=keeler a=lmandel - f3115a9f645c
David KeelerBug 1040446 - mozilla::pkix: add error code for CA cert used as end-entity cert r=briansmith a=lmandel - 15c382469fd1
David KeelerBug 1034124 - allow overrides when a CA cert is used as an end-entity cert r=briansmith a=lmandel - 198d06258284
Ryan VanderMeulenBug 1037618 - Skip ice_unittest on OSX. a=test-only - 0225b61c4f71
Jan de MooijBug 1057598 - Suppress the object metadata callback in RStringSplit::recover. r=nbp, a=sledru - 62f5d35f2210
Ryan VanderMeulenBug 1057598 - s/warmup/usecount on older release branches. rs=nbp, a=bustage - 3e6571e74e01
Dan GohmanBug 1054972 - IonMonkey: Truncation for phis. r=nbp, a=sledru - 94dc71a06159
Dan GohmanBug 1054972 - IonMonkey: GVN: More misc cleanups. r=nbp, a=sledru - c0d46e44a6cb
Dan GohmanBug 1054972 - IonMonkey: GVN: Avoid setting UseRemoved flags unnecessarily. r=nbp, a=sledru - 316374007734
Dan GohmanBug 1062612 - IonMonkey: Fix cast insertion for truncation of phi operands. r=nbp, a=lmandel - c5ee54bc44f8
Ryan VanderMeulenBacked out 3 changesets (Bug 1039064, Bug 1040446, Bug 1034124) for ASAN xpcshell hangs. - b8c9b76b6585
Chris CooperBug 1066403 - replace empty blocklist - a=blocklist-update - 06300676d4cd
Gabriel LuongBug 1061003 - Add New Rule won't work in non-english locales. r=harth, a=lmandel - bacdfedd7241
Brian SmithBug 1039064 - Use strongly-typed enum instead of NSPR-style error handling. r=keeler, a=lmandel - 1f599d357743
David KeelerBug 1040446 - mozilla::pkix: add error code for CA cert used as end-entity cert. r=briansmith, a=lmandel - 93cd4a068e9d
David KeelerBug 1034124 - Allow overrides when a CA cert is used as an end-entity cert. r=briansmith, a=lmandel - a6856f90ce36

Daniel StenbergSnaxx delivers

A pint of guinnessLate in the year 1999 I quit my job. I handed over a signed paper where I wrote that I quit and then I started my new job first thing in the year 2000. I had a bunch of friends at the work I left and together with my closest friends (who coincidentally also switched jobs at roughly the same time) we decided we needed a way to keep in touch with friends that isn’t associated with our current employer.

The fix, the “employer independent” social thing to help us keep in touch with friends and colleagues in the industry, started on the last of February 2000. The 29th of February, since it was a leap year and that fact alone is a subject that itself must’ve been discussed at that meetup.

Snaxx was born.

Snaxx is getting a bunch of friends to a pub somewhere in Stockholm. Preferably a pub with lots of great beers and a sensible sound situation. That means as little music as possible and certainly no TVs or anything. We keep doing them at a pace of two or three per year or so.

Bishops Arms logo

Yesterday we had the 31st Snaxx and just under 30 guests showed up (that might actually have been the new all time high). We had many great beers, food and we argued over bug reporting, discussed source code formats, electric car charging, C64 nostalgia, mentioned Linux kernel debugging methods, how to transition from Erlang to javascript development and a whole load of other similarly very important topics. The Bishops Arms just happens to be a brand of pubs here that have a really sensible view on how to run pubs to be suitable for our events so yesterday we once again visited one of their places.

Thanks for a great time yesterday, friends! I’ll be setting up a date for number 32 soon. I figure it’ll be in the January 2015 time frame…If you want to get notified with an email, sign up yourself on the snaxx mailing list.

A few pictures from yesterday can be found on the Snaxx-31 G+ event page.

Raniere SilvaMathml September Meeting

Mathml September Meeting

../../../_images/mathml2.jpg

This is a report about the Mozilla MathML September Meeting. The topics of the meeting can be found in this PAD (local copy of the PAD). This meeting happens all at appear.in and because of that we don’t have a log.

The next meeting will be in October 10th at 8pm UTC (note that October 10th is Friday, we change the day in our try make the meeting suitable to as many people possible). Please add topics in the PAD.

Leia mais...

Ludovic HirlimannGnupg / PGP key signing party in mozilla's San francisco space

I’m organizing a pgp Keysigning party in the Mozilla san francisco office on September the 26th 2014 from 6PM to 8PM.

For security and assurances reasons I need to count how many people will attend. I’ve setup a eventbrite for that at https://www.eventbrite.com/e/gnupg-pgp-key-signing-party-making-the-web-of-trust-stronger-tickets-12867542165 (please take one ticket if you think about attending - If you change you mind cancel so more people can come).

I will use the eventbrite tool to send reminders and I will try to make a list with keys and fingerprint before the event to make things more manageable (but I don’t promise).

For those using lanyrd you will be able to use http://lanyrd.com/ccckzw.(Please tweet the event to get more people in).

Clint TalbertGetting Ready for Fall

As leaves begin to turn in some places, the Bay area gets a nice blast of heat in September. It’s like a week of summer that forgot to happen back when we were socked in with fog in July. But, that nice last bit of warmth before our leaves start to turn (yes we have some leaves that turn) is also the harbinger of the fourth quarter at Mozilla. This year, I’m excited to start putting into place some of the changes we identified at the work week. So, to that end, three quick notes.

First, we are starting off with a Quality and Automation Community Call. This is going to be in the style of the WebMaker Community Calls, and the WebMaker folks have been graciously teaching us how they work. Our first one will be next Tuesday at 8AM Pacific/15:00 UTC. Unlike most of our other meetings where we talk in glib Mozilla shorthand, this will be a call that will focus on fully explaining a few specific projects that people can help with Right Now, regardless of their skill level. And, this will also be a forum for folks in the community to tell us about what they are doing. Giving our community a way to easily let us know what they are doing, and giving them a forum to talk to one another is not something we do enough of at Mozilla. So, I’m extremely excited to see where this goes. Additionally, we are going to do this in conjunction with the Automation and Tools team and I’m extremely psyched about starting this process off. Many thanks to Lyre Calliope who helped nudge us in this direction.

Second, we are going to define our efforts in quality around some core principles, which I can talk more about later, but I’ll briefly introduce them here. They are probably best envisioned as a set of overlapping circles in a Venn diagram:

Dependability, Delight, Security & Privacy, Performance underpinned by the web platform

There are likely only two things in there that you weren’t expecting. One is Delight–in this ultra competitive world, it’s not enough for our products to be right and correct. We have to go the extra mile and delight our users with our products. Nothing short of that is good enough.

The other one is the web platform. At Mozilla, our mission to extend, enhance and empower the web platform while ensuring it remains open underpins everything we do. As an odd anachronism, our QA systems and metrics have not historically taken into account how a given feature does or does not help to move the web platform forward. Likewise, we’ve not been very involved in looking at how we are implementing and testing our support for various web standards in any kind of systemic way. While I don’t believe that there are problems in these areas, I do believe that this is a core piece of quality at Mozilla and it is an area we should work to get more involved in and be more cognizant of.

Third, time is unfortunately a zero sum game.  The amount of work doesn’t shrink, especially when attempting to venture into new areas. So in order to make room for our new directions, we are going to experiment with stopping and/or pausing some of our endeavors. That can be scary when you’re changing how things have been done for years, but it’s what we need to do to move Mozilla forward. The world we live in has changed, and there is no going back. There is constant iteration toward our vision of being a more technically astute, more data-driven, more community empowering team that propels quality forward.

Armen ZambranoWhich builders get added to buildbot?

To add/remove jobs on tbpl.mozilla.org, we have to modify buildbot-configs.

Making changes can be learnt by looking at previous patches, however, there's a bit of an art to it to get it right.

I just landed a script that sets up buildbot for you inside of a virtualenv and you can pass a buildbot-config patch and determine which builders get added/removed.

You can run this by checking out braindump and running something like this:
buildbot-related/list_builder_differences.sh -j path_to_patch.diff

NOTE: This script does not check that the job has all the right parameters once live (e.g. you forgot to specify the mozharness config for it).

Happy hacking!


Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Tantek Çelik IndieWebCampUK 2014 Hack Day Demos: HTTPS, #webactions, new & improved #indieweb sites

One weekend ago, 18 IndieWebCampUK participants (including 2 remote) showed 25 demos in just under 75 minutes of what they designed and built that weekend in 19 different interoperable projects. Every single demo exemplified an indieweb community member scratching their own personal site itch(es), helping each other do so, and together advancing the state of the indieweb. We can all say:

I'm building Indie Web Camp.

During the demos I took realtime notes in IRC, with some help from Barnaby Walters. Archived on the IndieWebCamp wiki, here's a summary of what each of us got working.

Glenn Jones

Glenn Jones built improvements to Transmat. (IRC notes)

He built a map view that shows the venues nearest to his current location (via GeoLocation API).

He also found an open source HTML5 JS open source pedometer and repurposed it into Transmat so that when running on his Android as a web app, it can detect when he's walking, and only do GPS lookups when he's walking, so it saves battery.

Now he has an HTML5 JS app that can auto-checkin for him while he's walking.

Barnaby and Pelle

Barnaby WaltersPelle Wessman Barnaby Walters and Pelle Wessman built cross-site reply webactions that work purely via their websites - no browser extension needed! This is the first time this has been done. (IRC notes)

Barnaby has setup registerProtocolHandler on Taproot to register a handler for the "web+indie:" (since updated to "web+action:") protocol when he loads a particular page on his website so that his website is registered to handle webactions via the <indie-action> tag.

Barnaby demonstrates loading the page that calls registerProtocolHandler. The browser asks to confirm that he wants waterpigs.co.uk to handle "web+indie" URLs.

Then Barnaby goes to Pelle's website home page where he has a list of posts that he's written, now with "Reply", "Like", and "Tip" webactions next to each post, each webaction represented and wrapped by <indie-action> tags in the markup.

Pelle's site also has a web component ([https://github.com/voxpelli/indie-action-component open sourced on github]) to handle his <indie-action> tags, which creates an iframe that uses that same protocol handler using a Promise, which connects the iframe to calling the handler that Taproot registered.

Thus without anything installed in the browser, Barnaby can go to Pelle's site, click the "Reply" button next to a post which automatically goes to Barnaby's site's Taproot UI to post a reply!

Barnaby Walters

Barnaby Walters also built a map-view post aggregator that shows icons for people at the locations embedded in their recent posts. (IRC notes)

The map-view aggregator is at a self-standing demo URL for now, but Barnaby plans to include this view as another column type in Shrewdness, so you can have a map view of recent posts from people you're following.

Grant Richmond

Grant Richmond got a fancy new domain (grant.codes) and setup Glenn Jones's Transmat on it - which makes it the second installation of Transmat! (IRC notes)

Grant also built a contact page: grant.codes/contact that has links for various methods of communication:

All of the links are text links for now, no icons yet.

Grant has implemented a people focused communication UI on his site!

Jeremy Keith

Jeremy Keith added https on adactio.com, and implemented <indie-action> tag webactions. (IRC notes)

adactio https

Jeremy took his site adactio.com from no https support to https Level 4. All adactio.com URLs redirect to https. However subdomains (e.g. austin.adactio.com) are still http.

adactio webactions

Jeremy's also implemented the new <indie-action> tag for webactions around his existing Tweet action links, both on his post permalinks, and on his posts in-stream (e.g. on his home page or when paginated).

Shane Hudson

Shane Hudson went from no SSL and no comments yesterday to https level 5! He also imported the contents of all his old comments from his WordPress blog to his Craft install (the CMS he's dogfooding, contributing plugins to, selfdogfooding). (IRC notes)

He was able to get SSL setup on his site with an A rating, and forward secrecy, and is thus https level 5.

Shane also wrote a script to do the import of comments from WordPress to Craft. It's "a bit crude, dealing with XML to CSV a few times".

Nat Welch

Nat Welch (AKA icco on IRC) got his blog running (his own software) in Go (language) hosted on AppEngine with SSL, achieving https level 4! (IRC notes)

AppEngine does SSL for free if you're ok with SNI.

So now Nat has SSL Labs rating A- on writing.natwelch.com! And also automatic redirect works from http to https. Thus he has also achieved https Level 4!

Right now he's using AppEngine default auth, using his Google account. Eventually he wants to use indieauth to auth into his site.

Tim Retout

Tim Retout got pump.io running on his site and added support to it for POSSEing to Twitter. (IRC notes)

His goal is to add all the indieweb feature support too like webmentions, microformats etc. He has to run off to catch a train.

He is also too humble, as he helped numerous people in person at the camp get on SSL, https level 4 or 5 at that. A round of applause for Tim!

Tom Morris

Tom Morris added https to his site, made it responsive, and setup mf2py as a service. (IRC notes)

responsive tommorris.org

Tom showed his current site tommorris.org with different window sizes. His CSS is now "less sucky" and he has made his site more responsive on mobile / small display etc.

mf2py as a service

Tom also got the Python microformats2 parser (mf2py) running as a service that you can submit your URLs to and get back pretty-printed JSON.

tommorris https

Tom got his main site tommorris.org up to https Level 4 with an A- rating, but has not yet done so with *.tommorris.org (e.g. wiki.tommorris.org).

During the next demo, Tom got his SSL Labs rating from A- to A with some help from Aral. And during the demo after that took his rating up to A+ thanks to this blog post.

Kevin Beynon

Kevin Beynon got IndieAuth login to his own site working! (IRC notes)

Kevin started by showing us his site home page kevinbeynon.com using a tablet. We projected it by holding up to the Talky HD camera.

He pointed out that there is no admin link on the home page then went to his "secret" URL at /admin/ which has an IndieAuth login screen. He entered his own URL, and chose to RelMeAuth authenticate using Twitter which redirected to it and back and came back with the message "Log-in Successful".

Kevin went to his home page again, and showed that it now has visible links to "admin" and "log out". Next he plans to bring his post creating and editing interface into his home page front end, so that he can do inline editing and post notes from his home page.

Joschi Kuphal

Joschi Kuphal got his site's https support to SSL rating A+, fixed his webmention implementation, and implemented webactions on permalinks. (IRC notes)

jkphl https A+

Joschi noted that his site was running with SSL before but had some flaws. He worked on it and improved his site's rating from F to A+.

jkphl webmentions fixed

He also fixed some flaws with his webmention implementation thanks to feedback from Ryan Barrett online.

jkphl permalinks webactions

Third, Joschi implemented webactions on permalinks, in particular he added <indie-action> markup around his default Twitter, G+, Facebook "share" links. He then demonstrated his site working with Barnaby Walters's Web Action Hero Toolkit browser extension.

Chris Asteriou

Chris Asteriou is fairly new to the IndieWeb and started with going through IndieMark, adding h-entry and h-card markup, and a notes section to his site.(IRC notes)

digitalbliss microformats

Chris showed digitalbliss.uk.com, noted that he added h-entry on his page with entries. He clicked the "Play" link at top to show this. And then he marked up the info at bottom of his home page with h-card.

digitalbliss notes

Chris added a notes section and used the verification tools on indiewebify.me to check it and verify that he reached IndieMark Level 2.

Tantek

Tantek Çelik switched his permalink webactions from <action> tags to <indie-action> tags and researched the UX of webactions on posts in a stream (e.g. a home page).

tantek indie-action

Based on the webactions discussion session in the first day with Tantek, Jeremy, and Pelle, they concluded that the <indie-action> tag was more appropriate than the <action> tag.

Tantek initially publicly proposed the <action> tag for consideration in a session on Web Actions at Open Source Bridge 2012, and then later implemented them at last year's IndieWebcampUK 2013 which were then demonstrated working with Barnaby Walters's browser extension.

Changing from <action> to <indie-action> at a minimum better fits with the web component model. Jeremy Keith pointed out that an <indie-action> tag in particular would be a good example of a web component, worthy as a case-study for web components.

Tantek updated his permalink webactions to use <indie-action> tags and Barnaby updated his browser extension to support them as well.

in-stream webactions

Tantek analyzed the UI of various silos, in particular Instagram and Twitter.

Instagram has a very minimal simple webaction UI, with just "Like", "Comment", and "..." (more) buttons, the first two with both icon and text labels, which makes sense since their primary content is large (relative to the UI) images/video (visual media). Instagram's webactions are identical on photos viewed on their own screen, and when in a stream of media. Deliberately designed consistency.

Twitter on the other hand is horribly inconsistent between different views of tweets, and even different streams, sometimes their webactions are:

  • on the right with text labels
  • on the left with text labels
  • on the left without text labels

Their trend seems to be icon only, likely because the text label distracts from the tweet text content around it, especially in a stream of tweets that are primarily (nearly all) just text.

Tantek walked through comparisons of Twitter's different webactions button icon/text usage/placements with Aral, who came to the same conclusions from the data.

It may be ok to use both icon and text labels on note/post permalink pages, as there is more distinction between the (single) content area, and the footer of webactions.

However, the conclusions is that in-stream webactions should use just icons (clear ones at that) when among posts that are primarily, mostly, or perhaps even often just text.

Next Tantek is working on implementing icon-only webactions on his home page posts stream. He made some progress but realized it will require him to rework some storage code first.

Aral Balkan

Aral Balkan upgraded his site's https support to SSL rating A+ and https Level 5, and his how-to blog post about it! (IRC notes)

Aral already supported https on his site aralbalkan.com beforehand. On IndieWebCampUK hack day he added support for forward secrecy, which raised its SSL rating from A- to A+ and thus he achieved https Level 5!

Apparently it took him only 2 lines of code to implement that change on nginx, and noted that it's a bit harder on Apache.

After his demo, Aral also updated his blog post about SSL setup with nginx with what he learned and how to get to SSL rating A+.

Rosa Fox

Rosa Fox created a UI on her site for CRUD posting of projects. (IRC notes)

Rosa wanted to make her own CMS with support for posting images and tags. She demonstrated her local dev install of her new CMS with the following new features she built at Hack Day:

  • a UI for creating a new project
  • CRUD posting interface for projects
  • using Postgres to store data

Aaron Parecki

Aaron Parecki participated remotely, added support for posting bookmarks to his site, and added bookmarks posting via micropub to his Quill app! (IRC notes)

Aaron has been publishing bookmarks to another place for a long time in a WordPress install at aaron.pk/bookmarks and he wanted to integrate them into his main site aaronparecki.com.

Once Aaron got the bookmark post type implemented in his publishing software p3k and deployed to his site, he did a mass import from the aaron.pk/bookmarks WordPress XML export.

That was the last thing aaronpk was using WordPress for, so he's no longer using WordPress to publish any of his own content.

Now all of Aaron's bookmarks are at aaronparecki.com/bookmarks all marked up with microformats. Each bookmark is an h-entry, and embedded inside is an h-cite of the bookmark itself.

This also means you can comment, bookmark, and like his bookmarks themselves!

During later demos, Aaron also updated his Quill app with a bookmark posting interface, as well as a bookmarklet so you can quickly open the Quill UI to make a bookmark.

Kevin Marks

Kevin Marks built a feed coverter that takes legacy RSS/Atom feeds and produces modern readable and usable h-entry page, including such niceties as inline playable audio elements in converted podcasts. (IRC notes)

Kevin noticed that people are building h-feed readers, so he built a tool that takes legacy RSS Atom feeds and unmunges them and produces nice clean h-entry feeds.

The converter is at feed.unmung.com/. Unmung.com is a URL he bought ages ago, and set it up on Google AppEngine.

E.g. if you put in xkcd.com/rss.xml into it, it generates a nice readable HTML page with h-entry, which you can then subscribe to in an indie reader like Barnaby's Shrewdness.

Kevin demonstrated using unmung to convert a podcast feed feeds.wnyc.org/onthemedia into an h-feed with embedded playable HTML5 <audio> elements, providing an actual useful interface, much better than the original feed.

Kevin made the point that no one wants to parse RSS or Atom any more. Now by parsing the microformats JSON representation, you can get any existing RSS or Atom etc.

You can now subscribe to iTunes podcasts etc. in your indieweb reader!

Robin Taylor

Robin Taylor added support for https (including forward secrecy, getting an SSL "A" rating) to his site robintaylor.uk and automatic redirects from http to https, achieving https Level 5! (IRC notes)

UK Homebrew Website Clubs

As we were wrapping up, Tom Morris asked openly if anyone would be interested in coming to a Homebrew Website Club in London. Jeremy Keith similarly asked the group for interest in a Homebrew Website Club Brighton.

Both had quite a bit of interest, so we can expect to start seeing more Homebrew Website Club meetups in more locations!

See also

Join Us At The Next IndieWebCamp In Cambridge

IndieWebCamp Cambridge is next month on the East Coast.

Join us. Share ideas. Come work on your personal web site. Help grow and evolve the independent web. Be the change you want to see in the world wide web.

"The people I met at @indiewebcamp are the A-Team of the Internet. Give them some tape and an oxy-acetalyne torch and they'll fix the web."

Adam Lofting“Conclusions”

Mile long string of baloons (6034077499)

  • Removing the second sentence increases conversion rate (hypothesis = simplicity is good).
  • The button text ‘Go!’ increased the conversion rate.
  • Both variations on the headline increased conversion rate, but ‘Welcome to Webmaker’ performed the best.
  • We should remove the bullet points on this landing page.
  • The log-in option is useful on the page, even for a cold audience who we assume do not have accounts already.
  • Repeating the ask ‘Sign-up for Webmaker’ at the end of the copy, even when it duplicates the heading immediately above, is useful. Even at the expense of making the copy longer.
  • The button text ‘Create an account’ works better than ‘Sign up for Webmaker’ even when the headline and CTA in the copy are ‘Sign up for Webmaker’.
  • These two headlines are equivalent. In the absence of other data we should keep the version which includes the brand name, as it adds one further ‘brand impression’ to the user journey.
  • The existing blue background color is the best variant, given the rest of the page right now.

The Webmaker Testing Hub

If any of those “conclusions” sound interesting to you, you’ll probably want to read more about them on the Webmaker Testing Hub (it’s a fancy name for a list on a wiki).

This is where we’ll try and share the results of any test we run, and document the tests currently running.

And why that image for this blog post?

Because blog posts need and image, and this song came on as I was writing it. And I’m sure it’s a song about statistical significance, or counting, or something…

Lucas RochaIntroducing Probe

We’ve all heard of the best practices regarding layouts on Android: keep your view tree as simple as possible, avoid multi-pass layouts high up in the hierarchy, etc. But the truth is, it’s pretty hard to see what’s actually going on in your view tree in each UI traversal (measure → layout → draw).

We’re well served with developer options for tracking graphics performance—debug GPU overdraw, show hardware layers updates, profile GPU rendering, and others. However, there is a big gap in terms of development tools for tracking layout traversals and figuring out how your layouts actually behave. This is why I created Probe.

Probe is a small library that allows you to intercept view method calls during Android’s layout traversals e.g. onMeasure(), onLayout(), onDraw(), etc. Once a method call is intercepted, you can either do extra things on top of the view’s original implementation or completely override the method on-the-fly.

Using Probe is super simple. All you have to do is implement an Interceptor. Here’s an interceptor that completely overrides a view’s onDraw(). Calling super.onDraw() would call the view’s original implementation.

public class DrawGreen extends Interceptor {
    private final Paint mPaint;

    public DrawGreen() {
        mPaint = new Paint();
        mPaint.setColor(Color.GREEN);
    }

    @Override
    public void onDraw(View view, Canvas canvas) {
        canvas.drawPaint(mPaint);
    }
}

Then deploy your Interceptor by inflating your layout with a Probe:

Probe probe = new Probe(this, new DrawGreen(), new Filter.ViewId(R.id.view2));
View root = probe.inflate(R.layout.main_activity, null);

Just to give you an idea of the kind of things you can do with Probe, I’ve already implemented a couple of built-in interceptors. OvermeasureInterceptor tints views according to the number of times they got measured in a single traversal i.e. equivalent to overdraw but for measurement.

LayoutBoundsInterceptor is equivalent to Android’s “Show layout bounds” developer option. The main difference is that you can show bounds only for specific views.

Under the hood, Probe uses Google’s DexMaker to generate dynamic View proxies during layout inflation. The stock ProxyBuilder implementation was not good enough for Probe because I wanted to avoid using reflection entirely after the proxy classes were generated. So I created a specialized View proxy builder that generates proxy classes tailored for Probe’s use case.

This means Probe takes longer than your usual LayoutInflater to inflate layout resources. There’s no use of reflection after layout inflation though. Your views should perform the same. For now, Probe is meant to be a developer tool only and I don’t recommend using it in production.

The code is available on Github. As usual, contributions are very welcome.

ArkyFirefox OS: Designing Khmer Keyboards and Fonts

Back in Cambodia this week to participate in Barcamp Phnom Penh 2014. It is great to experience the energy and openness of Phnom Penh and the Cambodian youth's insatiable zeal to learn all things tech. Over the past few years, the barcamps helped us build the Mozilla community in Cambodia.

Cambodia is a fast growing economy in the region. One survey notes significant increase in smart phone ownership from last year. And also increase in Khmer supported smart phones and feature phone in the market. At Barcamp Phnom Penh I presented a Firefox OS talk about the on-going Khmer Internationalization (i18n) work and invited the audience to contribute to Firefox OS. Planning to organize hackathons to work on Khmer keyboards with the Mozilla community here.

After my talk, Vannak of Mozilla Cambodia community talked briefly about Mozilla community to the audience. And we did a presentation about Mozilla Web Maker tools. I hope we'll organize more web literacy events in future. Keep watching this space for more news from Cambodia, the kingdom of wonder.

Byron Joneshappy bmo push day!

the following changes have been pushed to bugzilla.mozilla.org:

  • [1064878] Use of uninitialized value in pattern match (m//) at /loader/0x7ffa9dedc498/Bugzilla/Extension/BugmailFilter/Filter.pm line 172
  • [1020558] Add Involved with Bugs and Never Visited Query to MyDashboard
  • [1062944] Product::Component autocomplete when filing new bug shows disabled components.
  • [1046213] datetime_from() generates wrong dates if year < 1901
  • [1053513] remove last-visited entries when a user removes involvement from a bug
  • [1021902] UI to view a user’s review history
  • [1064678] searching for tracking flag “is empty” is generating incorrect sql
  • [1064329] splinter displays patches that remove lines starting with hyphens incorrectly
  • [1065594] Enable ‘due date’ field in ‘Community Building’ product (all components)
  • [1052851] add the ability to search by “assignee last login date”
  • [1066777] The kick-off form isn’t creating dependent bugs
  • [1039940] serialisation of objects for webservice responses is extremely slow
  • [1058615] New Custom Bugzilla Form Needed For PR Team

discuss these changes on mozilla.tools.bmo.


Filed under: bmo, mozilla

William Lachancemozregression 0.24

I just released mozregression 0.24. This would be a good time to note some of the user-visible fixes / additions that have gone in recently:

  1. Thanks to Sam Garrett, you can now specify a different branch other than inbound to get finer grained regression ranges from. E.g. if you’re pretty sure a regression occurred on fx-team, you can do something like:

    mozregression --inbound-branch fx-team -g 2014-09-13 -b 2014-09-14

  2. Fixed a bug where we could get an incorrect regression range (bug 1059856). Unfortunately the root cause of the bug is still open (it’s a bit tricky to match mozilla-central commits to that of other branches) but I think this most recent fix should make things work in 99.9% of cases. Let me know if I’m wrong.
  3. Thanks to Julien Pagès, we now download the inbound build metadata in parallel, which speeds up inbound bisection quite significantly

If you know a bit of python, contributing to mozregression is a great way to have a high impact on Mozilla. Many platform developers use this project in their day-to-day work, but there’s still lots of room for improvement.

Mozilla Open Policy & Advocacy BlogFCC Reply Comments on Net Neutrality

Today is the final deadline to file comments as part of the Federal Communications Commission’s open proceeding on net neutrality in the United States. The show of support for real net neutrality over the past six months has been tremendous – so much so that this issue has now received more public comments than any other in FCC history, nearly 1.5 million in total.

Mozilla has been pulling out all the stops as well. In May, we submitted an original petition to the FCC to propose a new path forward on the difficult question of authority, and to shake up a debate that had not seen many new ideas. We’ve also launched global teach-ins, filed comments, and joined last week’s Day of Action, among other activities.

Our reply comments filed today build on these past actions, summarize the state of the debate, and respond to net neutrality opponents. Our comments are structured around four points:

  1. Most parties agree on most issues. The FCC should adopt enforceable rules, including some form of a no blocking and no unreasonable discrimination rule, with an exception for reasonable network management.
  2. Mozilla’s classification theory is a viable and strong path forward. Mozilla’s approach ensures real net neutrality while bypassing the political conversation over reclassification, by articulating a new, not yet classified Title II service offered to remote end points.
  3. The FCC must adopt a presumption against paid prioritization. Allowing prioritization would degrade other uses of the Internet, and thus cause harm to user choice, innovation, and competition.
  4. The same rules should apply to mobile and fixed services. There is one Internet and it must remain open for all. Technical requirements for mobile networks can be protected through reasonable network management.

This week, the FCC will conduct roundtables on net neutrality, with varying focuses including technical, legal, and enforcement aspects. I’ll be participating in one of the Friday sessions, focusing on the topic of enforcement. The roundtables will be held in DC, and will include a moderated discussion among a diversity of viewpoints. At this stage of the process, I don’t expect much in the way of agreement – but at least a range of options will be presented and defended for the agency’s consideration. The public has been invited to submit questions in advance over email or Twitter – roundtables@fcc.gov or #FCCRoundtables – though a caveat from the session description: Your questions and identifying information will be made public and included in the official record.

We’ve seen comments, petitions, roundtables, protests, and events for months now. The main thing left is for the agency to make some decisions – and as we note in our comments, the outcome will set the course for the future of the industry, for better or for worse.

There’s still time for you to make your voice heard. You can contact the FCC and members of Congress, and ask them to protect net neutrality, and the choice, innovation, and freedom enjoyed today by all Internet users and developers.

Eric ShepherdThe Sheppy Report: September 12, 2014

You may notice that I’m posting this on Monday instead of Friday (despite what the title says). Last week was a bit of a mess; I had a little bit of a surprise on Wednesday that involved my being taken to the hospital by ambulance (no lights and siren — it wasn’t that big a surprise), and that kept me away from work for most of the rest of the week.

I’m feeling fine now, although there’s some follow-up medical testing to do to check things out.

Anyway! On to the important stuff!

What I did this week

  • Deleted an inappropriate article from the MDN inbox and emailed the author explaining why.
  • Succeeded in first deployment of a test of my new live sample server-side component server project.
  • Discovered the native GitHub application for Mac, which makes my life enormously easier.
  • Added more information to the comments on bug 1063580, about how shift-refreshing of macros no longer lets you pick up changes to macros and submacros.
  • Updated “How to create an MDN account” to consider that you can use either Persona or GitHub now as your authentication service. This fixed bug 1049972.
  • Added the dev-doc-needed keyword to bug 1064843, which is about implementing ::backdrop in Gecko.
  • Updated the draft of the State of MDN for August to include everything I could think of, so Ali can get it posted (which has happened!).
  • Created the soapbox message to appear at the top of MDN pages about the net neutrality petition. This has since been removed once again, since the event in question has passed. This resolved bug 1060483.
  • Finished debugging the SubpageMenuByCategories and MakeColumnsForDL macros. The former is new and the latter has been updated to support multiple <dl> lists with interspersed headings.

Meetings attended this week

Monday

  • #mdndev planning meeting.
  • #mdndev triage meeting.

Last week started off on a high note, with some very productive activity, and then went sideways on Wednesday and never really recovered. I have high hopes for this week though!

Daniel StenbergDaladevelop hackathon

On Saturday the 13th of September, I took part in a hackathon in Falun Sweden organized by Daladevelop.

20-something hacker enthusiasts gathered in a rather large and comfortable room in this place, an almost three hour drive from my home. A number of talks and lectures were held through the day and the difficulty level ranged from newbie to more advanced. My own contribution was a talk about curl followed by one about HTTP/2. Blabbermouth as I am, I exhausted the friendly audience by talking a good total of almost 90 minutes straight. I got a whole range of clever and educated questions and I think and hope we all had a good time as a result.

The organizers ran a quiz for two-person teams. I teamed up with Andreas Olsson in team Emacs, and after having identified x86 assembly, written binary, spotted perl, named Ada Lovelace, used the term lightfoot and provided about 15 more answers we managed to get first prize and the honor of having beaten the others. Great fun!

Daniel GlazmanMolly needs you, again!

There are bad mondays. This is a bad monday. And this is a bad monday because I just discovered two messages - among others - posted by our friend Molly Holzschlag (ANC is Absolute Neutrophil Count):

First message

Second message

If you care about our friend Molly and value all what she gave to Web Standards and CSS across all these years, please consider donating again to the fund some of her friends set up a while ago to support her health and daily life expenses. There are no little donations, there are only love messages. Send Molly a love message. Please.

Thank you.

Marco ZeheYour must read post for this week

This goes out to all my readers who are web developers, or who work with web developers closely enough to hand this to them.

It’s Monday morning, and for this week, I have a must read post for you which you will now bookmark and reference and use with every single web component you build! No, this is not a suggestion, it’s an order which you will follow. Because if you don’t, you’ll miss out on a lot of fun and grattitude! I’m serious! So here goes:

Web Components punch list by Steve Faulkner of the Paciello Group

Read. Read again. Begin to understand. Read again. Understand more. Read yet another time. Get the tools referenced in the post. Check your web component(s) against this list top to bottom. If even a single point is answered “no”, fix it, or get on Twitter and ask for help in the accessibility community on how to fix it. Listen and learn. And repeat for every future web component you build!

And don’t be shy! Tell the world about that your web component is accessible from the start, usable by at least twenty percent of people more than would otherwise! I kid you not!

Happy Monday, and happy coding!

Andy McKayWorking on Open Source

When hiring at Mozilla, having potential candidates who know open source software is almost a requirement. But there's a huge difference between people that work with open source software and those who work on open source.

About 14 years ago when I started interviewing candidates for open source software, even seeing candidates who knew what open source was could be unusual and seen as a advantage. That's not enough now. These days working with open source software is seen as a base requirement. But that's not still enough.

In fact it's almost staggering these days to understand how anyone can build any systems, especially web sites, without using a large amount of open source. So go ahead fill resumes with how you've used Linux, MySQL, PostgreSQL, Python, JavaScript, Ruby and so on. Show me your github, bitbucket, whatever account. Those are buzzwords that keep recruiters happy.

What I really want to see is that you've worked on open source. Have you:

  • contributed to an open source project?
  • published an open source project that is used by someone other yourself (or your company)?
  • given a talk at a open source conference?
  • helped out an open source foundation?
  • written some documentation on open source?
  • participated on mailing lists with other developers about an open source project?
  • dealt with those awesome people who want to help and those trolls who don't?

There's a reason we look for open source developers at Mozilla. It's partly because Mozilla is basically a collection of open source projects with some funding behind it. But also because developers on open source are great at developing code and at working with other people.

Working on open source separates you from those who just use it.

Jeff WaldenRacism from a United States judge. You’ll never guess which one!

A couple days ago I found this ugly passage in a United States legal opinion:

The white race deems itself to be the dominant race in this country. And so it is in prestige, in achievements, in education, in wealth and in power. So, I doubt not, it will continue to be for all time if it remains true to its great heritage and holds fast to the principles of constitutional liberty.

Take a guess who wrote it, and in what context.

A hint

The same person who wrote this immediately continued with these further words, some of which might sound familiar (if improbable):

But in view of the Constitution, in the eye of the law, there is in this country no superior, dominant, ruling class of citizens. There is no caste here. Our Constitution is color-blind, and neither knows nor tolerates classes among citizens. In respect of civil rights, all citizens are equal before the law. The humblest is the peer of the most powerful. The law regards man as man, and takes no account of his surroundings or of his color when his civil rights as guaranteed by the supreme law of the land are involved.

I’ll give you a little space to try to come up with the name and context, if you haven’t already gotten it.

The answer

These passages were written by the first Justice Harlan, dissenting in the notorious Plessy v. Ferguson case. It’s interesting how we now remember Justice Harlan for this solo dissent and for his statement that, “Our Constitution is color-blind, and neither knows nor tolerates classes among citizens.” Yet I’d never heard before, anywhere, that in the exact same paragraph he validated the idea of a dominant race and basically asserted that whites would always be so in the United States.

Justice Harlan certainly deserves credit as the only one of eight justices to hold in favor of Homer Plessy, the New Orleans Comité des Citoyens, and the railroad company that ejected him from a whites-only car (all of whom conspired in a test case to overturn the law). (The ninth justice, David Josiah Brewer, didn’t participate in the case because of the abrupt death of his daughter. It’s unclear how he would have voted had he participated, with his personal history and voting record pointing in somewhat different directions.) As the only Southerner on the Court, and a former slave owner at that, it’s far from what one might have expected of Harlan, or of his colleagues.

Yet at the same time, Justice Harlan adhered to some of the beliefs and prejudices of his time. It is an unfortunate gloss on history that we are less aware of this, than we are of his better-known, more admirable words. We should be aware of both: to correctly understand history, to not fall prey to knowing only that which we want to be true, and to place a historical figure in full context.

Tantek Çelik Happy 8-bit day 2014! #8bitday

8-bit day is the 256th day of the year. This year (and most years) that happens to be Gregorian September 13th. Five years ago I proposed making today an (un)official holiday in honor of all things 8-bit: art, music, video, games, and sure programmers too.

The Math

If you start the year with day 0, in the year 2014, 2014-09-13 (or 2014-256) is day number 255.

  • 255 decimal = FF hex
  • FF hex = 11111111 binary
  • 11111111 binary = 8 bits.

Enjoy some 8-bit stuff

Music
Videos

See Also

Related

Previously

Previously I kept this on my wiki, which is unfortunately still on pbworks.com, so starting this year, I'm retaking that content and blogging it here on my site, until I've implemented my own wiki pages. I'll write a new post once a year, like I have in past years.

Post your favorite 8-bit stuff

Take a moment today to post and celebrate the 8-bit things that you've found and enjoy, and hashtag it #8bitday (e.g. on your own site, Twitter, Instagram, etc.)

Tim TaubertTalk: Keeping secrets with JavaScript - An Introduction to the WebCrypto API

With the web slowly maturing as a platform the demand for cryptography in the browser has risen, especially in a post-Snowden era. Many of us have heard about the upcoming Web Cryptography API but at the time of writing there seem to be no good introductions available. We will take a look at the proposed W3C spec and its current state of implementation.

Slides

Code

https://github.com/ttaubert/secret-notes

Mozilla WebDev CommunityWebdev Extravaganza – September 2014

Once a month, web developers from across Mozilla gather to continue work on our doomsday robot that will force the governments of the world to relinquish control of the internet to us. Crafting robotic monsters is hard work, so we take frequent breaks to avoid burnout, and we find these breaks are a convenient time to talk about the work that we’ve shipped, share the libraries we’re working on, meet new folks, and talk about whatever else is on our minds. It’s the Webdev Extravaganza! The meeting is open to the public; you should stop by!

You can check out the wiki page that we use to organize the meeting, view a recording of the meeting in Air Mozilla, or attempt to decipher the aimless scrawls that are the meeting notes. Or just read on for a summary!

Shipping Celebration

The shipping celebration is for anything we finished and deployed in the past month, whether it be a brand new site, an upgrade to an existing one, or even a release of a library.

Marketplace Redesign

clouserw stopped by to tell us that Firefox Marketplace shipped a redesign! The front page now has a set of modules that can be customized using a set of admin tools, including changing what apps are shown, setting colors and features, and more. Of particular note is the fact that the admin interface for the modules was given a lot of UX attention as well (as opposed to our standard practice of using the default Django admin design), and includes a live preview of what the modules will look like.

Socorro: Out of Memory Crashes and new ADI source

lonnen informs us that Socorro has landed support for logging out-of-memory crashes, meaning that crashes that are suspected of relating to memory now include about:memory logs in the crash data, to help us diagnose those problems. In addition, Socorro is now fetching data about the number of active daily instances of Firefox instead of depending on the data being sent to Socorro in bulk. Socorro uses this data to normalize crash data, and the new source reduces the time spent pulling in the data to under ten minutes.

Air Mozilla now supports pop-out videos

peterbe shared the news that Air Mozilla now supports pop-out videos, meaning you can now launch a new window with the video you want to watch. This gives the viewer more options in how to watch a video while working on something else, as previously you were limited to in-page viewing or full-screen viewing.

Open-source Citizenship

Here we talk about libraries we’re maintaining and what, if anything, we need help with for them.

contribute.json

peterbe had a few pieces of news about contribute.json. First, Air Mozilla and Peekaboo both have live contribute.json files, and Socorro is deploying one soon. Second,  seanbolton and espressive are working on a redesign of the contribute.json webpage. And finally, the validator now supports text and file upload as well as URLs.

New Hires / Interns / Volunteers / Contributors

Here we introduce any newcomers to the Webdev group, including new employees, interns, volunteers, or any other form of contributor. Unfortunately we had no one new to introduce this month.

Roundtable

The Roundtable is the home for discussions that don’t fit anywhere else.

Markeplace in multiple datacenters

clouserw shared an “exploration” he’s working on for moving Marketplace into being hosted in multiple datacenters. While the primary goals are redundancy (if a datacenter goes down) and performance (geographically close to users who normally have to reach servers in the US), one major issue that was raised was handling differing privacy laws between countries that we have datacenters in. Feedback is welcome!

Bedrock running on Cloud9

jgmize wanted to let everyone know that Bedrock can now be set up on Cloud9, allowing developers and contributors to get a running instance of Bedrock with almost no interaction or software installed on their own machine. There’s a quickstart guide for setting it up, and he’s looking for people to try it out and also to consider trying out the model on their own projects as a way of helping on-board new contributors.


If you’re curious, the robot is coming along nicely. Once we’re able to get the imported railgun to clear customs, we should be good to go!

If you’re interested in web development at Mozilla, or want to attend next month’s Extravaganza, subscribe to the dev-webdev@lists.mozilla.org mailing list to be notified of the next meeting, and maybe send a message introducing yourself. We’d love to meet you!

See you next month!

 

Nick CameronA gotcha with raw pointers and unsafe code

This bit me today. It's not actually a bug and it only happens in unsafe code, but it is non-obvious and something to be aware of.

There are a few components to the issue. First off, we must look at `&expr` where `expr` is an rvalue, that is a temporary value. Rust allows you to write (for example) `&42` and through some magic, `42` will be allocated on the stack and `x` will be a reference to it with an inferred lifetime shorter than the value. For example, `let x = &42i;` works as does

struct Foo<'a> {
    f: &'a int,
}
fn main() {
    let x = Foo { f: &42 };
}

Next, we must know that borrowed pointers (`&T`) can be implicitly coerced to raw pointers (`*T`). So if you write `let x: *const int = &42;`, `x` is a raw pointer produced by coercing the borrowed pointer. Once this happens, you have no safety guarantees - a raw pointer can point at memory that has already been freed. This is fine, since you see the raw pointer type and must be aware, but if the type comes from a struct field what looks like a borrowed pointer could actually be a raw pointer:

struct Bar {
    f: *const int,
}
fn main() {
    let x = Bar { f: &42 };
}

Imagine that `Bar` is some other module or crate, then you might assume that `main` is ok here. But it is not. Since the borrowed pointer has a narrow scope and is not stored (the raw pointer does not count for this analysis), the compiler can choose to delete the `42` allocated on the stack and reuse that memory straight after (or even during, probably) the `let` statement. So, `x.f` is potentially a dangling pointer as soon as `x` is available and accessing it will give you bugs. That is OK, you can only do so in an `unsafe` block and thus you should check (i.e., as a programmer you should check) that you can't get a dangling pointer. You must do this whenever you dereference a raw pointer, and the fact that it must happen in unsafe code is your cue to do so.

The final part of this gotcha was that I was already in unsafe code and was transmuting. Of course transmuting is awful and you should never do it, but sometimes you have to. If you do `unsafe { transmute(x) }` then there is no cue in the code that you have a raw pointer. You have no cue to check the dereference, because there is no dereference! You just get a weird bug that only appears on some platforms and depends on the optimisation level of compilation.

Unfortunately, there is nothing we can really do from the language point of view - you just have to be super-careful around unsafe code, and especially transmutes.

Hat-tip to eddyb for figuring out what was going on here.

Mark SurmanSnapping the puzzle together

I’ve had a picture in mind for a while: a vision of FirefoxOS + Appmaker + Webmaker mentor programs coming together to drive a new wave of creativity and content on the web. I believe this would be a way to really show what Mozilla stands for right now: putting access to the Internet in more hands and then helping people unlock the full potential of the web as a part of their lives and their livelihoods.

Puzzle pieces

The thing is: this picture has felt a bit like a puzzle until recently — I can see where it’s going, but we don’t have all the pieces. It’s like a vision or a theory more than a plan. However, over the past few months, things are getting clearer — feels like the puzzle pieces are becoming real and snapping together.

Bangladesh

Dinner w/ Mozilla Bangladesh

I had this ‘it’s coming together’ feeling in spades the other day as I had dinner w/ 20 members of the Mozilla community in Bangladesh. Across from me was a college student named Ani who was telling me about the Bengali keyboard he’d written for FirefoxOS. To his right was a woman named Maliha who was explaining how she’d helped the Mozilla Bangladesh community organize nearly 50 Webmaker workshops in the last two months. And then beside me, Mak was enthusiastically — and accurately — describing Mozilla’s new Mobile Webmaker to the rest of the group. I was rapt. And energized.

More importantly, I was struck by how the people around the table had nearly all the pieces of the puzzle amongst them. At a practical level, they are all actively working on the practicalities of localizing FirefoxOS and making it work on the ground in Bangladesh. They are finding people and places to teach Webmaker workshops. They have offered to help develop and test Appmaker to see if it can really work for users in Bangladesh. And, they see how these things fit together: people around the table talked about how all these things combined have the potential for huge impact. In particular, they talked about the role phones, skills and publishing tools built with Mozilla values could unleash a huge wave of Bengali language content onto the mobile internet. In a country where less than 10% of people speak English. This is a big deal.

The overall theory behind this puzzle is: open platforms + digital skills + local content = an opportunity to disrupt and open up the mobile Internet.

IMG_20140912_204932

Well, at least, that’s my theory. I see local platforms like Firefox OS — and HTML5 in general — as the baseline. They make it possible for anyone to create apps and content for the mobile web on their own terms — and they are easy to learn. In order to unlock the potential of these platforms, we also need large numbers of people to have the skills to create their own apps and content. Which is what we’re trying to tee up with our Webmaker program. Finally, we need a huge wave of local content that smartphone users make for each other — which both Webmaker and Appmaker are meant to fuel. These are the puzzle pieces I think we need.

On this last point: the content needn’t be local per se — but it does need to be something of value to users that the web / HTML5 can provide this better than existing mobile app stores and social networks. Local apps and content — and especially local language content — is a very likely sweet spot here. The Android Play Store and Facebook are bad — or at least limited — in how they support people creating content and apps. In languages like Bengali, the web — and Mozilla — have historically been much better.

But it’s a theory with enough promise — with enough pieces of the puzzle coming together — that we should get out there and test it out in practice. Doing this will require both discipline and people on the ground. Luckily, the Mozilla community has these things in spades.

India Community

Mozillians at Webmaker event in Pune

Talking with a bunch of people from the Mozilla India community underlined this part of things for me — and helped my thinking on how to test the local content theory. Vineel, Sayak and others told me about the recent launch of low cost Firefox OS smartphones in India — including a $33/R1999 phone from a company called Intex. As with Firefox releases in many other countries, the core launch team behind this effort were volunteer Mozilla contributors.
Working with Mozilla marketing staff from Taiwan, members of the Mozilla India community made a plan, trained Intex sales staff and promoted the phone. Early results: Intex sold 15,000 units in the first three days. And things have been picking up from there.

It’s exactly this kind of community driven plan and discipline that we will need to test out the Firefox OS + Appmaker + Webmaker theory. What we need is something like:

  1. Pick a couple of places to test out our theory — India and Bangladesh are likely options, maybe also Brazil and Kenya.
  2. Work with the community to test out the ‘everyone can author an app’ software first — find out what regular users want, adapt the software with them, test again.
  3. Make sure this test includes a strong Webmaker / training component — we should be testing how to teach skills at the same time as testing the software idea.
  4. Make sure we have both phones and a v1 of Mobile Webmaker in local languages
  5. Also, work with community to develop a set of basic app templates in local language — it’s important not to have an ‘empty shelf’ and also to build around things people actually want to make.
  6. Move from research to ‘market’ testing — put Mobile Webmaker on FirefoxOS phones and do a campaign of related Webmaker training sessions.
  7. Step back. See what worked. What didn’t. Iterate. In the market.

This sort of thing is doable in the next six months — but only if we get the right community teams behind us. I’m going to work on doing just that at ReMoCamp in Berlin this weekend. If there is interest and traction, we’ll start moving ahead quickly.

In the meantime, I’d be interested in comments on my theory above. We’re going to do something like this — we need everybody’s feedback and ideas to increase the likelihood of getting it right.


Filed under: mozilla, webmakers

Doug BelshawWeeknote 37/2014

This week I’ve been:

  • Interviewing more people about Web Literacy Map 2.0:
  • Agreeing (with Lyndsey Britton & Lauren Summers) on 8th November 2014 as the date for a re-arranged Maker Party North East.
  • Selling my iPad Mini. I’ve bought a 6.4″ Sony Xperia Z Ultra phablet to replace both it and my Moto G (the 3G version). I’m still waiting for delivery as it was about £50 cheaper to order it from Amazon Germany instead of Amazon UK!
  • Giving feedback on designs for a (much needed) Webmaker badge landing page.
  • Attending GitClub, led by Ricardo Vazquez.
  • Connecting with Gordon Gow about some potentially overlapping interests in web literacy-related mobile learning projects in Sri Lanka.
  • Speaking at (and attending) a Code Acts in Education seminar at the University of Stirling, organised by Ben Williamson. It was a great event, at which I met great people and learned loads! My slides are here.
  • Recording videos as part of the guidance for people to earn Mozilla’s ‘remix the web’ badge on the forthcoming iDEA award platform.

Next week I’m at home all week, interviewing more people about the Web Literacy Map, and starting to think about synthesizing what I’ve been hearing so far. I should also start thinking about my Mozilla Festival sessions and deliverable for the Badge Alliance Digital & Web Literacy working group….

Image CC BY Michael Himbeault

Will Kahn-GreeneInput status: September 12th, 2014

Development

High-level summary:

  • Updated to ElasticUtils v0.10 which will allow us to upgrade our cluster to Elasticsearch 1.1. I'm working on a fix that'll let us to go to Elasticsearch 1.2, but that hasn't been released, yet.
  • Integrated the spicedham library prototype and set it up to classify abusive Input feedback. It's not working great, but that's entirely to be expected. I'm hoping to spend more time on spicedham and classification in Input in 2014q4. Ian did a great job with laying the foundation! Thank you, Ian!
  • Implemented a data retention policy and automated data purging.
  • Made some changes to the Input feedback GET and POST APIs to clarify things in the docs, fix some edge cases and make it work better for Firefox for Android and Loop.
  • Fixed the date picker in Chrome. Thank you, Ruben!

Landed and deployed:

  • c4e8e34 [bug 1055520] Update to ElasticUtils v0.10
  • e023fa4 [bug 1055520] Fix two reshape issues post EU 0.10 update
  • f9ba829 [bug 1055785] Codify data retention policy
  • 91396a8 Generalize About page text so it works for all products
  • 6fc03bf [bug 1053863] Update django to 1.5.9
  • 85709b2 [bug 1055788] Implement data purging
  • c0677a1 [bug 965796] Add a products update page
  • 121588d [bug 1057353] Update django-statsd and pystatsd
  • fe1c740 Add PII-related notes to the API fields
  • f77ecfa [bug 799562] Clarify API field documentation
  • c5eec03 [bug 1055789] Restrict front page dashboard and api to 6 months
  • 0892546 [bug 1059826] Add max_length to url field in API
  • f192f84 [bug 1057617] Fix url data validation
  • aad961d [bug 1030901] Document Input GET API
  • 2f212c5 [bug 1015788] Add flake8 linting
  • d673947 Update coding conventions
  • 27a1b6b Add "maximum" arg to GET API
  • 4f671e4 [bug 1062436] Add flags app and Flag model
  • 9d03d4b Fix flake8_lint issues
  • 0411d91 [bug 1062453] Add flagged view
  • 56f7e24 [bug 1062439] Celery task for classification
  • 7aa2930 [bug 1062455] Add spicedham to vendor (Ian Kronquist)
  • 0d90df3 We don't need spicedham under vendor/packages (Ian Kronquist)
  • a2a491d fix bug 1012965 - Date picker looks broken in chrome (Ruben Vereecken)
  • 0c42213 [bug 1063825] Integrate spicedham into fjord
  • 78a2d63 [bug 1062444] Initial training data
  • 5ca816e [bug 1020307] Prepare for adding gradient to generic form

Current head: 5ca816e

Rough plan for the next two weeks

  1. Working on Dashboards-for-everyone bits. Documenting the GET API. Making it a bit more functional. Writing up some more examples. (https://wiki.mozilla.org/Firefox/Input/Dashboards_for_Everyone)
  2. Gradients (https://wiki.mozilla.org/Firefox/Input/Gradient_Sentiment)

What I need help with

  1. (django) Update to django-rest-framework 2.3.14 (bug #934979) -- I think this is straight-forward. We'll know if it isn't if the tests fail.
  2. (django, cookies, debugging) API response shouldn't create anoncsrf cookie (bug #910691) -- I have no idea what's going on here because I haven't looked into it much.

For details, see our GetInolved page:

https://wiki.mozilla.org/Webdev/GetInvolved/input.mozilla.org

If you're interested in helping, let me know! We hang out on #input on irc.mozilla.org and there's the input-dev mailing list.

Additional thoughts

I've been codifying project plan details on the wiki:

https://wiki.mozilla.org/Firefox/Input

I have no idea who's going to use that information or whether it helps. If you see things that are missing, let me know. It'll help me hone the project management templates I'm using and know which information is important to keep up to date and which information I can let slide until rainy days.

That's it!

Matěj CeplOn bibshare

(this is originally a comment on the post about “scientific Markdown”)

In my previous life I was using heavily TeX and BibTeX for writing a scholarly articles when working on my PhD in sociology. When doing a large BibTeX database of bibliopgraphy there is a certain moment when one needs to establish some order in creating new keys for the individual references. When I hit that moment, I started to look around whether somebody didn’t do some thinking about the design of the bibliography keys. I found almost nothing on the Web perhaps because there was actually a file bibshare (originally in $TEXMF/doc/bibtex/base/bibshare now I cannot find it anywhere, so I have download a version from older tetex RPM to my website). It describes pretty nice standard, which really should be rewritten into RFC or something of that sort. The two biggest advantages are stable keys (so bibliographies can be exchanged) and a more rememberable ones. So, whenever I see now granovetter:AJS-1973-1360 I do remember (and it has been couple of years, since I used BibTeX last time) that it is an awesome article "The Strength of Weak Ties" by Mark Granovetter.

Mozilla Release Management TeamFirefox 33 beta2 to beta3

  • 21 changesets
  • 55 files changed
  • 696 insertions
  • 410 deletions

ExtensionOccurrences
list14
js11
h5
cpp5
html4
jsm2
css2
cc2
xul1
webidl1
java1
ini1
inc1
in1
build1

ModuleOccurrences
layout15
dom9
toolkit8
mobile5
js5
widget3
media3
services2
gfx1
content1

List of changesets:

Mark FinkleBug 1042715 - Add support for Restricted Profiles r=rnewman a=lmandel - 080344c7c80b
Ryan VanderMeulenBacked out changeset 080344c7c80b (Bug 1042715) for Android bustage. - fcf16a67fed4
Tim TaubertBug 1046645 - Mark moz-page-thumb:// as local resources to prevent mixed content warnings f=Mardak r=gavin a=lmandel - e0b583b1210e
Jason OrendorffFollow-up 2 to Bug 1041631, part 1 - Make one last test work when Symbol is not defined. Backported from rev 74637aa07226. a=testonly. - 337d96ca1194
Jason OrendorffBug 1041631, part 2 - Make ES6 Symbols Nightly-only for now. r=Waldo, a=sledru. - eee93220473c
Martijn WargersBug 1058797 - Intermittent test_303567.xul | Result logged after SimpleTest.finish(). r=mak, a=test-only - d2d97af8ecdd
Hiroyuki IkezoeBug 1041262 - Disable autofilling of search engines to avoid failures in unified complete tests when searchengines is in the platform directory. r=mak, a=test-only - 320e081cac62
Landry BreuilBug 1014375 - Properly define JS_PUNBOX64 or JS_NUNBOX32 depending on the CPU arch r=nbp a=lmandel - 31a06334affd
Landry BreuilBug 1014375 followup - add missing ;; to unbreak the tree a=ryanvm. - f89c8ca38b12
Ryan VanderMeulenBug 1023323 - Mark 413361-1.html as fuzzy on Android 4.0. a=test-only - 8338468a7588
Benoit JacobBug 1063048 - Backout 35ff4bfb198f because on DriverVersionMismatch our blacklisting logic is fooled and doesn't protect us against real crashes. r=Bas, a=lmandel - d0885f177e37
Wes JohnstonBug 763671 - Remove gradient from form elements. r=mfinkle, a=lmandel - 7b4d4b3b7598
Mark CapellaBug 1057685 - Regression: Tweak Browser:Quit to maintain existing support for add-ons - part deux. r=wesj, a=lmandel - dbfd31597299
Matt WoodrowBug 1059807 - Mark OSX printing surfaces as being write-only. r=roc, a=lmandel - 7b689c3657e4
Gian-Carlo PascuttoBug 1053264 - Do not use CAPTUREBLT when Desktop Composition is enabled. r=jimm, a=lmandel - 5638564e0d94
Gian-Carlo PascuttoBug 1060796 - Limit screen capture FPS. r=jesup, a=lmandel - 18ba9aece9bd
Benjamin SmedbergBug 1053745 - Add GMP plugin data to FHR. r=gfritzsche, a=lmandel - 02474d192901
Jan-Ivar BruaroeyBug 1062981 - Disable bfcache for pages active MediaManager. r=smaug, r=jesup, a=lmandel - 46abad0899f9
Xidorn QuanBug 1063856 - Add more counter styles from the Predefined Counter Styles document, for better interop and web-compat. r=jfkthame, a=lmandel - 8e9b139e30b9
Nils Ohlmeier [:drno]Bug 1021220 - Verify absence of loopback in SDP offer. r=bwc, a=test-only - fdf2f580b665
Jan-Ivar BruaroeyBug 1063808 - Support old constraint-like RTCOfferOptions for a bit. r=smaug, r=abr, a=lmandel - d4082d3a082c

Mozilla Open Policy & Advocacy BlogReflections on the 9th Internet Governance Forum

I recently returned from Istanbul, Turkey where I attended the 9th annual Internet Governance Forum. This was my third IGF in a row, and my second with Mozilla. Like the others I’ve attended, it was a vibrant event, with over 3000 registrants from very different regions and interests culminating in an energizing, inspiring forum.

This year’s event reinforced my positive position on the IGF. It has a crucial role to play at the core of the Internet governance ecosystem, and it continues to fulfill that role far, far better than any other event. The IGF brings people from all walks of life into the same venue and it gets them to interact with each other and talk about difficult issues, face to face and in real time. This year, even remote participation worked fairly smoothly, as I attended a couple sessions that included speakers on video-conference connections.

Some viewed Turkey as an odd choice for a host, given the country’s history of social media blocking and other interference with free expression and activity online (including a law adopted just after the conclusion of IGF to make it even easier to block Web pages). The sentiment was strong enough to inspire the creation of a competing “Internet Ungovernance Forum” focused on promoting an open, secure, and free-as-in-speech Internet. Despite the undercurrents, both forums were well attended, and featured a broad range of interesting and expert speakers (and even some who were both!).

There is always a spotlight on IGF in the international Internet policy world. This year’s comes from NETmundial in Brazil, and, looking ahead a bit, this October’s ITU Plenipotentiary Conference in Korea, a once-every-four-years convening for high-level intergovernmental activity at the core of the ITU’s mission.

So, what did that spotlight illuminate? As always, there were many broad-ranging discussions on Internet policy issues, and no structural mechanisms to move from policy development to any formalized decision-making. (But for the IGF, this is a feature, not a bug.)

Topically, if last year was the Snowden/surveillance IGF, this year was the net neutrality IGF, with at least three feeder sessions and a three-hour “main session” focused on the topic. I spoke at two of the net neutrality sessions, and attended the others. One of my sessions examined “network enhancement” and its relationship to net neutrality – a timely topic here in the United States, where opponents of strong net neutrality rules often indicate that excessive regulation will discourage investment in infrastructure. The other was the annual working session of the Dynamic Coalition on Network Neutrality, which was praised by conference organizers as one of the most effective examples of the ad-hoc IGF working coalitions. I also contributed a paper to the Coalition’s second annual report, drawing from Mozilla’s petition to the FCC and our July comments.

Surveillance had its moments in the spotlight as well, though it was less emphasized than last year. I spoke on two surveillance-related panels. A session organized by CIGI went straight to one of our core policy themes, trust, and how revelations of expansive surveillance have harmed trust, and what we can do to restore it. A separate session, co-organized by the Internet Society and CDT, focused on responses to surveillance, such as proposals to build additional IXPs and undersea cables, and new laws to mandate localization of data within a country. The group collectively opposed localization mandates as both unhelpful for protecting Internet users from surveillance and potentially disastrous to the global free and open Internet.

The IGF isn’t perfect. But it deserves the role it has as the first stop for collaborative discussion of issues related to governance “on” the Internet. Its mandate from the UN runs for one more year, through the 10th IGF in 2015, and then unless renewed the events will stop. But with massive support from many stakeholder groups in many regions of the world – and a host country for 2016 already lined up, by some accounts – I think the IGF will, and should, continue for many years to come.

Benjamin KerensaOff to Berlin

Right now, as this post is published, I’m probably settling into my seat for the next ten hours headed to Berlin, Germany as part of a group of leaders at Mozilla who will be meeting for ReMo Camp. This is my first transatlantic trip ever and perhaps my longest flight so far, so I’m both […]

William LachanceHacking on the Treeherder front end: refreshingly easy

Over the past two weeks, I’ve been working a bit on the Treeherder front end (our interface to managing build and test jobs from mercurial changesets), trying to help get things in shape so that the sheriffs can feel comfortable transitioning to it from tbpl by the end of the quarter.

One thing that has pleasantly surprised me is just how easy it’s been to get going and be productive. The process looks like this on Linux or Mac:


git clone https://github.com/mozilla/treeherder-ui.git
cd treeherder-ui/webapp
./scripts/web-server.js

Then just load http://localhost:8000 in your favorite web browser (Firefox) and you should be good to go (it will load data from the actually treeherder site). If you want to make modifications to the HTML, Javascript, or CSS just go ahead and do so with your favorite editor and the changes will be immediately reflected.

We have a fair backlog of issues to get through, many of them related to the front end. If you’re interested in helping out, please have a look:

https://wiki.mozilla.org/Auto-tools/Projects/Treeherder#Bugs_.26_Project_Tracking

If nothing jumps out at you, please drop by irc.mozilla.org #treeherder and we can probably find something for you to work on. We’re most active during Pacific Time working hours.

Liz HenryHow to test new features in Firefox 34 Aurora

If you’re a fan of free and open source software and would like to contribute to Firefox, join me for some Firefox feature testing!

There are some nifty features under development right now for Firefox 34 including translation in the browser, making voice or video calls (a feature called “Hello” or “Loop”), debugging information for web developers in the Dev Tools Inspector, and recent improvements to HTML5 gaming.

I’ve written step by step instructions on these
ways to test Firefox 34. If you would like to see what it’s like to improve a popular open source project, trying out these tasks is a good introduction.

Aurora

First, Install the Aurora version of Firefox. It is best to set it up to use multiple profiles. That ensures you don’t use your everyday version of Firefox for testing, so you won’t risk losing your usual profile information. It also makes it easy to restart Firefox with a new, clean profile with all the default settings, very useful for testing. Sometimes I realize I’m running 5 different versions of Firefox at once!

To test “Hello”, try making some voice or video calls from Firefox Aurora. You will need a friend to test with. Or, use two computers that you control. This is a good task to try while joining our chat channels, #qa or #testday on irc.mozilla.org; ask if anyone there wants to test Hello with you. The goal here is mostly to find and report new bugs.

If you test the translation infobar in Aurora you may find some new bugs. This is a fun feature to test. I like trying it on Wikipedia in many different languages, and also looking at newspapers!

If you’re a web developer, you may use Developer Tools in Firefox. I’m asking Aurora users to go through some unconfirmed bug reports, to help improve the Developer Tools Inspector.

If you like games you can test HTML5 web-based games in Firefox Aurora. This helps us improve Firefox and also helps the independent game developers. We have a list of demo games so you can play them, report glitches, and feel like a virtuous open source citizen all at once. Along the way you have opportunities to learn some interesting stuff about how graphics on the web can work (or not work).

Monster madness

These testing tasks are all set up in One and Done, Mozilla QA’s site to start people along the path to joining our open source community. This site was developed with a lot of community contribution including the design and concept by long-time community member Parul and a lot of code by two interns this summer, Pankaj and Maja.

Testing gives a great view into the development process for people who may not (yet) be programmers. I especially love how transparent Mozilla’s process can be. Anyone can report a bug, visible to the entire world in bugzilla.mozilla.org. There are many people watching that incoming stream of bug reports, confirming them and routing them to developer teams, sometimes tagging them as good first bugs for new contributors. Developers who may or may not be Mozilla employees show up in the bugs, like magic . . . if you think of bugmail notifications as magic . . .

It is amazing to see this very public and somewhat anarchic collaboration process at work. Of course, it can also be extremely satisfying to see a bug you discovered and reported, your pet bug, finally get fixed.

Related posts:

Gervase MarkhamPraise and Criticism

Praise and criticism are not opposites; in many ways, they are very similar. Both are primarily forms of attention, and are most effective when specific rather than generic. Both should be deployed with concrete goals in mind. Both can be diluted by inflation: praise too much or too often and you will devalue your praise; the same is true for criticism, though in practice, criticism is usually reactive and therefore a bit more resistant to devaluation.

– Karl Fogel, Producing Open Source Software

Wounds from a friend can be trusted, but an enemy multiplies kisses.

Proverbs 27:6

Armen ZambranoRun tbpl jobs locally with Http authentication (developer_config.py) - take 2

Back in July, we deployed the first version of Http authentication for mozharness, however, under some circumstances, the initial version could fail and affect production jobs.

This time around we have:

  • Remove the need for _dev.py config files
    • Each production config had an associated _dev.py config file
  • Prevented it from running in production environment
    • The only way to enable the developer mode is by appending --cfg developer_config.py
If you read How to run Mozharness as a developer you should see the new changes.

As quick reminder, it only takes 3 steps:

  1. Find the command from the log. Copy/paste it.
  2. Append --cfg developer_config.py
  3. Append --installer-url/--test-url with the right values
To see a real example visit this

Niko MatsakisAttribute and macro syntax

A few weeks back pcwalton introduced a PR that aimed to move the attribute and macro syntax to use a leading @ sigil. This means that one would write macros like:

@format("SomeString: {}", 22)

or

@vec[1, 2, 3]

One would write attributes in the same way:

@deriving(Eq)
struct SomeStruct {
}

@inline
fn foo() { ... }

This proposal was controversial. This debate has been sitting for a week or so. I spent some time last week reading every single comment and I wanted to lay out my current thoughts.

Why change it?

There were basically two motivations for introducing the change.

Free the bang. The first was to “free up” the ! sign. The initial motivation was aturon’s error-handling RFC, but I think that even if we decide not to act on that specific proposal, it’s still worth trying to reserve ! and ? for something related to error-handling. We are very limited in the set of characters we can realistically use for syntactic sugar, and ! and ? are valuable “ASCII real-estate”.

Part of the reason for this is that ! has a long history of being the sigil one uses to indicate something dangerous or surprising. Basically, something you should pay extra attention to. This is partly why we chose it for macros, but in truth macros are not dangerous. They can be mildly surprising, in that they don’t necessarily act like regular syntax, but having a distinguished macro invocation syntax already serves the job of alerting you to that possibility. Once you know what a macro does, it ought to just fade into the background.

Decorators and macros. Another strong motivation for me is that I think attributes and macros are two sides of the same coin and thus should use similar syntax. Perhaps the most popular attribute – deriving – is literally nothing more than a macro. The only difference is that its “input” is the type definition to which it is attached (there are some differences in the implementation side presently – e.g., deriving is based off the AST – but as I discuss below I’d like to erase that distiction eventually). That said, right now attributes and macros live in rather distinct worlds, so I think a lot of people view this claim with skepticism. So allow me to expand on what I mean.

How attributes and macros ought to move closer together

Right now attributes and macros are quite distinct, but looking forward I see them moving much closer together over time. Here are some of the various ways.

Attributes taking token trees. Right now attribute syntax is kind of specialized. Eventually I think we’ll want to generalize it so that attributes can take arbitrary token trees as arguments, much like macros operate on token trees (if you’re not familiar with token trees, see the appendix). Using token trees would allow more complex arguments to deriving and other decorators. For example, it’d be great to be able to say:

@deriving(Encodable(EncoderTypeName<foo>))

where EncoderTypeName<foo> is the name of the specific encoder that you wish to derive an impl for, vs today, where deriving always creates an encodabe impl that works for all encoders. (See Issue #3740 for more details.) Token trees seem like the obvious syntax to permit here.

Macros in decorator position. Eventually, I’d like it to be possible for any macro to be attached to an item definition as a decorator. The basic idea is that @foo(abc) struct Bar { ... } would be syntactic sugar for (something like) @foo((abc), (struct Bar { ... })) (presuming foo is a macro).

An aside: it occurs to me that to make this possible before 1.0 as I envisioned it, we’ll need to at least reserve macro names so they cannot be used as attributes. It might also be better to have macros declare whether or not they want to be usable as decorators, just so we can give better error messages. This has some bearing on the “disadvantages” of the @ syntax discussed below, as well.

Using macros in decorator position would be useful for those cases where the macro is conceptually “modifying” a base fn definition. There are numerous examples: memoization, some kind of generator expansion, more complex variations on deriving or pretty-printing, and so on. A specific example from the past was the externfn! wrapper that would both declare an extern "C" function and some sort of Rust wrapper (I don’t recall precisely why). It was used roughly like so:

externfn! {
    fn foo(...) { ... }
}

Clearly, this would be nicer if one wrote it as:

@extern
fn foo(...) { ... }

Token trees as the interface to rule them all. Although the idea of permitting macros to appear in attribute position seems to largely erase the distinction between today’s “decorators”, “syntax extensions”, and “macros”, there remains the niggly detail of the implementation. Let’s just look at deriving as an example: today, deriving is a transform from one AST node to some number of AST nodes. Basically it takes the AST node for a type definition and emits that same node back along with various nodes for auto-generated impls. This is completely different from a macro-rules macro, which operates only on token trees. The plan has always been to remove deriving out of the compiler proper and make it “just another” syntax extension that happens to be defined in the standard library (the same applies to other standard macros like format and so on).

In order to move deriving out of the compiler, though, the interface will have to change from ASTs to token trees. There are two reasons for this. The first is that we are simply not prepared to standardize the Rust compiler’s AST in any public way (and have no near term plans to do so). The second is that ASTs are insufficiently general. We have syntax extensions to accept all kinds of inputs, not just Rust ASTs.

Note that syntax extensions, like deriving, that wish to accept Rust ASTs can easily use a Rust parser to parse the token tree they are given as input. This could be a cleaned up version of the libsyntax library that rustc itself uses, or a third-party parser module (think Esprima for JS). Using separate libraries is advantageous for many reasons. For one thing, it allows other styles of parser libraries to be created (including, for example, versions that support an extensible grammar). It also allows syntax extensions to pin to an older version of the library if necessary, allowing for more independent evolution of all the components involved.

What are the objections?

There were two big objections to the proposal:

  1. Macros using ! feels very lightweight, whereas @ feels more intrusive.
  2. There is an inherent ambiguity since @id() can serve as both an attribute and a macro.

The first point seems to be a matter of taste. I don’t find @ particularly heavyweight, and I think that choosing a suitable color for the emacs/vim modes will probably help quite a bit in making it unobtrusive. In constrast, I think that ! has a strong connotation of “dangerous” which seems inappropriate for most macros. But neither syntax seems particularly egregious: I think we’ll quickly get used to either one.

The second point regarding potential ambiguities is more interesting. The ambiguities are easy to resolve from a technical perpsective, but that does not mean that they won’t be confusing to users.

Parenthesized macro invocations

The first ambiguity is that @foo() can be interpreted as either an attribute or a macro invocation. The observation is that @foo() as a macro invocation should behave like existing syntax, which means that either it should behave like a method call (in a fn body) or a tuple struct (at the top-level). In both cases, it would have to be followed by a “terminator” token: either a ; or a closing delimeter (), ], and }). Therefore, we can simply peek at the next token to decide how to interpret @foo() when we see it.

I believe that, using this disambiguation rule, almost all existing code would continue to parse correctly if it were mass-converted to use @foo in place of the older syntax. The one exception is top-level macro invocations. Today it is common to write something like:

declaremethods!(foo, bar)

struct SomeUnrelatedStruct { ... }

where declaremethods! expands out to a set of method declarations or something similar.

If you just transliterate this to @, then the macro would be parsed as a decorator:

@declaremethods(foo, bar)

struct SomeUnrelatedStruct { ... }

Hence a semicolon would be required, or else {}:

@declaremethods(foo, bar);
struct SomeUnrelatedStruct { ... }

@declaremethods { foo, bar }
struct SomeUnrelatedStruct { ... }

Note that both of these are more consistent with our syntax in general: tuple structs, for example, are always followed by a ; to terminate them. (If you replace @declaremethods(foo, bar) with struct Struct1(foo, bar), then you can see what I mean.) However, today if you fail to include the semicolon, you get a parser error, whereas here you might get a surprising misapplication of the macro.

Macro invocations with braces, square or curly

Until recently, attributes could only be applied to items. However, recent RFCs have proposed extending attributes so that they can be applied to blocks and expressions. These RFCs introduce additional ambiguities for macro invocations based on [] and {}:

  • @foo{...} could be a macro invocation or an annotation @foo applied to the block {...},
  • @foo[...] could be a macro invocation or an annotation @foo applied to the expression [...].

These ambiguities can be resolved by requiring inner attributes for blocks and expressions. Hence, rather than @cold x + y, one would write (@!cold x) + y. I actually prefer this in general, because it makes the precedence clear.

OK, so what are the options?

Using @ for attributes is popular. It is the use with macros that is controversial. Therefore, how I see it, there are three things on the table:

  1. Use @foo for attributes, keep foo! for macros (status quo-ish).
  2. Use @foo for both attributes and macros (the proposal).
  3. Use @[foo] for attributes and @foo for macros (a compromise).

Option 1 is roughly the status quo, but moving from #[foo] to @foo for attributes (this seemed to be universally popular). The obvious downside is that we lose ! forever and we also miss an opportunity to unify attribute and macro syntax. We can still adopt the model where decorators and macros are interoperable, but it will be a little more strange, since they look very different.

The advantages of Option 2 are what I’ve been talking about this whole time. The most significant disadvantage is that adding a semicolon can change the interpretation of @foo() in a surprising way, particularly at the top-level.

Option 3 offers most of the advantages of Option 2, while retaining a clear syntactic distinction between attributes and macro usage. The main downside is that @deriving(Eq) and @inline follow the precedent of other languages more closely and arguably look cleaner than @[deriving(Eq)] and @[inline].

What to do?

Currently I personally lean towards options 2 or 3. I am not happy with Option 1 both because I think we should reserve ! and because I think we should move attributes and macros closer together, both in syntax and in deeper semantics.

Choosing between options 2 and 3 is difficult. It seems to boil down to whether you feel the potential ambiguities of @foo() outweigh the attractiveness of @inline vs @[inline]. I don’t personally have a strong feeling on this particular question. It’s hard to say how confusing the ambiguities will be in practice. I would be happier if placing or failing to place a semicolon at the right spot yielded a hard error.

So I guess I would summarize my current feeling as being happy with either Option 2, but with the proviso that it is an error to use a macro in decorator position unless it explicitly opts in, or Option 3, without that proviso. This seems to retain all the upsides and avoid the confusing ambiguities.

Appendix: A brief explanation of token trees

Token trees are the basis for our macro-rules macros. They are a variation on token streams in which tokens are basically uninterpreted except that matching delimeters ((), [], {}) are paired up. A macro-rules macro is then “just” a translation from a token tree to another token. This output token tree is then parsed as normal. Similarly, our parser is actually not defined over a stream of tokens but rather a token tree.

Our current implementation deviates from this ideal model in some respects. For one thing, macros take as input token trees with embedded asts, and the parser parses a stream of tokens with embedded token trees, rather than token trees themselves, but these details are not particularly relevant to this post. I also suspect we ought to move the implementation closer to the ideal model over time, but that’s the subject of another post.

Ricky RosarioSUMO Development Update 2012.1

SUMO Dev goes agile

Inspired by the MDN Dev team, the SUMO Dev team decided to try an agile-style planning process in 2012.

To be fair, we have always been pretty agile, but perhaps we were more on the cowboy side than the waterfall side. We planned our big features for the quarter and worked towards that. Along the way, we picked up (or were thrown) lots of other bugs based on the hot issue of the day or week, contributor requests, scratching our own itch, etc. These bugs ended up taking time away from the major features we set as goals and, in some cases, ended up delaying them. This new process should help us become more predictable.

Starting out by copying what MDN has been doing for some time now, we are doing two week sprints. We will continue to push out new code weekly for now, so it is kind of weird in that each sprint has two two milestones within it. We will continue to name the milestones by the date of the push (ie, "2012-01-24" for today's push) and we are naming sprints as YEAR.sprint_number (ie, "2012.1" was our first sprint). We hope to will be doing continuous deployment soon. At that point we will only have to track one milestone (the sprint) at a time. For more details on our process, check out our Support/SUMOdev Sprints wiki page.

2012.1 sprint

We just pushed the second half of our first sprint to production. Some data:

  • Closed Stories: 26
  • Closed Points: 34
  • Developer Days: 36
  • Velocity: .94 pts/day

Our major focus of this sprint was getting our Elastic Search implementation (we are in the process of switching from Sphinx) to the point where we can index and start rolling it out to users. After today's push, we will find out whether this is working properly. *fingers crossed* (UPDATE: we did hit an issue with the indexing.)

Other stuff we landed:

  • Initial support for the apps marketplace. Basically, a landing page and a question workflow that integrates with zendesk for 1:1 help.
  • KPI (Key Performance Indicator) Dashboard. We landed the first chart which displays % of solved questions (it has a math bug in it that will get fixed in the next push).
  • Some minor UI fixes and improvements.

2012.2 sprint

We are currently halfway through our second sprint. Our main goals with this sprint are to get Elastic Search out to 15% of our users and to add a bunch of new metrics charts to the KPI Dashboard.

In my opinion, this new planning process is going well so far. The product team has better insight into what the dev team is up to day to day. And the dev team has better sense about what the short term priorities are. Probably the most awesome thing about it is that we are collecting lots of great data. The part I have liked the least so far has been the actual planning sessions, I end up pretty tired after those. I think it just needs a little getting used to and it is only 1-2 hours every two weeks.

:-)

Ricky RosarioSUMO Development: 2012.3 and 2012.4 Update

Oops, I procrastinated forgot to post an update for 2012.3 and we are done with 2012.4 too now.

2012.3 sprint

  • Closed Stories: 26
  • Closed Points: 37 (3 aren't used in the velocity calculation as they were fixed by James and Kadir - Thanks!)
  • Developer Days: 28
  • Velocity: 1.21 pts/day

The 2012.3 sprint went very well. We accomplished most of the goals we set out to do. We rolled out Elastic Search to 50% of our users and had it going for several days. We fixed some of the blocker bugs and came up with a plan for reindexing without downtime. Everything was great until we decided to add some timers to the search view in order to compare times of the Elastic Search vs the Sphinx code path. As soon as we saw some data, we decided to shut down Elastic Search. Basically, the ES path was taking about 4X more time than the Sphinx path. Yikes! We got on that right away and started looking for improvements.

On the KPI Dashboard side, we landed 4 new charts as well as some other enhancements. The new charts show metrics for:

  • Search click-through rate
  • Number of active contributors to the English KB
  • Number of active contributors to the non-English KB
  • Number of active forum contributors

We did miss the goal of adding a chart for active Army of Awesome contributors, as it turned out to be more complicated than we initially thought. So that slipped to 2012.4.

2012.4 sprint

  • Closed Stories: 20
  • Closed Points: 24
  • Developer Days: 19
  • Velocity: 1.26 pts/day

The 2012.4 sprint was sad. It was the first sprint without ErikRose :-(. We initially planned to have TimW help us part time, but he ended up getting too busy with his other projects. We did miss some of our initial goals, but we did as good as we could.

The good news is that we improved the search performance with ES a bunch. It still isn't on par with Sphinx but it is good enough to where we went back to using it for 50% of the users. We have plans to make it faster, but for now it looks like the click-through rates on results are already higher than what we get with Sphinx. That makes us very happy :-D.

We added two new KPI dashboard charts: daily unique visitors and active Army of Awesome contributors. We also landed new themes for the new Aurora community discussion forums.

2012.5 sprint

This week we started working on the 2012.5 sprint. Our goals are:

  • Elastic Search: refactor search view to make it easier to do ES-specific changes.
  • Elastic Search: improve search view performance (get us closer to Sphinx).
  • Hide unanswered questions that are over 3 months old. They don't add any value, so there is no reason to show them to anybody or have them indexed by google and other search engines.
  • Branding and styling updates for Marketplace pages
  • KPI Dashboard: l10n chart
  • KPI Dashboard: Combine solved and responded charts

We are really hoping to be ready to start dialing up the Elastic Search flag to 100% by the time we are done with this sprint.

Ricky RosarioSUMO Development: 2012.2 Update

Yesterday we shipped the second half of the 2012.2 sprint. We ended up accomplishing most of our goals:

  • [Elastic Search] Perform full index in prod - DONE
  • [Elastic Search] Roll out to 15% of users - DONE
  • Add more metrics to KPI dashboard - INCOMPLETE (We landed 3 out of the 4 new graphs we wanted).

Not too bad. In addition to this, we made other nice improvements to the site:

Great progress for two weeks of work! Some data from the sprint:

  • Closed Stories: 30
  • Closed Points: 38
  • Developer Days: 35
  • Velocity: 1.08 pts/day

Onward to 2012.3

We are now a little over halfway into the 2012.3 sprint. Our goals are to roll out Elastic Search to 50% of users, be ready to roll out to 100% (fix all blockers) and add 5 new KPI metrics to the KPI dashboard. So far so good, although we keep finding new issues as we continue to roll out Elastic Search to more users. That deserves it's own blog post though.

Ricky RosarioJoined the Mozilla Web Team

After 3 great years at Razorfish, I decided to move on and joined Mozilla 2 weeks ago. I will be working remote, but I spent the first week in Mountain View doing new hire orientation, setting up my shiney new MBP i7, setting up development environments for zamboni (new addons site) and kitsune (new support site), and fixing some easy bugs to start getting familiar with the codebase.

So far, I am loving it. Some of my initial observations:

  • My coworkers are super smart and awesome.
  • The main communication channel is through IRC (even when people are sitting nearby in the office). This works out great for the remote peeps like myself.
  • We use git/github for the our branch -> work on bug/feature -> review -> commit workflow. I am loving the process and github helps a ton with their UI for commenting on code.
  • Continuous Integration is the nuts.
  • Automated functional testing ^^.
  • Writing open source software full-time, and getting paid? Unreal!

I am working on SUMO (support.mozilla.com). It is currently going through a rewrite from tiki wiki to django (kitsune project). Working full time with django is like a dream come true for me (a very nerdy dream :).

Anyway, it is very exciting to work for Mozilla serving over 400 million Firefox users. I am looking forward to this new chapter in my career!

Ricky Rosariodotjs: My first Firefox Add-on

Inspired by defunkt's dotjs Chrome extension, I finally decided to play with the new add-on sdk to port the concept to Firefox. dotjs executes JavaScript files in ~/.js based on their filename and the domain of the site you are visitng. For example, if you navigate to http://www.twitter.com, dotjs will execute ~/.js/twitter.com.js. It also loads in jQuery so you can use jQuery with in your scripts even if the site doesn't use jQuery (it is loaded with .noConflict so it doesn't interfere with any existing jQuery on the page).

You can get the add-on for Firefox 4 on AMO and it doesn't require a browser restart (woot!). The code is on github. Feedback and patches welcome!

Ricky Rosariosupport.mozilla.org (SUMO) +dev in 2013

This is my first and last blog post for 2013!

Whewww, 2013 has been another splendid year for SUMO and the SUMO/INPUT Engineering team. We did lose (and missed a ton) our manager, James Socol, early in the year and I took over the managerial duties for the team, but the core dev team stayed intact.

Some metrics

Here are some metrics about what our platform, team and community was up to in 2013:

  • Page views: 502,812,271
  • Visits: 255,122,331
  • Unique visits: 190,633,959
  • Questions asked: 33,482
  • Questions replied to: 31,746 (94.8%)
  • Questions solved: 9,048 (27%)
  • Replies to questions: 119,440
  • Support Forum contributors:
    1+ answers: 8,723
    2+ answers: 3,436
    3+ answers: 1,764
    5+ answers: 742
    10+ answers: 247
    25+ answers: 97
    50+ answers: 63
    100+ answers: 42
    250+ answers: 22
    500+ answers: 17
    1000+ answers: 11
    2500+ answers: 7
    5000+ answers: 3
    10000+ answers: 1 (20,057 answers by cor-el)
  • Army of Awesome tweets handled: 46,030
  • Army of Awesome contributors: 911
  • Knowledge Base (KB) Revisions: 16,561
    en-US KB Revisions: 2,975
    L10n KB Revisions: 13,586
  • Locales with activity: 55
  • en-US KB Contributors: 165
  • L10n KB Contributors: 607
  • KB Helpful votes: 4,214,528 (72.6%)
  • KB Unhelpful votes: 1,587,416 (27.4%)

More metrics

Willkg wrote a blog post with that contains a lot more metrics specific to our development (bugs filed, bugs resolved, commits, major projects, etc.). Go check it out!

I wanted to highlight a few things he mentioned:

In 2011, we had 19 people who contributed code changes.
In 2012, we had 23 people.
In 2013, we had 32 people.

YAY!

Like 2011 and 2012, we resolved more bugs than we created in 2013. That's three years in a row! I've never seen that happen on a project I work on.

WOOT!

Input also had a great year in 2013. Check out willkg's blog post about it.

Onward

2013 was a great year for the SUMO platform. We finetuned the KB information architecture work we began in 2012 and simplified all of the landing pages (home, product, topic). In 2014, I am hoping we can make the Support Forum as awesome as the KB is today.

In addition to making the KB awesomer... The Support Forums now support more locales than just English. We now send HTML and localized emails! We added Open Badges! We switched to YouTube for videos. We improved search for locales. We made deployments better. We implemented persona (not enabled yet). We implementated escalation of questions to the helpdesk. We added lots of new and improved dashboards and tools for contributors and community managers. At the same time, we made lots of backend and infrastructure improvements that make the site more stable and resilient and our code more awesome.

As a testament to the awesomeness of the platform, new products have come to us to be their support platform. We are now the support site for Webmaker and will be adding Open Badges and Thunderbird early in 2014.

Thanks to the amazing awesome splendid dev team, the SUMO staff and the community for an awesome 2013!

Christie KoehlerAn Update from the MozillaWiki Team, including a report from Wikimania London

Last week we pushed a major upgrade to MozillaWiki, one that was months in the making. This post discusses the process of that upgrade and also talks about work the MozillaWiki Team did while together in London for Wikimania.

Who is the MozillaWiki Team?

The MozillaWiki team (formerly called the Wiki Working Group) is a mix of paid and volunteer contributors working to improve MozillaWiki. It is facilitated by MozillaWiki module owner (myself) and peers Gordon P. Hemsley and Lyre Calliope (both volunteer contributors).

Results from MozillaWiki user survey informs current roadmap

This summer, OPW (GNOME Outreach Program for Women) intern Joelle conducted a survey of MozillaWiki users. Much of our current roadmap is informed by the results of this survey, including re-organizing the Main Page, making information easier to find, improving the mobile experience and making editing easier.

If you’re interested in the results of that survey, watch her presentation Improving the Gateway: Mozilla Wiki User Research.

Why upgrade Mozilla Wiki now?

The primary motivation for this upgrade was to make current the version of MediaWiki, the software that runs MozillaWiki. Running a relatively older version of MediaWiki (1.19) prevented us from utilizing newer, beneficial features as well as useful extensions that require current versions of MediaWiki.

The Mozilla Wiki now utilizes MediaWiki version 1.23, and you can read about key features and improvements here: https://wiki.mozilla.org/MozillaWiki:News/2014-08/Upgrade_to_MediaWiki_1.23#MediaWiki_changes

This upgrade was carried out in two steps. The first was to change the default skin to Vector, which we did at the beginning of August. The second was to upgrade the software and require all users to use the new skin. This work we did last week.

Why did we choose Vector and drop support for all other skins?

Creating and maintaining MediaWiki skins is a complex and time-consuming process.

The two previous custom skins used on MozillaWiki were Cavendish and GMO. Already these themes, particularly GMO, were missing features available to users in officially supported skins. Our planned upgrade would make this disparity in user experience even greater. While planning the upgrade, we determined it didn’t make sense to expend resources keeping these skins tested and up to date, nor did it make sense to continue to offer a broken user experience just to maintain familiarity.

We selected Vector as the default skin because it is the one supported by MediaWiki itself and is thereby guaranteed to be stable and fully-featured. MonoBook is another theme supported by MediaWiki and we have left that enabled and available to use for those users who want an alternative look and feel. (You can make this change on your preferences page.)

Report from Wikimania London

As I mentioned, the MozillaWiki team has been preparing for and planning for this upgrade for several months. A small group of us gathered in London this August to have dedicated time to work together together and learn about MediaWiki and how to best utilize it at Mozilla by attending Wikimania, the annual MediaWiki community conference

The group included an even mix of paid and volunteer contributors who had been regularly participating in MozillaWiki team activities: Lyre Calliope, Jennie Halperin, Joelle F, Gordon P. Hemsley, C Liang and myself.

We spent the first two days hacking on MozillaWiki and the other three attending conference sessions and hacking together in between.

Having this rare time together in one place allowed us to get a lot done in a relatively short period of time.

Tasks we accomplished include:

  • updated sidebar (only visible in Vector and MonoBook)
  • created and deployed a new Main Page
  • roadmap planning through 2015 q1
  • planned and tested an upgrade to MediaWiki 1.23
  • continued to work on category planning

During the Wikimania conference, we accomplished the following:

  • learned about upcoming changes in MediaWiki, such as the new search extension (elastic search)  and visual editor
  • generated ideas for engaging new contributors across Mozilla projects, via targeted campaigns and directed play
  • generated ideas for recognizing different kinds of contributions leveraging badges and other projects at Mozilla
  • increased awareness of the Mozilla Wiki in the larger wiki community
  • learned about ways to enable real-time collaboration on the wiki
  • invited a number of Wikimedians to join Mozilla via the Wiki Working Group, CBT, and other areas

All of this information and collaboration helped us create our current roadmap.

Improvements planned for rest of 2014

We’re really proud of the work we’ve done on the Mozilla Wiki so far, but we’ve no intention to slow down yet. Improvements we’re planning to roll out this year, include:

  • Bug 1051201 – Audit and adjust user rights (to restore important feature to users and make wiki easier to use)
  • Bug 1051189 – Install MobileFrontend extension (to provide a mobile-friendly interface)
  • Bug 915187 – Improve search
  • Bug 1051204 – Implement real-time collaborative editing
  • Bug 1051206 – Improve discussion and collaboration
  • Bug 1064994 – Improve page categorization

An invitation to Participate

We hope you’re liking our work on MozillaWiki so far! We invite all those who would like to contribute to the wiki to join our regular MozillaWiki team meetings which are every other Tuesday at 8:30am PT (15:30 UTC). Our next meeting is 16 September. Participation details.

Monica ChewMaking decisions with limited data

It is challenging but possible to make decisions with limited data. For example, take the rollout saga of public key pinning.

The first implementation of public key pinning included enforcing pinning on addons.mozilla.org. In retrospect, this was a bad decision because it broke the Addons Panel and generated pinning warnings 86% of the time. As it turns out, the pinset was missing some Verisign certificates used by services.addons.mozilla.org, and the pinning enforcement on addons.mozilla.org included subdomains. Having more data lets us avoid bad decisions.

To enable safer rollouts, we implemented a test mode for pinning. In test mode, pinning violations are counted but not enforced. With sufficient telemetry, it is possible to measure how badly sites would break without actually breaking the site.

Due to privacy restrictions in telemetry, we do not collect per-organization pinning violations except for Mozilla sites that are operationally critical to Firefox. This means that it is not possible to distinguish pinning violations for Google domains from Twitter domains, for example. I do not believe that collecting the aggregated number of pinning violations for sites on the Alexa top 10 list constitutes a privacy violation, but I look forward to the day when technologies such as RAPPOR make it easier to collect actionable data in a privacy-preserving way.

Fortunately for us, Chrome has already implemented pinning on many high-traffic sites. This is fantastic news, because it means we can import Chrome’s pin list in test mode with relatively high assurance that the pin list won’t break Firefox, since it is already in production in Chrome.

Given sufficient test mode telemetry, we can decide whether to enforce pins instead of just counting violations. If the pinning violation rate is sufficiently low, it is probably safe to promote the pinned domain from test mode to production mode. The screenshot below shows a 3 week period where we promoted cdn.mozilla.com and media.mozilla.com and Google domains to production, as well as expand coverage on Twitter to include all subdomains.



Because the current implementation of pinning in Firefox relies on built-in static pinsets and we are unable to count violations per-pinset, it is important to track changes to the pinset file in the dashboard. Fortunately HighStock supports event markers which somewhat alleviates this problem, and David Keeler also contributed some tooltip code to roughly associate dates with Mercurial revisions. Armed with the timeseries of pinning violation rates, event markers for dates that we promoted organizations to production mode (or high-traffic organizations like Dropbox were added in test mode due to a new import from Chromium) we can see whether pinning is working or not.

Telemetry is useful for forensics, but in our case, it is not useful for catching problems as they occur. This limitation is due to several difficulties, which I hope will be overcome by more generalized, comprehensive SSL error-reporting and HPKP:
  • Because pinsets are static and built-in, there is sometimes a 24-hour lag between making a change to a pinset and reaching the next Nightly build.
  • Telemetry information is only sent back once per day, so we are looking at a 2-day delay between making a change and receiving any data back at all.
  • Telemetry dashboards (as accessible from telemetry.js and telemetry.mozilla.org) need about a day to aggregate, which adds another day.
  • Update uptake rates are slow. The median time to update Nightly is around 3 days, getting to 80% takes 10 days or longer.
Due to these latency issues, pinning violation rates take at least a week to stabilize. Thankfully, telemetry is on by default in all pre-release channels as of Firefox 31, which gives us a lot more confidence that the pinning violation rates are representative.

Despite all the caveats and limitations, using these simple tools we were able to successfully roll out pinning pretty much all sites that we’ve attempted (including AMO, our unlucky canary) as of Firefox 34 and look forward to expanding coverage.

Thanks for reading, and don’t forget to update your Nightly if you love Mozilla! :)

Kim MoirMozilla pushes - August 2014

Here's August 2014's monthly analysis of the pushes to our Mozilla development trees.  You can load the data as an HTML page or as a json file.



Trends
It was another record breaking month.  No surprise here!

Highlights
  • 13090 pushes
    • new record
  • 422 pushes/day (average)
    • new record
  • Highest number of pushes/day: 690 pushes on August 20.  This same day corresponded with our first day where we ran over 100,000 test jobs.
    • new record
  • 23.12 pushes/hour (average)

General Remarks
Both Try and Gaia-Try have about 36% each of the pushes.  The three integration repositories (fx-team, mozilla-inbound and b2g-inbound) account around 21% of all the pushes.


Records
August 2014 was the month with most pushes (13,090  pushes)
August 2014 has the highest pushes/day average with 620 pushes/day
July 2014 has the highest average of "pushes-per-hour" with 23.51 pushes/hour
August 20, 2014 had the highest number of pushes in one day with 690 pushes






Prashish RajbhandariThe Plan – After MozDrive

IMG_8370

“You should install a GPS tracking app in your phone so that we can track your location anytime.”

“If we don’t see your social media update every hour, we’re going to call and find you.”

“When you go through this route, don’t go wandering around except the highway. Just drive, don’t stop.”

“You better be in one piece at the end of the 25 days. We need you here!”

It was my last day as an intern in the Silicon Valley. My colleagues and supervisor jokingly threw comments and suggestions at me while I was debugging my final piece of production code before I leave. All my luggage and equipments were packed and stationed in the office so that I could directly go to the airport to catch my flight. My schedule was so tight that I had to drive out the day I landed in Cincinnati. There was no time to relax nor meet friends after I got home. I had to immediately check-in with my friend who was suppose to drive out with me, pack my remaining things, make sure we have everything for the journey (food, medicines and utilities) and head out. The fatigue after the 12-hour flight along with the timezone difference was the last thing on my mind. To understand the scenario, let me take you back a bit.

It had been 23 hours since I took a short nap, let alone good sleep. I was participating in an intern hackathon in LinkedIn HQ (probably one of the best hackathons that I’ve been to). It was 3 am in the morning and I was so caffeinated to a point that I was lost in my own code base. As I was working to fix a nagging problem, I received a notification in my inbox. You know those situation when you are feeling so helpless that you wander off as an legitimate excuse in the slightest opportunity in front of you. Yep, that was one of them. I even started opening some unread “Deal of the Week’ emails to reset my brain.

The message read -

“This is approved by the council. We really can’t wait to see the first report from this. : D Good luck : )”

As my brain was still trying to process the email because of how sleep deprived I was, I got another notification.

“Hi Prashish, Please document your trip thoroughly. We are very excited and waiting to see all your videos, pictures, blogposts and reports. : D”

Wow!

It had been little over a week since I sent my proposal to the Mozilla Reps Council and to be honest, I didn’t have much hope for my mega-drive to get approved. I had to stay calm, control my emotions and send out a ‘Thanks!’ email sounding happy and excited. I did that. Before telling this to all my friends and Mozillians who had been constantly supporting me, I had to finish my project in the hackathon. It was a test of control. I shut down the emotions and continued working on the project without sleep for another 17 hours. Even after the presentations was done and the event was officially over, I didn’t want to think with my super-tired head. It was a test of patience. I wanted a sound sleep and then think of the super exciting journey that I would be taking from the month of August.  I couch surfed at a nearby friend’s house in Santa Clara and woke up fresh after a full 12-hour uninterrupted sleep. I passed all my tests. It was the beginning of couch surfing and what was to come in the next one month.

The next few days kept me super busy as I planned and launched the official website, social networking profiles (Twitter, Facebook) and a Q/A page. As I was working on the MozDrive website, I asked several Mozillians for suggestions and testimonials.

William Reynolds, Product Manager at Mozilla – “I’m excited about the Mozilla Awareness Drive. This is one of the most ambitious campaigns organized by a Mozillian. There’s nothing like visiting Mozilla and Firefox fans and having casual chats with them.”

Sayak  Sarkar, Mozilla Reps Super Mentor  – “I think that this is perhaps one of the most ambitious yet promising initiatives towards spreading the Mozilla mission and awareness about the open web since the Firefox Crop Circle initiative. This initiative speaks out a great lot about how passionate Mozillians are towards the project and how much they are inspired towards contributing towards a common goal of a Free and Open Web.”

The testimonials by Sayak and William really caught my eyes as both of them used the phrase – ‘one of the most ambitious’. To be honest, I didn’t realize the scale of this project until the very last moment. It isn’t that I didn’t understand the project but the desire to do something meaningful for the Mozilla community made the whole planning process look very straightforward. You see, it had been little less than a year since I came to the United States as a graduate student. Back in Kathmandu, Nepal, I would be attending/organizing Mozilla related events in a regular basis; be it orientations, hackathons or meet ups. That drastically changed after I stepped in the United States as I was adapting to the new environment and getting myself caught in the new world around me. To be fair, I did attend the Mozilla Festival in London and Mozilla Reps Meet up in Portland the same year. But, I felt I didn’t make a lot of impact that I would have liked to. In Nepal, there was a huge movement in Mozilla and Open Source that you could actually see the community growing and getting more active. That was something that I wanted to do here too.

The Mozilla Community is very close to my heart because everyone really cares about their work and how that impacts lives around the world. Every Mozillians that I meet are passionate about their work and the community. There is no ‘I’ but ‘We’ in our community. You don’t see a lot of these sort of things in the world we live in. And to be part of this, always makes me a proud Mozillian. I could have easily spent my 25-days break completing full seasons of TV series that I’ve always wanted to. Or if I wanted to be productive, work on a hobby project. Both of them sounded fun as I had spent months working on many products in a very competitive startup in the heart of Silicon Valley. But that’s not something Mozillians do. A Mozillian would spend his free time taking actions on how he/she could build communities together. A Mozillian would plan and work to make the web free and open. A Mozillian would create a movement. That’s what I wanted to do. I wanted to inspire thousands of Mozillians around the world to take actions on their dreams to make them a reality. That’s the reason I set out on this incredible journey to travel around the United States to spread the love about Mozilla and the Open Web.

To tell you the truth, I’m freaking scared of driving. But who isn’t? When there are cars zooming from every direction, the only thought in my head is reaching my destination safely. I never drove a car for more than 10 hours total in my life (which includes me sitting on the driver’s seat and being amazed by all the buttons in front of me). I never had a driving license in Nepal and I barely passed the maneuvering exam on the same day that I was set to fly for San Francisco (internship). That left me with a learner’s permit to drive with a legal driver next to me.

But that didn’t stop me from driving almost half the United States in less than a month. It didn’t stop me from gathering the courage to say ‘YES!’ to the most amazing adventure even though I had no prior experience. It didn’t stop me from taking that risk that would drastically change my life for good.

You might think I’m crazy.

Ask John – that guy who we found in Craigslist to rideshare with us to Los Angeles. He had traveled to almost all the US States and when I told him about our drive, he immediately responded –  ‘You(‘re) crazy man!’.

Or ask Laura – the lady who I met in Nelson-Atkins Museum of Art, Kansas City and had to convince her by showing MozDrive’s Facebook page after she rejected my approach saying – ‘I don’t buy this sh*t’.

Or ask my mom whom I had to convince 4 times everyday that everything will be alright and is under control.

Because driving 13,000 miles in 25 days which is around 8-10 hours everyday is not a joke in any sense. The body and mind could take so much that you needed a lot of self control and motivation throughout the journey for you not to burn out. Yes, there were times when I questioned the entire journey and why I was doing this. Yes, there were times where I wanted to chicken out half way through thinking people will forget about this. But, when you are on a journey which carried such powerful mission and values, that becomes your driving force. When you truly believe in a cause, your physical body will somehow find a way to make it happen and keep you moving forward.

The journey itself was immense where I had opportunities to meet people from all walks of life, culture and countries. I have so much stories to share that I don’t even know where to start. But I promise, I will. That’s why you are reading this. I want you to know what’s in store for everyone in the next 3-4 months. I’m not a writer by any means nor do I have any experience in professional writing. It took me two days just to think and come up this amateur 2,000 word chapter. But, I’m a strong believer of Growth Mindset, and I believe that I can eventually learn the art of expressing my thoughts and ideas through words. My final goal is to write at least 20 chapters of my experience during MozDrive. And to take it one step further – publish it as an ebook in future. That’s the dream!

It is impossible to accomplish a goal without taking actions on it. And this is my first step towards that goal. I know it will take a longer time, but I feel that it will be worth it at the end. And I do hope that you find a positive progress in my writing over time. By taking actions, I simply aim to inspire and awaken hearts of people to do something that they believe in.

If you are reading this – I thank you for taking time and interest in my next journey for MozDrive. Since, I am no writer, I’m looking for people who would be interested to proof-read and edit my future articles for MozDrive. Please send me a message or tweet if you have any suggestions, feedback or are interested in being part of this journey with me.

‘Til then.


Filed under: MozDrive, Mozilla Tagged: mozdrive, mozilla, mozrep

Jorge VillalobosTaller de Firefox OS en Panamá

Me invitaron a dar una charla de desarrollo para apps en Firefox OS este fin de semana, en Ciudad de Panamá. El evento fue organizado por CascoStation, un coworking ubicado en un área muy interesante de la ciudad. Harold de CascoStation hizo un trabajo excepcional para asegurarse que todo saliera bien y todos estuviéramos muy cómodos.

El taller fue similar a los que he dado en el pasado, con algunas mejoras gracias a las lecciones aprendidas. La charla introductoria se puede encontrar aquí: Introducción a Firefox OS. Algunas de las páginas no tienen mucho sentido sin la charla, pero los vínculos son útiles para empezar a trabajar con Firefox OS.

La asistencia fue buena, alrededor de 20 personas. Lo más importante es que la mayoría tuvo suficiente interés para jugar un rato con Firefox OS durante el taller. Tomamos una foto grupal, pero ya varios se habían ido.

Foto de grupo al final

También nos hicieron una nota en El Espectador de Panamá, donde se puede apreciar un poco el ambiente del taller.

Una sorpresa muy grata es que en CascoStation trabaja un artista 3D muy habilidoso, que además aplica sus talentos para crear impresiones 3D con un nivel de detalle espectacular. Él nos creó una figura de Firefox OS que no podría haber quedado mejor.

Modelo de la figura de Firefox OS Figura de Firefox OS

Espero que podamos ver más de estos en el futuro :).

Por último, pude conocer a algunos de los miembros de la nueva comunidad de Mozilla Panamá (y en Facebook). Pudimos hablar sobre los lanzamientos en América Central y los retos que tenemos adelante. Esperamos tener noticias de Costa Rica muy pronto.

La experiencia estuvo excelente, y esperamos ver de nuevo a nuestros amigos panameños en unas semanas para el Encuentro Centroamericano de Software Libre 2014.

Mike ShalBuild System Partial Updates

There is a fairly long dev-platform thread about partial updates - specifically, running './mach build subdirectory'. In this post, we'll compare how this is handled in make-based systems, as well as in tup.

Michael KaplyChanges to CCK2 Support

When I originally came up my CCK2 support options, I thought that folks would use the basic support option as a way to simply show their support for the CCK2. It hasn't really turned out that way, and so effective immediately, I will no longer offer the CCK2 basic support option. I'm simply not getting enough business at that level to warrant the overhead.

Anyone that has already purchased basic support or is in the process of purchasing it will still receive the rest of their 1 year term. After that expires, they will have to choose the free or premium support option.

As far as premium support goes, I'm not planning any changes to that right now, but honestly it hasn't been as successful as I thought it would. I know there are hundreds of companies using the CCK and CCK2, so I'm surprised how few are willing to pay for your support. If anyone has any suggestions on things I can do to encourage folks make the support more appealing, I would appreciate them.

I'm continuing to update the CCK2, so make sure you grab the latest version (2.0.12). There were some update issues, so not everybody was updated.

Christian HeilmannHow to draft a speaker information email

I just had a real happy moment when I got an email from a conference organiser telling me all they need from me and all I need to know in a simple email:


Hi Christian.
I hope you are doing fine.
Your talk “$title” is scheduled for $date at 9:45am (it’s a 40 min talk, plus 5 for Q&A). This is the link to your slot.
$conference will be hosted at $place, $address (map).
Please, send me some options of flights and I will book one for you. I may need your passport number.
We will organize a (free) dinner for all speakers the night before, so you should arrive at least on $dinnerdate.
We will book a room for you for the following nights: $dates. The hotel is the $hotel **** .
Please remember to bring your laptop, charger and A/C adapter. In Spain we use Plug Type C, and you shouldn’t need any current transformer for your laptop.
There will be reinforced WiFi at the event and a separate segment for speakers, but please be prepared to deliver your presentation without access to the Internet, just in case. Remember to include any fonts or alternatively use a PDF version.
We are providing our speakers with a template that can be used for your talk, but feel free to use your own format if you have one.
Your talk may be recorded and shared later in our Youtube channel. We understand to be authorized to do so, unless you say otherwise.
Looking forward to hearing from you!

This is excellent, and a great blueprint to re-use. Well done, codemotion.

I have a similar way to tell conference organisers all I expect and give them the things they need with my conference organiser cheatsheet.

Nick ThomasZNC and Mozilla IRC

ZNC is great for having a persistent IRC connection, but it’s not so great when the IRC server or network has a blip. Then you can end up failing to rejoin with

nthomas (…) has joined #releng
nthomas has left … (Max SendQ exceeded)

over and over again.

The way to fix this is to limit the number of channels ZNC can connect to simultaneously. In the Web UI, you change ‘Max Joins’ preference to something like 5. In the config file use ‘MaxJoins = 5′ in a <User foo> block.

Gregory SzorcOn Monolithic Repositories

When companies or organizations deploy version control, they have to make many choices. One of them is how many repositories to create. Your choices are essentially a) a single, monolithic repository that holds everything b) many separate, smaller repositories that hold all the individual parts c) something in between.

The prevailing convention today (especially in the open source realm) is to create many separate and loosely coupled repositories, each repository mapping to a specific product or service. That does seem reasonable: if you were organizing files on your filesystem, you would group them by functionality or role (photos, music, documents, etc). And, version control tools are functionally filesystems. So it makes sense to draw repository boundaries at directory/role levels.

Further reinforcing the separate repository convention is the scaling behavior of our version control tools. Git, the popular tool in open source these days, doesn't scale well to very large repositories due to - among other things - not having narrow clones (fetching a subset of files). It scales well enough to the overwhelming majority of projects. But if you are a large organization generating lots of data (read: gigabytes of data over hundreds of thousands of files and commits) for version control, Git is unsuitable in its current form. Other tools (like Mercurial) don't currently fare that much better (although Mercurial has plans to tackle these scaling vectors).

Despite popular convention and even limitations in tools, companies like Google and Facebook opt to run large, monolithic repositories. Google runs Perforce. Facebook is on Mercurial, or at least is in the process of migrating to Mercurial.

Why do these companies run monolithic repositories? In Google's words:

We have a single large depot with almost all of Google's projects on it. This aids agile development and is much loved by our users, since it allows almost anyone to easily view almost any code, allows projects to share code, and allows engineers to move freely from project to project. Documentation and data is stored on the server as well as code.

So, monolithic repositories are all about moving fast and getting things done more efficiently. In other words, monolithic repositories increase developer productivity.

Furthermore, monolithic repositories are also more compatible with the ebb and flow of large organizations and large software projects. Components, features, products, and teams come and go, merge and split. The only constant is change. And if you are maintaining separate repositories that attempt to map to this ever-changing organizational topology, you are going to have a bad time. Either you'll be constantly copying, moving, merging, splitting, etc data and repositories. Or your repositories will be organized in a very non-logical and non-intuitive manner. That translates to overhead and lost productivity. I think that monolithic repositories handle the realities of large organizations much better. Big change or reorganization you want to reflect? You can make a single, atomic, history-preserving commit to move things around. I think that's much more manageable, especially when you consider the difficulty and annoyance of history-preserving changes across repositories.

Naysayers will decry monolithic repositories on principled and practical grounds.

The principled camp will say that separate repositories constitute a loosely coupled (dare I say service oriented) architecture that maps better to how software is consumed, assembled, and deployed and that erecting barriers in the form of separate repositories deliberately enforces this architecture. I agree. However, you can still maintain a loosely coupled architecture with monolithic repositories. The Subversion model of checking out a single tree from a larger repository proves this. Furthermore, I would say architecture decisions should be enforced by people (via code review, etc), not via version control repository topology. I believe this principled argument against monolithic repositories to be rather weak.

The principled camp living in the open source realm may also decry monolithic repositories as an affront to the spirit of open source. They would say that a monolithic repository creates unfairly strong ties to the organization that operates it and creates barriers to forking, etc. This may be true. But monolithic repositories don't intrinsically infringe on the basic software freedoms, organizations do. Therefore, I find this principled argument rather weak.

The practical camp will say that monolithic repositories just don't scale or aren't suitable for general audiences. These concerns are real.

Fully distributed version control systems (every commit on every machine) definitely don't scale past certain limits. Depending on your repository and user base, your scaling limits include disk space (repository data terabytes in size), bandwidth (repository data terabytes in size), filesystem (repository hundreds of thousands or millions of files), CPU and memory (operations on large repositories take too many system resources), and many heads/branches (tools like Git and Mercurial don't scale well to tens of thousands of heads/branches). These limitations with fully distributed version control are why distributed version control tools like Git and Mercurial support a partially-distributed mode that behaves more like your classical server-client model, like those employed by Subversion, Perforce, etc. Git supports shallow clone and sparse checkout. Mercurial supports shallow clone (via remotefilelog) and has planned support for narrow clone and sparse checkout in the next release or two. Of course, you can avoid the scaling limitations of distributed version control by employing a non-distributed tool, such as Subversion. Many companies continue to reach this conclusion today. However, users adapted to the distributed workflow would likely be up in arms (they would probably use tools like hg-subversion or git-svn to maintain their workflows). So, while scaling of version control can be a real concern, there are solutions and workarounds. However, they do involve falling back to a partially-distributed model.

Another concern with monolithic repositories is user access control. You inevitably have code or data that is more sensitive and want to limit who can change or even access it. Separate repositories seem to facilitate a simpler model: per-repository access control. With monolithic repositories, you have to worry about per-directory/subtree permissions, an increased risk of data leaking, etc. This concern is more real with distributed version control, as distributed data and access control aren't naturally compatible. But these issues can be resolved. And if the tooling supports it, there is only a semantic difference between managing access control between repositories versus components of a single repository.

When it comes to repository hosting conversions, I agree with Google and Facebook: I prefer monolithic repositories. When I am interacting with version control, I just want to get stuff done. I don't want to waste time dealing with multiple commands to manage multiple repositories. I don't want to waste time or expend cognitive load dealing with submodule, subrepository, or big files management. I don't want to waste time trying to find and reuse code, data, or documentation. I want everything at my fingertips, where it can be easily discovered, inspected, and used. Monolithic repositories facilitate these workflows more than separate repositories and make me more productive as a result.

Now, if only all the tools and processes we use and love would work with monolithic repositories...

Mozilla Release Management TeamFirefox 33 beta1 to beta2

  • 31 changesets
  • 72 files changed
  • 2046 insertions
  • 532 deletions

ExtensionOccurrences
js23
cpp12
html5
h5
mn4
jsm4
ini4
xhtml3
java3
jsx2
xml1
sh1
list1
css1

ModuleOccurrences
browser36
security10
mobile5
gfx4
content4
layout3
toolkit2
services2
dom2
netwerk1

List of changesets:

Jeff MuizelaarBug 1057716 - d3d11: Properly copy the background. r=bas, a=sledru - 9eb4dff42df0
Richard NewmanBug 993885 - Refactor SendTabActivity to avoid a race condition. r=mcomella, a=sledru - 764591e4e7f3
Lucas RochaBug 1050780 - Avoid disabled items in GeckoMenu's adapter. r=margaret, a=sledru - 7cf512b6b64c
Tim NguyenBug 891258 - Use Australis styling for the findbar buttons. r=Unfocused, a=sledru - 4815ff146c57
Dave TownsendBacking out Bug 891258 due to broken styling issues on OSX. r=backout - e7d6edff44d3
Cosmin MalutanBug 1062224 - [tps] Fix test_tabs.js for non-existent testcase pages. r=hskupin a=testonly DONTBUILD - 292839cc6594
David KeelerBug 1057128 - Add --clobber to generate_certs.sh, disabled by default (don't unnecessarily regenerate all certificates). r=rbarnes, a=sledru - 3f1e228fac54
David KeelerBug 1009161 - mozilla::pkix: Allow the Netscape certificate type extension if more standardized information is present. r=briansmith, a=sledru - 03029d16e697
JW WangBug 1034957 - Don't spin decode task queue waiting for audio frames since it hangs with gstreamer 1.0. r=cpearce, a=sledru - 46ffe60377d9
Neil RashbrookBug 1054289 - Scroll to the current ref, not the original one. r=smaug, a=sledru - 8865201cd18e
Neil RashbrookBug 1054289 - Add testcase. r=smaug, a=sledru - e47ff024eec1
Jan-Ivar BruaroeyBug 1060708 - Detect user and environment cameras on Android. r=gcp, r=blassey, r=snorp, a=sledru - fbc322c42d06
Mark FinkleBug 1063893 - Enable casting on beta and release. r=rnewman a=mfinkle - 32560f800b2e
Ed LeeBug 1062683 - Remove urls from new tab pings [r=adw a=lmandel] - c81810e5f3a5
Bas SchoutenBug 1040187 - Combine update regions properly when upload hasn't executed yet. r=nical, a=lmandel - 872fe12f9214
Matt WoodrowBug 1060114 - Fix partial surface uploading through BufferTextureClient. r=Bas, a=lmandel - 09d840603713
Chenxia LiuBug 1060678 - Notify Gecko when browser history is cleared from HistoryPanel. r=margaret, a=lmandel - 957e1ef7f769
Gijs KruitboschBug 1035536 - Add blank theme file for net error pages. r=Unfocused, a=lmandel - f9e4f36ba116
Ryan VanderMeulenBacked out changeset 09d840603713 (Bug 1060114) for bustage. - c3ecb4c952ec
Matt WoodrowBug 1060114 - Fix partial surface uploading through BufferTextureClient. r=Bas, a=lmandel - bca701646487
Chris KarlofBug 1056523 - Ensure sync credentials are reset during reauth flow. r=markh, a=lmandel - 8b409f2dfcb1
Steve WorkmanBug 1058099 - Cancel CacheStorageService::mPurgeTimer if it's still set during shutdown. r=mayhemer, a=lmandel - ede2300e8733
Michael ComellaBug 1046017 - Backed out changesets 1c213218173f & 8588817f7f86 (bugs 1017427 & 1006797). a=lmandel - 7984a6ceffb8
Randell JesupBug 1063971 - Allow SetRemoteDescription to omit callbacks again. r=jib, a=lmandel - 880228a5208a
Richard NewmanBug 1045085 - Remove main Product Announcements code. r=mcomella, a=lmandel - 776ddfd41f21
Ryan VanderMeulenBacked out changeset 776ddfd41f21 (Bug 1045085) for Android bustage. - 70930f30da0e
Benjamin SmedbergBug 1012924 - Experiments should cancel their XMLHttpRequest on shutdown and should also set a reasonable timeout on them. r=gfritzsche, a=lmandel - db5539e42eb5
Mark BannerBug 1022594 - Part 1: Change Loop's incoming call handling to get the call details before displaying the incoming call UI. r=nperriault, a=lmandel - e0ad01b2e26e
Mark BannerBug 1022594 - Part 2: Desktop client needs ability to decline an incoming call - set up a basic websocket protocol and use for both desktop and standalone UI. r=dmose, a=lmandel - 062929c9ff5d
Mark BannerBug 1045643 - Part 1: Notify the Loop server when the desktop client accepts the call, so that it can update the call status. r=nperriault, a=lmandel - be539410c211
Mark BannerBug 1045643 - Part 2: Notify the Loop server when the client has local media up and remote media being received, so that it can update the call connection status. r=nperriault, a=lmandel - d820ef3b256d

Byron Joneshappy bmo push day!

the following changes have been pushed to bugzilla.mozilla.org:

  • [913647] Deploy YUI 3.17.2 for BMO
  • [1054138] add the ability to filter on “fields containing the string”
  • [1062344] contrib/reorg-tools/sync* do not clear memcached
  • [1051058] Auto-CC Erica Choe into Finance Review and Master Kick-Off Bugs

discuss these changes on mozilla.tools.bmo.

 

the new bugmail filtering ability allows you to filter on specific flags:

bugmail filtering with substrings

these two rules will prevent bugzilla from emaling you the changes to the “qa whiteboard” field or the “qe-verify” flag for bugs where you aren’t the assignee.


Filed under: bmo, mozilla

Matt BrubeckLet's build a browser engine! Part 5: Boxes

This is the latest in a series of articles about writing a simple HTML rendering engine:

This article will begin the layout module, which takes the style tree and translates it into a bunch of rectangles in a two-dimensional space. This is a big module, so I’m going to split it into several articles. Also, some of the code I share in this article may need to change as I write the code for the later parts.

The layout module’s input is the style tree from Part 4, and its output is yet another tree, the layout tree. This takes us one step further in our mini rendering pipeline:

I’ll start by talking about the basic HTML/CSS layout model. If you’ve ever learned to develop web pages you might be familiar with this already—but it may look a bit different from the implementer’s point of view.

The Box Model

Layout is all about boxes. A box is a rectangular section of a web page. It has a width, a height, and a position on the page. This rectangle is called the content area because it’s where the box’s content is drawn. The content may be text, image, video, or other boxes.

A box may also have padding, borders, and margins surrounding its content area. The CSS spec has a diagram showing how all these layers fit together.

Robinson stores a box’s content area and surrounding areas in the following structure. [Rust note: f32 is a 32-bit floating point type.]

// CSS box model. All sizes are in px.
struct Dimensions {
    // Top left corner of the content area, relative to the document origin:
    x: f32,
    y: f32,

    // Content area size:
    width: f32,
    height: f32,

    // Surrounding edges:
    padding: EdgeSizes,
    border: EdgeSizes,
    margin: EdgeSizes,
}

struct EdgeSizes {
    left: f32,
    right: f32,
    top: f32,
    bottom: f32,
}

Block and Inline Layout

Note: This section contains diagrams that won't make sense if you are reading them without the associated visual styles. If you are reading this in a feed reader, try opening the original page in a regular browser tab. I also included text descriptions for those of you using screen readers or other assistive technologies.

The CSS display property determines which type of box an element generates. CSS defines several box types, each with its own layout rules. I’m only going to talk about two of them: block and inline.

I’ll use this bit of pseudo-HTML to illustrate the difference:

<container>
  <a></a>
  <b></b>
  <c></c>
  <d></d>
</container>

Block boxes are placed vertically within their container, from top to bottom.

a, b, c, d { display: block; }

Description: The diagram below shows four rectangles in a vertical stack.

a
b
c
d

Inline boxes are placed horizontally within their container, from left to right. If they reach the right edge of the container, they will wrap around and continue on a new line below.

a, b, c, d { display: inline; }

Description: The diagram below shows boxes `a`, `b`, and `c` in a horizontal line from left to right, and box `d` in the next line.

a
b
c
d

Each box must contain only block children, or only inline children. When an DOM element contains a mix of block and inline children, the layout engine inserts anonymous boxes to separate the two types. (These boxes are “anonymous” because they aren’t associated with nodes in the DOM tree.)

In this example, the inline boxes b and c are surrounded by an anonymous block box, shown in pink:

a    { display: block; }
b, c { display: inline; }
d    { display: block; }

Description: The diagram below shows three boxes in a vertical stack. The first is labeled `a`; the second contains two boxes in a horizonal row labeled `b` and `c`; the third box in the stack is labeled `d`.

a
b
c
d

Note that content grows vertically by default. That is, adding children to a container generally makes it taller, not wider. Another way to say this is that, by default, the width of a block or line depends on its container’s width, while the height of a container depends on its children’s heights.

This gets more complicated if you override the default values for properties like width and height, and way more complicated if you want to support features like vertical writing.

The Layout Tree

The layout tree is a collection of boxes. A box has dimensions, and it may contain child boxes.

struct LayoutBox<'a> {
    dimensions: Dimensions,
    box_type: BoxType<'a>,
    children: Vec<LayoutBox<'a>>,
}

A box can be a block node, an inline node, or an anonymous block box. (This will need to change when I implement text layout, because line wrapping can cause a single inline node to split into multiple boxes. But it will do for now.)

enum BoxType<'a> {
    BlockNode(&'a StyledNode<'a>),
    InlineNode(&'a StyledNode<'a>),
    AnonymousBlock,
}

To build the layout tree, we need to look at the display property for each DOM node. I added some code to the style module to get the display value for a node. If there’s no specified value it returns the initial value, 'inline'.

enum Display {
    Inline,
    Block,
    DisplayNone,
}

impl StyledNode {
    /// Return the specified value of a property if it exists, otherwise `None`.
    fn value(&self, name: &str) -> Option<Value> {
        self.specified_values.find_equiv(&name).map(|v| v.clone())
    }

    /// The value of the `display` property (defaults to inline).
    fn display(&self) -> Display {
        match self.value("display") {
            Some(Keyword(s)) => match s.as_slice() {
                "block" => Block,
                "none" => DisplayNone,
                _ => Inline
            },
            _ => Inline
        }
    }
}

Now we can walk through the style tree, build a LayoutBox for each node, and then insert boxes for the node’s children. If a node’s display property is set to 'none' then it is not included in the layout tree.

/// Build the tree of LayoutBoxes, but don't perform any layout calculations yet.
fn build_layout_tree<'a>(style_node: &'a StyledNode<'a>) -> LayoutBox<'a> {
    // Create the root box.
    let mut root = LayoutBox::new(match style_node.display() {
        Block => BlockNode(style_node),
        Inline => InlineNode(style_node),
        DisplayNone => fail!("Root node has display: none.")
    });

    // Create the descendant boxes.
    for child in style_node.children.iter() {
        match child.display() {
            Block => root.children.push(build_layout_tree(child)),
            Inline => root.get_inline_container().children.push(build_layout_tree(child)),
            DisplayNone => {} // Skip nodes with `display: none;`
        }
    }
    return root;
}

impl LayoutBox {
    /// Constructor function
    fn new(box_type: BoxType) -> LayoutBox {
        LayoutBox {
            box_type: box_type,
            dimensions: Default::default(), // initially set all fields to 0.0
            children: Vec::new(),
        }
    }
}

If a block node contains an inline child, create an anonymous block box to contain it. If there are several inline children in a row, put them all in the same anonymous container.

impl LayoutBox {
    /// Where a new inline child should go.
    fn get_inline_container(&mut self) -> &mut LayoutBox {
        match self.box_type {
            InlineNode(_) | AnonymousBlock => self,
            BlockNode(_) => {
                // If we've just generated an anonymous block box, keep using it.
                // Otherwise, create a new one.
                match self.children.last() {
                    Some(&LayoutBox { box_type: AnonymousBlock,..}) => {}
                    _ => self.children.push(LayoutBox::new(AnonymousBlock))
                }
                self.children.mut_last().unwrap()
            }
        }
    }
}

This is intentionally simplified in a number of ways from the standard CSS box generation algorithm. For example, it doesn’t handle the case where an inline box contains a block-level child. Also, it generates an unnecessary anonymous box if a block-level node has only inline children.

To Be Continued…

Whew, that took longer than I expected. I think I’ll stop here for now, but don’t worry: Part 6 is coming soon, and will cover block-level layout.

Once block layout is finished, we could jump ahead to the next stage of the pipeline: painting! I think I might do that, because then we can finally see the rendering engine’s output as pretty pictures instead of just numbers.

However, the pictures will just be a bunch of colored rectangles, unless we finish the layout module by implementing inline layout and text layout. If I don’t implement those before moving on to painting, I hope to come back to them afterward.

Jeff WaldenQuote of the day

Snipped from irrelevant context:

<jorendorff> In this case I see nearby code asserting that IsCompiled() is true, so I think I have it right

Assertions do more than point out mistakes in code. They also document that code’s intended behavior, permitting faster iteration and modification to that code by future users. Assertions are often more valuable as documentation, than they are as a means to detect bugs. (Although not always. *eyes fuzzers beadily*)

So don’t just assert the tricky requirements: assert the more-obvious ones, too. You may save the next person changing the code (and the person reviewing it, who could be you!) a lot of time.

David BoswellCreating community contribution challenges

There is something magical about how anyone anywhere can contribute to Mozilla—people show up and help you with something you’re doing or offer you something completely new and unexpected.

The Code Rush documentary has a great example of this from the time when the Mozilla project first launched. Netscape opened it’s code to the world in the hope that people would contribute, but there was no guarantee that anyone would help.

One of the first signs they had that this was working was when Stuart Parmenter started contributing by rewriting a key part of the code and this accelerated development work by months. (This is about 27 minutes into the documentary.)

code_rush_pavlov_scene

It is hard to plan and schedule around magic though. This year we’ve been building up a participation system that will help make contributions more reliable and predictable, so that teams can plan and schedule around leveraging the Mozilla community.

Pathways, tools and education are part of that system. Something else we’re trying is contribution challenges. These will identify unmet needs where scale and asynchronous activities can provide impact in the short-term and where there is strong interest within the volunteer community.

The challenges will also specify the when, where, who and how of the idea, so that we can intentionally design for participation at the beginning and have a prepared way that we’re rallying people to take action.

For next steps, leadership of the Mozilla Reps program is meeting in Berlin from September 12-14 and they’ll be working on this concept as well as on some specific challenge ideas. There will be more to share after that.

RemoCamp-berlin

If you’re interested in helping with this and want to get involved, take a look at the contribution challenges etherpad for more background and a list of challenge ideas. Then join the community building mailing list and share your thoughts, comments and questions.


Nathan Froydxpcom and move constructors

Benjamin Smedberg recently announced that he was handing over XPCOM module ownership duties to me.  XPCOM contains basic data structures used throughout the Mozilla codebase, so changes to its code can have wide-ranging effects.  I’m honored to have been given responsibility for a core piece of the Gecko platform.

One issue that’s come up recently and I’m sure will continue to come up is changing XPCOM data structures to support two new C++11 features, rvalue references and their killer app, move constructors.  If you aren’t familiar with C++11’s new rvalue references feature, I highly recommend C++ Rvalue References Explained.  Move constructors are already being put to good use elsewhere in the codebase, notably mozilla::UniquePtr, which can be used to replace XPCOM’s nsAutoPtr and nsAutoRef (bug 1055035).  And some XPCOM data structures have received the move constructor treatment, notably nsRefPtr (bug 980753) and nsTArray (bug 982212).

A recent discussion and the associated bug, however, decided that the core referenced-counted smart pointer class in XPCOM, nsCOMPtr, shouldn’t support move constructors.  While move constructors could have replaced the already_AddRefed usage associated with nsCOMPtr, such as:

already_AddRefed<nsIMyInterface>
NS_DoSomething(...)
{
  nsCOMPtr<nsIMyInterface> interface = ...;
  // do some initialization stuff
  return interface.forget();
}

with the slightly shorter:

nsCOMPtr<nsIMyInterface>
NS_DoSomething(...)
{
  nsCOMPtr<nsIMyInterface> interface = ...;
  // do some initialization stuff
  return interface;
}

There were two primary arguments against move constructor support.  The first argument was that the explicitness of having to call .forget() on an nsCOMPtr (along with the explicitness of the already_AddRefed type), rather than returning it, is valuable for the code author, the patch reviewer, and subsequent readers of the code.  When dealing with ownership issues in C++, it pays to be more explicit, rather than less.  The second argument was that due to the implicit conversion of nsCOMPtr<T> to a bare T* pointer (a common pattern in smart pointer classes), returning nsCOMPtr<T> from functions makes it potentially easy to write buggy code:

// What really happens in the below piece of code is something like:
//
// nsIMyInterface* p;
// {
//   nsCOMPtr<nsIMyInterface> tmp(NS_DoSomething(...));
//   p = tmp.get();
// }
//
// which is bad if NS_DoSomething is returning the only ref to the object.
// p now points to deleted memory, which is a security risk.
nsIMyInterface* p = NS_DoSomething(...);

(I should note that we can return nsCOMPtr<T> from functions today, and in most cases, thanks to compiler optimizations, it will be as efficient as returning already_AddRefed.  But Gecko culture is such that a function returning nsCOMPtr<T> would be quite unusual, and therefore unlikely to pass code review.)

The changes to add move constructors to nsRefPtr and nsTArray?  They were reviewed by me.  And the nixing of move constructors for nsCOMPtr?  That was also done by me (with a lot of feedback from other people).

I accept the charge of inconsistency.  However, I offer the following defense.  In the case of nsTArray, there are no ownership issues like there are with nsCOMPtr: you either own the array, or you don’t, so many of the issues raised about nsCOMPtr don’t apply in that case.

For the case of nsRefPtr, it is true that I didn’t seek out as much input from other people before approving the patch.  But the nsRefPtr patch was also done without the explicit goal of removing already_AddRefed from the code base, which made it both smaller in scope and more palatable.  Also, my hunch is that nsRefPtr is used somewhat less than nsCOMPtr (although this may be changing somewhat given other improvements in the codebase, like WebIDL), and so it provides an interesting testbed for whether move constructors and/or less explicit transfers of ownership are as much of a problem as argued above.

Henrik SkupinFirefox Automation report – week 29/30 2014

In this post you can find an overview about the work happened in the Firefox Automation team during week 29 and 30.

Highlights

During week 29 it was time again to merge the mozmill-tests branches to support the upcoming release of Firefox 31.0. All necessary work has been handled on bug 1036881, which also included the creation of the new esr31 branch. Accordingly we also had to update our mozmill-ci system, and got the support landed on production.

The RelEng team asked us if we could help in setting up Mozmill update tests for testing the new update server aka Balrog. Henrik investigated the necessary tasks, and implemented the override-update-url feature in our tests and the mozmill-automation update script. Finally he was able to release mozmill-automation 2.6.0.2 two hours before heading out for 2 weeks of vacation. That means Mozmill CI could be used to test updates for the new update server.

Individual Updates

For more granular updates of each individual team member please visit our weekly team etherpad for week 29 and week 30.

Meeting Details

If you are interested in further details and discussions you might also want to have a look at the meeting agenda, the video recording, and notes from the Firefox Automation meetings of week 29 and week 30.

Benoit GirardB2G Performance Polish: Unlock Animation (Part 2)

Our destination

Our starting point is a 200ms unlock delay, and an uniform ~25 FPS animation. Our aim should be a 0ms unlock delay and a uniform 60 FPS (or whatever vsync is). The former we will minimize as much as we can but the latter is entirely possible.

Let’s talk about how we should design a lock screen animation in the optimal case. When we go and apply it in practice we often hit some requirements and constraint that make it impossible to behave like we want but let’s ignore that for a second and discuss we want to get to.

In the ideal case we would have the lockscreen rendered offscreen to a GPU texture(s) set. We would have the background app ready in another GPU texture(s) set. These are ‘Layers’. We place the background app behind the lockscreen. When the transition begins we notify the compositor to start fading out the lockscreen. Keep them around costs memory but if we keep the right thing around we can reduce or eliminate repaints entirely.

Our memory requirement is what’s required by the background app + about one fullscreen layer required for the lockscreen. This should be fine for even low end of B2G phones. Our overdraw should be about 200%-300%, again enough for mobile GPUs to keep up at 60 FPS/vsync.

Ideal Lockscreen Layer Tree

Now let’s look at what we hope our timeline for our Main Thread and our Compositor Thread to look like:

Ideal Unlock Timeline

We want to use Off-Main-Thread-Animation to perform the fade entirely on the Compositor. This will be initiated on the main thread and will require a style flush to set a CSS transform transition. If done right we don’t expect to have to reflow or repaint any part of the page if we properly built the layer like shown in the first figure. Note that the style flush will contribute to the unlock delay (and so will the first composite time as incorrectly shown in the diagram). If we can keep that style flush + first composite in under say 50ms and each composite in 16ms or less then we should have a smooth unlock animation.

Next up let’s look at what’s actually happening in the unlock animation in practice…


Adam LoftingSomething special within ‘Hack the snippet’

Here are a couple of notes about ‘Hack the snippet‘ that I wanted to make sure got documented.

  1. It significantly changed peoples’ predisposition to Webmaker before they arrived on the site
  2. Its ‘post-interaction’ click-through-rate was equivalent to most one-click snippets

Behind these observations, something special was happening in ‘Hack the snippet’. I can’t tell you exactly what it was that had the end-effect, but it’s worth remembering the effect.

1. It ‘warmed people up’ to Webmaker

  • The ‘Hack the snippet’ snippet
    • was shown to the same audience (Firefox users) as eight other snippet variations we ran during the campaign
    • had the same % of users click through to the landing page
    • had the same on-site experience on webmaker.org as all the other snippet variations we tested (the same landing page, sign-up ask etc)
  • But when people who had interacted with ‘Hack the snippet’ landed on the website, they were more than three times as likely to signup for a webmaker account

Same audience, same engagement rate, same ask… but triple the conversion rate (most regular snippet traffic converted ~2%, ‘Hack the snippet’ traffic converted ~7%).

Something within that experience (and likely the overall quality of it) makes the Webmaker proposition more appealing to people who ‘hacked the snippet’. It could be one of many things: the simplicity, the guided learning, the feeling of power from editing the Firefox start page, the particular phrasing of the copy or many of the subtle design decisions. But whatever it was, it worked.

We need to keep looking for ways to recreate this.

Not everything we do going forwards needs to be a ‘Hack the snippet’ snippet (you can see how much time and effort went into that in the bug).

But when we think about these new-user experiences, we have a benchmark to compare things too. We know how much impact these things can have when all the parts align.

2. The ‘post-interaction’ CTR was as good as most one-click snippets

This is a quicker note:

  • Despite the steps involved in completing the ‘Hack the snippet’ on page activity, the same total number of people clicked through when compared to a standard ‘one-click’ snippet.
  • We got the same % of the audience to engage with a learning activity and then click through to the webmaker site, as we usually get just giving them a link directly to Webmaker
    • This defies most “best practice” about minimizing number of clicks

Again, this doesn’t give us an immediate thing we can repeat, but it gives us a benchmark to build on.

Lucas RochaIntroducing dspec

With all the recent focus on baseline grids, keylines, and spacing markers from Android’s material design, I found myself wondering how I could make it easier to check the correctness of my Android UI implementation against the intended spec.

Wouldn’t it be nice if you could easily provide the spec values as input and get it rendered on top of your UI for comparison? Enter dspec, a super simple way to define UI specs that can be rendered on top of Android UIs.

Design specs can be defined either programmatically through a simple API or via JSON files. Specs can define various aspects of the baseline grid, keylines, and spacing markers such as visibility, offset, size, color, etc.

Baseline grid, keylines, and spacing markers in action.

Baseline grid, keylines, and spacing markers in action.

Given the responsive nature of Android UIs, the keylines and spacing markers are positioned in relation to predefined reference points (e.g. left, right, vertical center, etc) instead of absolute offsets.

The JSON files are Android resources which means you can easily adapt the spec according to different form factors e.g. different specs for phones and tablets. The JSON specs provide a simple way for designers to communicate their intent in a computer-readable way.

You can integrate a DesignSpec with your custom views by drawing it in your View‘s onDraw(Canvas) method. But the simplest way to draw a spec on top of a view is to enclose it in a DesignSpecFrameLayout—which can take an designSpec XML attribute pointing to the spec resource. For example:

<DesignSpecFrameLayout
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    app:designSpec="@raw/my_spec">
    ...
</DesignSpecFrameLayout>

I can’t wait to start using dspec in some of the new UI work we’re doing Firefox for Android now. I hope you find it useful too. The code is available on Github. As usual, testing and fixes are very welcome. Enjoy!

Doug BelshawWeb Literacy: More than just coding; an enabling education for our times [EdTech Digest]

Web Literacy | edtechdigest.com 2014-09-08 14-03-43

Last week, my colleague Lainie Decoursy got in touch wondering if I could write a piece about web literacy. It was a pretty tight turnaround, but given pretty much all I think about during my working hours is web literacy, it wasn’t too much of a big ask!

The result is a piece in EdTech Digest entitled Web Literacy: More than just coding; an enabling education for our times. It’s an overview of Mozilla’s work around Webmaker and, although most of the words are mine, I have to credit my colleagues for some useful edits.

Click here to read the post

I’ve closed comments here to encourage you to add your thoughts on the original post.

Robert O'Callahanrr 2.0 Released

Thanks to the hard work of our contributors, rr 2.0 has been released. It has many improvements over our 1.0 release:

  • gdb's checkpoint, restart and delete checkpoint commands are supported.
    These are implemented using new infrastructure in rr 2.0 for fast cloning of replay sessions.
  • You can now run debuggee functions from gdb during replay.
    This is a big feature for rr, since normally a record-and-replay debugger will only replay what happened during recording --- and of course, function calls from gdb did not happen during recording. So under the hood, rr 2.0 introduces "diversion sessions", which run arbitrary code instead of following a replay. When you run a debuggee function from gdb, we clone the current replay session to a diversion session, run your requested function, then destroy the diversion and resume the replay.
  • Issues involving Haswell have been fixed. rr now runs reliably on Intel CPU families from Westmere to Haswell.
  • Support for running rr in a VM has been improved. Due to a VMWare bug, rr is not as reliable in VMWare guests as in other configurations, but in practice it still works well.
  • Trace compression has been implemented, with compression ratios of 5-40x depending on workload, dramatically reducing rr's storage and I/O usage.
  • Many many bugs have been fixed to improve reliability and enable rr to handle more diverse workloads.

All the features normally available from gdb now work with rr, making this an important milestone.

The ability to run debuggee functions makes it much easier to use rr to debug Firefox. For example you can dump DOM, frame and layer trees at any point during replay. You can debug Javascript to some extent by calling JS engine helpers such as DumpJSStack(). Some Mozilla developers have successfully used rr to fix real bugs. I use it for most of my Gecko debugging --- the first of my research projects that I've actually wanted to use :-).

Stephen Kitt has packaged rr for Debian.

Considerable progress has been made towards x86-64 support, but it's not ready yet. We expect x86-64 support to be the next milestone.

I recorded a screencast showing a quick demo of rr on Firefox:

Robert O'CallahanVMWare CPUID Conditional Branch Performance Counter Bug

This post will be uninteresting to almost everyone. I'm putting it out as a matter of record; maybe someone will find it useful.

While getting rr working in VMWare guests, we ran into a tricky little bug. Typical usage of CPUID. e.g. to detect SSE2 support, looks like this pseudocode:

CPUID(0); // get maximum supported CPUID subfunction M
if (S <= M) {
CPUID(S); // execute subfunction S
}
Thus, CPUID calls often occur in pairs with a conditional branch between them. The bug is that in a VMWare guest, when we count the number of conditional branches executed, the conditional branch between those two CPUIDs is usually (but not always) omitted from the count. We assume this is a VMWare bug because it does not happen on the same hardware outside of a VM, and it does not happen in a KVM-based VM.

Experiments show that some code sequences trigger the bug and other equivalent sequences don't. Single-stepping and other kinds of interference suppress the bug. My best guess is that VMWare optimizes some forms of the above code, perhaps to reduce the number of VM exits, and in so doing skips execution of the conditional branch, without taking into account that this might perturb performance counter values. Admittedly, it's unusual for software to rely on precise performance counter values the way rr does.

This sucks for rr because rr relies on these counts being accurate. We sometimes find that replay diverges because one of these conditional branches was not counted during recording but is counted during replay. (The other way around is possible too, but less frequently observed.) We have some heuristics and workarounds, but it's difficult to fully work around without adding significant complexity and/or slowdown.

The bug is easily reproduced: just use rr to record and replay anything simple. When replaying, rr automatically detects the presence of the bug and prints a warning on the console:

rr: Warning: You appear to be running in a VMWare guest with a bug
where a conditional branch instruction between two CPUID instructions
sometimes fails to be counted by the conditional branch performance
counter. Partial workarounds have been enabled but replay may diverge.
Consider running rr not in a VMWare guest.

Steps forward:

  • Find a way to report this bug to VMWare.
  • Linux hosts can run rr in KVM-based VMs or directly on the host. Xen VMs might work too.
  • Parallels apparently supports PMU virtualization now; if Parallels doesn't have this bug, it might be the best way to run rr on a Mac or Windows host.
  • We can add a "careful mode" that would probably almost always replay successfully, albeit with additional overhead.
  • The bug is less likely to show up once rr supports x86-64. At least in Firefox, CPUID instructions are most commonly used to detect the presence of SSE2, which is unnecessary on x86-64.
  • In practice, recording Firefox in VMWare generally works well without hitting this bug, so maybe we don't need to invest a lot in fixing it.

Daniel StenbergVideo perhaps?

I decided to try to do a short video about my current work right now and make it available for you all. I try to keep it short (5-7 minutes) and I’m certainly no pro at it, but I will try to make a weekly one for a while and see if it gets any fun. I’m going to read your comments and responses to this very eagerly and that will help me decide how I will proceed on this experiment.

Enjoy.

Jordan LundThis Week In Releng - Sept 1st, 2014

Major Highlights:

Completed work (resolution is 'FIXED'):


In progress work (unresolved and not assigned to nobody):