Planet Mozilla

Seif Lotfy: Globaleaks 0.2 Alpha

Seif Lotfy — Sat, 18 May 2013 21:25:46 +0000

Globaleaks 0.2 Alpha is out.

Globaleaks is an open source project aimed at creating a worldwide, anonymous, censorship-resistant, distributed whistle-blowing platform. It enables organizations interested in running whistle-blowing initiatives to setup their own safe zone, where whistle-blowers and recipients can exchange data.

2 Years ago I helped out with the development of Globaleaks 0.1. And although I am not active anymore, I really support the initiative behind it. Now with the HERMES Center for Transparency and Digital Human Rights backing it up, it has grown a lot and shaped up to be a very organized and thought through project.

TL;DR:

Full rewrite
More flexible and extensible
Linux ready-made system and network hardened installation
Written in python using twisted
New Frontend

Try it out:

Try out the demo. It is pretty straight forward.

Help out:

As young project, Globaleaks can use some help fixing bugs. Just head to the wiki and read through it. It is pretty straight forward, and explains the modules, security concepts and set up instructions.

Globaleaks already has Debian and Ubuntu ready packages. An easy way to help out is to set up a PPA for us on Launchpad.

Get in touch:

You can contact the Globaleaks team at info () globaleaks org or on IRC on #globaleaks at irc.oftc.net

Here are some screenshots of the new frontend

Congratulations you are using Tor

Receiver selection page

The submission receipt

Configuring a receiver

Configuring a context

Amir Aharoni: Always define the language and the direction of your HTML documents, part 02: Backwards English

aharoni — Sat, 18 May 2013 12:14:26 +0000

In part 01 of these series, I showed why is it important to always define the language and the direction of all HTML content and not rely on the defaults: The content may get embedded in a document with different direction and be displayed incorrectly.

This issue is laughably easy to avoid: If you are writing the content, you are supposed to know in what language it is written, so if it’s English, just write <html lang=”en” dir=”ltr”> even though these seem to be the defaults. Nineteen or so characters that ensure your content is readable and not displayed backwards. Please do it always and tell all your friends to do it.

The problem is that you don’t only have to explicitly set the language and the direction, but, as silly as it sounds, you have to set them correctly, too. A more subtle, but nevertheless quite frequent and disruptive bug is displaying presumably, but not actually, translated content in a different direction. This happens quite frequently when a website supports the browser language detection feature, known as Accept-Language:

The web server sees that the browser requests content in Hebrew.
The web server sends a response with <html lang=”he” dir=”rtl”>, but because the website is not actually translated, the text is shown in the fallback language, which is usually English.
The user sees the content just like this numbered list, which I intentionally set to dir=”rtl”: with the numbers and the punctuation on the wrong side, and possibly invisible, because English is not a right-to-left language.

Of course, it can go even worse. Arrows can point the wrong way and buttons and images can overlap and hide each other, rendering the page not just hard to read, but totally unusable.

This bug is also an example of the Software Localization Paradox: It manifests itself when Accept-Language is not English, but most developers install English operating systems and don’t bother to change the preferred language settings in the browser, so they never see how this bug manifests itself. The site developers don’t bother to test for it either.

The solution, of course, is to set a different language and direction only if the site is actually translated, and not to pretend that it is if it’s not translated.

Here are two examples of such brokenness. Both sites are important and useful, but hard to use for people whose Accept-Language is Hebrew, Persian or Arabic.

Here’s how the Mozilla Developer Network website looks in fake Hebrew:

Mozilla Developer Network website, in English, but right-to-left

Notice how the full stops are on the left end and how the text overlaps the images in the tiles on the right-hand side. This is how it is supposed to look, more or less:

Mozilla Developer Network home page in English, left-to-right

I manually changed dir=”rtl” to dir=”ltr” using the element inspector from Firefox’s developer tools and I also had to tweak a CSS class to move the “mozilla” tab at the top.

The above troubles are reported as bug 816443 – lang and dir attributes must be used only if the page is actually translated.

After showing an example of a web development bug from a site for, ahem, web developers, here is an even funnier example: The home page Unicode’s CLDR. That’s right: Unicode’s own website shows text with incorrect direction:

The Unicode CLDR website, in English but right-to-left

The only words translated here are “Contents” (תוכן) and “Search this site” (חיפוש באתר זה), which is not so useful. The rest is shown in English, and the direction is broken: Notice the strange alignment of the content and the schedule table. A few months ago that table was so broken that its content wasn’t visible at all, but that was probably patched.

Here’s how it is supposed to look:

The CLDR home page in English, appropriately left-to-right

I tried reporting the CLDR home page direction bug, but it was closed as “out-of-scope”: The CLDR developers say that the Google Sites infrastructure is to blame. This is frustrating, because as far as I know Google Sites doesn’t have a proper bug reporting system and all I can do is write a question about that direction problem in the Google Sites forum and hope that somebody notices it or poke my Googler friends.

One thing that I will not do is switch my Accept-Language to English. Whenever I can, I don’t just want to see the website correctly, but to try to help my neighbor: see the possible problems that can affect other users who use different language. Somebody has to break the Software Localization Paradox.

Filed under: Firefox, Free Software, localization

Will Kahn-Greene: Proposal: LDAP password resets as a unit of measure

Sat, 18 May 2013 04:00:00 +0000

Backstory

Every 3 months, we at Mozilla have to reset our LDAP passwords. The system helpfully sends the first reminder 2 weeks before your password expires, then the second reminder 1 week before your password expires and the last reminder 2 days before your password expires.

Sometimes time passes by faster than you know and you end up with a Locked out of LDAP account.

The 3 month LDAP password reset is such a large part of our lives that I propose it become a standard unit of measure for elapsed time.

Usage

Used in casual conversation:

Pat: Hi!

Jordan: Hi!

Pat: I haven't seen you before. How long have you been at Mozilla?

Jordan: I've been here for 6 LDAP password resets.

Pat: Oh, weird. I've been here for 7. Good to meet you! Would you like a banana?

Jordan: Would I ever!

Used in casual conversation on IRC:

<patbot> anyone use less?
<corycory> i only use sass. it's the best.
* riledupriley has quit (Quit: riledupriley)
<patbot> :(
<hugbot> (patbot)
* r1cky has joined #casualconversationexample
<r1cky> morning!
<nigelb> r1cky: hai!
<nigelb> Ah, it's nearly mfbt.
<mtjordan> sure. been using it for 3 ldap password resets.
<mtjordan> patbot: why do you ask?

Used in Bugzilla comments:

Jordan [:jordan]  1 day ago       Comment 0 [reply] [-]

Readonly mode causes the site to ISE.

Pat [:pat]  1 day ago             Comment 1 [reply] [-]

I looked into it. Turns out we haven't used readonly
mode in at least 4 LDAP password resets.

I think we just need to add a fake authentication
module. Easy peasy.

Used when joining a new group:

From: Pat
To: some-group@mozilla.org
Subject: Welcome Jordan to some-group!

Hi all!

I'd like to welcome Jordan to some-group! Jordan brings
expertise that is invaluable. I'm excited! Yay!

Jordan: Tell us about yourself!

Pat

From: Jordan
To: some-group@mozilla.org
Subject: Re: Welcome Jordan to some-group!

Hi!

I'm excited to join some-group! Hopefully I bring something
useful to the table.

I've been at Mozilla for 7 LDAP password resets, I like
top-posting and I make a mean cold brew coffee.

Looking forward to my first meeting!

Jordan

On Blah blah blah at blah blah blah, Pat wrote:
> Hi all!
>
> I'd like to welcome Jordan to some-group! Jordan brings
> expertise that is invaluable. I'm excited! Yay!
>
> Jordan: Tell us about yourself!
>
> Pat

Used in an email to everyone@ about departing:

Dear everyone!

It is with sadness that I tell you I'm leaving as of next
Friday. As you know, I've been with Mozilla for 32 LDAP
password resets and frankly, I'm totally out of usable
Sherlock Holmes story titles, so I'm off to new challenges.

I will miss you all.

[Comments]

Tantek Çelik: #UX: "Learn more" Links in Warning Boxes Should Go To A Page With These Three Things

Fri, 17 May 2013 22:13:00 +0000

Sometimes web pages display brief warning boxes at the top with "learn more" links. The learn more link in a specific warning box should go to a page specifically about that warning with, in rough order:

screenshot of warning box
quoted full text of the warning (for searchability / search engine discovery)
detailed text answering:
- how could have the issue occurred?
- what should the user do to resolve the issue?
- how can the user avoid the issue in the future?

E.g. the "Learn more ›" link in the yellow warning box in this screenshot:

links to: https://support.twitter.com/articles/82050-i-m-having-trouble-confirming-my-email which:

Neither has screenshot nor text of warning
Covers several topics unrelated to the warning
Does not answer the above questions

And could be improved by linking to a specific page about this particular warning, containing the above points 1-3, and answering all three questions in point 3.

J. Paul Reed: Eulogy for a Founding Father, revisited

preed — Fri, 17 May 2013 20:47:13 +0000

In response to my post earlier this week on Tinderbox’s end-of-life, reader Carsten Mattner asked:

Reading [your post], I couldn’t figure out what replaced Tinderbox for the Mozilla builds. What feeds tbpl? Does Mozilla not use Tinderbox to build continuously?

When I left Mozilla in 2007, there was a Release Engineering project in progress to actively replace Tinderbox (Client) with buildbot. So in short, no, Mozilla does not use Tinderbox Client to drive its continuous integration builds, and hasn’t for some time.

Do they still use buildbot today?

I didn’t know the answer to that question, so I tracked down Coop on IRC, who graciously gave me a few minutes of his time to answer exactly that.

He said:

Mozilla currently uses “95% buildbot, with 5% Jenkins for random small projects”
There are multiple buildbot masters that drive the buildbot clients
Unlike the out-of-the-box buildbot master setup, the masters query a job scheduling database instead of monitoring source control for changes themselves; they then report their results to a database, which tbpl (and other services) use to generate their reports/dashboards; the buildbot master waterfall pages aren’t accessible to the external world (which makes sense, because they include unsecured administrative functionality¹)
There are about 60 masters right now, but Coop said “number keeps growing though, so we need to rethink the whole solution”

So there’s your answer, Carsten!

_______________
¹ A long standing criticism of mine, among others ↵

Selena Deckelmann: Migrations with Alembic: a lightspeed tour

selena — Fri, 17 May 2013 20:44:46 +0000

I’ve got a Beer & Tell to give about alembic. Alembic is a migration tool that works with SQLAlchemy. I’m using it for database migrations with PostgreSQL.

So, here’s what I want to say today:

Written by SQLAlchemy wiz Mike Bayer
Here’s the tutorial. Socorro is now using alembic in production with SQLAlchemy 0.6.x. I’m hoping to get us upgraded to 0.8.x soon.
Here’s what running an upgrade in production for Socorro looks like. Awesome right?
Here’s what a migration looks like.
Here’s a configuration file.
Generating a migration from the command line might look something like:
alembic revision -m "bug XXXXXX Add a new table" --autogenerate

The most difficult thing to deal with so far are the many User Defined Functions that we use in Socorro. This isn’t something that any migration tools I tested deal well with.

Happy to answer questions! And I’ll see about making a longer talk about this transition soon.

Matt Thompson: Using Bugzilla for Webmaker

openmatt — Fri, 17 May 2013 20:23:12 +0000

cross-posted from the Webmaker blog

We use Bugzilla to work open and get stuff done

Webmaker, like many Mozilla projects, uses an issue tracker called Bugzilla for filing tickets and getting stuff done. These two new pages provide tips and tricks for filing bugs, and for getting the most out of Bugzilla:

Bugzilla for Webmaker — the best place to start. How to file a Webmaker bug, plus simple tweaks for making Bugzilla easier to use.
Bugzilla for Webmaker: PRO TIPS – for digging deeper. How to make it easier for users to file tickets, tagging, searching and tracking bugs, Frequently Asked Questions and more.

Anyone can create, comment or contribute to a ticket

We work open. Webmaker is an open source, non-profit project powered by a global community of friendly humans like you. Anyone can create a ticket, comment on a ticket, and contribute. Just because it’s called a “bug” doesn’t necessarily mean there’s something wrong. It could just be a to-do, or a suggestion. All your tickets are welcome — don’t worry if you’re doing it right. We’re a friendly community, and we want your ideas!

Lawrence Mandel: May Open Web Open Mic Toronto: Call for Presenters

Lawrence Mandel — Fri, 17 May 2013 19:40:29 +0000

What: Mozilla Open Web Open Mic
When: May 22, 6-9pm ET.
Where: Mozilla Toronto, 366 Adelaide St. W, 5th Floor

Mozilla Open Web Open Mic (OWOM) is back for May. Put on by the Mozilla Toronto community, OWOM features a science share exploration and lightening talks (5 min). April’s event had a great turnout (50-60 people) and included a variety of content from WEBVTT, to MakerFaire, to my talk Building for Mobile Web Compatibility.

The call is currently open for science share participants and speakers. Sign up at
https://mozillianstoronto.etherpad.mozilla.org/owom-may-2013

Interested in attending? Sign up now at
http://owommay2013.eventbrite.ca/#

Tagged: mozilla, mozilla community, toronto

David Ascher: You are more than your job title

david — Fri, 17 May 2013 18:02:24 +0000

In grad school, I remember a conversation across the campus green with an visiting psychologist from Harvard. I don’t remember much about the conversation except that he introduced me to Isaiah Berlin’s notion of the Hedgehog and the Fox, and correctly pegged me as a Fox. I think I was a bit offended at the simplification, but time has proven him right. I’m certainly no hedgehog.

I got into a silly argument on twitter last night, about whether my looking to hire someone who I labeled (as job descriptions make us do) a “Coding Designer” was not just foolish (I’d seen the Unicorn references in my tweetstream already) but apparently a bad idea, because, so the ultra-simplified argument goes, you somehow can’t be both. And so I’ll use the energy to rant a bit about what seem to be prevailing attitudes around titleism and narrow definitions of “professionalism”.

We all need to define ourselves to others. It helps us be understood, and hopefully valued. Labels can be useful for that. We also, even more, like to label others. It helps us simplify our approach to them. If I can find a label for you, then I can rely on a prioris about how people with that label tend to think and behave, and I don’t need to actually get to know you too much. The more people we interact with, the more important these shortcuts are. Some roles are particularly subject to that (Recruiters, VCs, politicians, etc. — people who routinely talk to dozens if not hundreds of people a day). And the best at these roles are those who use a different labeling system than their peers. Recruiters who see the latent ambition or genius in a shy candidate; VCs who see the determination behind a stutter, or, conversely, the lack of self-confidence behind the bravado, etc.

Labels are useful and practical in the short term. And I don’t know how one could run a large HR department without them. But we should be careful to not take them too seriously, as in the long term, they can hurt. They hurt because people, especially interesting, worth-getting-to-know people, are much more subtle, complicated, confusing and hard to categorize creatures. Whether you take the label too seriously when thinking about others (e.g., refuse to see the valid opinion about a design expressed by a non-Designer) or about yourself (and limit your impact on the world because “oh, that’s not something that a mere ____ like me could say/do”), you’re not getting the most out of anyone involved.

As I write this, I realize that I feel quite strongly about this topic. Part of it is probably because I grew up in an educational system, which at least then believed way too much in labeling people and determining their fate based on that label. Much waste ensued. Part of it is probably because I can’t for the life of me figure out what my label should be, and if I can’t, then that must be bad. I’ve had a range of professional labels, from scientist to engineer, architect, team lead, vice president, CTO, CEO, blah blah blah. I’ve been called a designer, strategist, entrepreneur, boss, blah blah blah. None of those words will, I hope, be in my epitaph. And so I get cranky on twitter at night, because if there are people who strive to be both excellent at design and at coding, then by golly we should encourage them.

Titles are a poor approximation of a professional ideal, and a profession is a poor approximation of a human’s breadth, contributions, and talents. Embrace your inner fox, and if you happen to have both design and coding skills, can see a problem, conjure up a solution, prototype it, welcome challenges to your idea from peers, data, and users, apply.

Michelle Thorne: Webmaker Train the Trainer

thornet — Fri, 17 May 2013 15:40:46 +0000

Back in March, we kicked off the first in hopefully a series of train-the-trainer (TTT) events for webmaking.

The idea is to run events that train people who go on to train others how to teach the web. We focused on practicing an open and participatory ethos, adapting lesson plans, and facilitating events.

This is a post to share what we did and encourage people in designing their own train-the-trainer events.

How to run a Webmaker Train the Trainer

Our prototype, the Reps Training Days, ran for four days in Athens, Greece with 40 Reps from around the world. The agenda was based on Laura Hilliger’s research and insights on successful TTT program and on Allen Gunn’s participatory event methodology. It was made possible by the amazing Mozilla Greek community.

Our participants were Mozilla Reps, a fantastic ambassador program with some of the most active and thoughtful Mozillians. Reps have been early adopters and innovators with Webmaker. They organized nearly 50 events during last year’s Summer Code Party and are leading the way in developing tools, tutorials, and localization for Webmaker. It seemed like a natural fit to run our first TTT with them.

1. Participate in a Webmaker event

The first day of Training Days was spent observing and participating in a Hive Pop-Up, organized by Hive Athens. This was an opportunity for the participants to experience a webmaker event firsthand, to see the tools and activities in action, to learn about the logistics, and to understand the vibe.

We then circled up to discuss what we saw. Participants shared their reflections on what worked well at the pop-up and what they would change if they did their own.

2. Build the training agenda

Then we opened up the training days properly. While we had topics in mind we wanted to hack on together, it was more important that everyone in the room thought about what they want to learn or discuss. So we had an agenda brainstorm.

To do this: we split into groups for 3 people. On post-it notes, we wrote down topics. 1 topic per post-it and the encouragement to write it as concretely as possible.

Then everyone pasted the notes on the wall. We read them all and then clustered them by themes. This collaborative board formed both critical event documentation as well as agenda fodder for the coming days.

3. Teach someone something

To warm up to the idea of teaching, we then got into pairs. The task: teach someone something in 5 minutes.

One person would go and then switch. Even if you knew what was being taught, you were encouraged to play a good learner, asking good questions and prompting the teacher.

After this exercise, we circled up and discussed what we observed from this experience. For many, it was a great way to think about how to explain something clearly, using metaphors and knowledge building blocks. It helped bring people into a teaching mindset.

4. Make a learner profile

Now that we’ve been thinking about teachers and learners, we made small groups and hacked together a learner’s profile.

This goal of this activity was to think about who our learners are. We used Webmaker tools to make these profiles, which was also a fun, maker-y way to be introduced to these tools. Participants were encouraged to think about real people they want to teach.

5. Hack an event invitation

After we’ve made our learner profiles, we thought about the kind of event we wanted to run. Most of the participants have already organized Webmaker events in the past, so there was already some familiarity with the format.

Nevertheless, it was helpful to hack together an event invitation. The idea was to think about your target learner and to make an invitation that would speak to them. Again, we used Webmaker tools to quickly pull these invitations together on the web.

6. Deep dive into lesson plans

With a learner profile, an event invitation and some familiarity with Webmaker tools, we then introduced the hackable kits. These are remixable lesson plans that help mentors, trainers, etc. to teach the web. The idea is that they are adaptable to different contexts and that people can share new ways of teaching in a shared format.

Participants poked around in the kits and asked questions. We also did some fun icebreakers so they could see the activities in action and get some energy going.

7. Playtest lesson plans

Now came the fun part. We had to plan for a real live event the next day. So participants got into groups of five with one group facilitator.

They had to design a four-hour agenda for local youth. Using three recommended activities from the kits, they adapted the lesson plans. Then they walked through a script for the next day, including having people role-play as learners. It was a lot of fun to see and a great way to prepare for the big day.

8. Put training to practice at a live event

So with some nervousness, we got ready for the live event. About a hundred youth were coming. We split into different rooms, each group of five trainers getting about 20 learners.

While there were the inevitable challenges (the internet is down! one kid won’t listen!), the Reps did a terrific job. They rolled with their scripts, adapting them as they saw what was working. They also taught well in smaller pairs with their learners, sometimes adding new challenges or tools to fit their needs.

It was a beautiful and fun thing to see. All the training the days before paid off: the youth had a lot of fun and so did we.

9. Reflect on event, lessons learned and where from here

We ended the event with a closing circle. We talked about what we saw that day, what worked well, what didn’t. We each shared one thing we appreciated about the experience, and what we’re excited about doing next.

With that, we headed out into the city to enjoy the day and the rest of our time together.

10. Go forth and teach!

Each participant left the Training Days with a local plan. It was a short list of possible collaborators in their hometown, a date for a small team huddle to bring those people together, and then a date for a larger Webmaker event to organize with their new collaborators.

We also started interest groups in topics like localization and offline tools. And now, a few months later, the participants from Training Days are now “Webmaker Super Mentors”, mentoring people in an online course to learn how to teach the web.

In the coming months, we hope to keep remixing and improving these agendas, as well as work with people who are interested in TTT in their own cities or communities.

Let us know if you’d like to get involved! #teachtheweb

Jan Odvarko: New Firebug feature: Use in Command Line

Honza — Fri, 17 May 2013 15:22:06 +0000

Firebug 1.12 alpha 6 introduces one new feature called simply: Use in Command Line

This feature allows referring various page objects (HTML elements, JS objects, network requests, cookies, etc.) from within Firebug Command Line.
The user can also use object's properties in JS expressions.

See issue 6422 for more details.

This post explains the feature in detail and also asks for feedback.

How it works

As depicted on the previous screenshot the feature is represented by a new context menu item called Use in Command Line. This item is available for various objects (in various panels).

The action is simple: an expression $p is inserted into the Command Line if you execute the command. It's a variable that refers to the clicked object. You can also use the variable to access any property of the object.

What happens on the screenshot:

The DOM panel shows an object person implemented on the page.
The user clicks on the person's value (the green right hand part) and executes Use in Command Line.
Command Line popup is opened within the DOM panel (so, the user doesn't have to leave the DOM panel).
$p expression is inserted into the Command Line (and the command line automatically gets focus).
The user presses the Enter key to evaluate the expression.

Note that the person object implemented on the page looks as follows:

var person = {
name: "Bob",
age: 38
}

If you want to access properties of an object just use dot notation. You can do following in our example.

$p.name

See, even the auto-completion works as expected!

Supported Objects

The feature is implemented for various objects across all Firebug panels. Here is a list of those objects.

Any JS object (even functions etc.)
HTML elements
CSS rules and properties
Cookies
Network requests

Call for Feedback

All objects accessible through the $p variable are coming from the page content except of network requests and cookies.

We are actually logging nsIHttpChannel for network requests and nsICookie for cookies. So, the object comes from chrome (privileged scope) and isn't accessible in the page content. This way, the user has more info, but it could be confusing since the user could expect that the data are actually accessible from the page content (and Firebug is for page content debugging).

Possible options:

Do not implement the feature for network requests and cookies
Implement the feature only for XHRs (and show the real XMLHttpRequest instance) and cookie (coming from document.cookie)
Keep it as it is
Anything else?

What do you think?

Also, would you like to refer more objects at the same time? Something like having more variables (a history) available: $p0, $p1, $p2, ...

How would you expect it to be working?

Thanks for any feedback!

Arky: Mozilla Localization Makes a Positive Social Impact

noreply@blogger.com (Arky r) — Fri, 17 May 2013 10:31:09 +0000

Mozilla brings power of the web into ordinary people's hands. Every day I spend countless hours working with volunteer communities around the world to translate Firefox web browser. Reading Sudheesh Singanamalla's blog post about his encounter with a farmer in rural India was such a touching experience.

A Localization journey - A Farmer's tale - A Delightful Experience
It was on my way back in a cramped out bus, travelling researching about language changes and variations within the state of Andhra Pradesh, that I sat next to a man, quite old.

Me : What do you use in the internet? How do you talk to your son?
Old man: I go to Rajat's Net Cafe nearby by house and then talk from there on Google (meant Google+)
Me : Do you know how to read English and understand which button to click and so on?
Old man : Oh, i don't know English, but i use it in Telugu. The shop guy Rajat has seen me since he was small, so after my son went to Delhi, he separately bought a Telugu keyboard so that i can be using the keyboard.
Me: Okay, but then how do you read the information on the computer screen? Isn't that in English?
Old man : (Laughs) Don't you know, there is this software something called Firefox, it is in Telugu.
Me : Really? Can you tell me how the software looks?
Old man : You should know more, you're an engineering student but if you ask i'll tell you, its a small thing like this earth picture but a small cat , orange in colour is holding it.
Me: (smiling crazily) You know how to use it in Telugu?
Old man : Yeah, its not hard, I know how to read Telugu and also know how to use mouse, so clicking gets me the job done.

Anton Kovalyov: Google Mobile Summit

Fri, 17 May 2013 07:00:00 +0000

Yesterday, Google organized an invite-only summit for mobile web and tools developers. As an engineer working on Firefox Developer Tools, I was invited to be on a panel about mobile testing and tooling. This is my attempt to write down my answers in hope they’ll be useful for someone out there. I suggest reading them as separate, individual mini essays.

Read it on Medium

Lucas Rocha: Introducing The Layout

Lucas Rocha — Fri, 17 May 2013 05:28:00 +0000

As engineers, I believe the way we approach a problem is as important as the code we write. This is especially relevant in the context of UI engineering where design is such a vital element.

Unfortunately, it seems quite hard to find good content about everything that happens around us and inside our heads when we are building user interfaces. This is what The Layout is about.

My intent is to create a space for high quality content discussing the principles, mindset, and practices that I believe shape the craft of UI engineering. It is meant to be a shared space with many voices—so, expect some awesome guest authors.

I’ve just posted the very first article, Mind the Gap. My plan is to publish a new article every other week-ish. For now, subscribe to the RSS feed or simply follow The Layout on Twitter or Google+ to get future updates.

I really hope you enjoy it!

Christian Heilmann: Giving “image swivel” the vanilla web diet treatment

Chris Heilmann — Thu, 16 May 2013 23:05:05 +0000

With my chapter for the upcoming Smashing Book 4 done and talking about “The Vanilla Web Diet” and some workshops in the making I thought it is a good idea to show in an example how to make a seemingly good solution better by rethinking it with the principles of the vanilla web diet in mind.

The solution I am targeting is the image swivel effect that was posted a few days ago on CSS tricks. I sent my solution to the author and he was very interested in getting more information as to the why. The effect as it stands works and does the job to inspire people to play with it. The comments showed that where lots of developers had a go at creating their own solutions fixing some of the issues I am about to cover here. My favourite probably is Eduardo García Sanz’s Pure CSS solution which to me is a good plan to achieve an effect like that if you don’t need touch support or keyboard access.

Check out this Pen!

Before I get accused of “hating” – the code is OK as a demo to get creative juices flowing but it has many issues that would show up when used in production. It is also mouse-dependent and doesn’t work on touch devices. Therefore I’ll start by stating what I find troublesome about the published solution and then explain how to approach the problem to make it as simple and maintainable as possible. I also add support for mouse, keyboard and touch access and try to achieve the effect whilst not blocking people out.

If you prefer to see this as a screencast, here is one I recorded explaining the ideas, the issues I found with the original code and how my solution works. This is live and unscripted.

How much markup do we need?

Let’s start by looking at the solution that is shown in the CSS tricks article.
The HTML is the following:

<div id="faces">
  <div id="face-area">
    <div id="image-1" style="display: none;">
      <img src="/images/look-left-3.jpg">
    </div>
    <div id="image-2" style="display: none;">
      <img src="/images/look-left-2.jpg">
    </div>
    <div id="image-3" style="display: none;">
      <img src="/images/look-left-1.jpg">
    </div>
    <div id="image-4" style="display: none;">
      <img src="/images/look-center.jpg">
    </div>
    <div id="image-5" style="display: none;">
      <img src="/images/look-right-1.jpg">
    </div>
    <div id="image-6" style="display: none;">
      <img src="/images/look-right-2.jpg">
    </div>
    <div id="image-7" style="display: none;">
      <img src="/images/look-right-3.jpg">
    </div>
    <div id="the_faces_overlay">
      <div class="the_faces" data-number="1"></div>
      <div class="the_faces" data-number="2"> </div>
      <div class="the_faces" data-number="3"></div>
      <div class="the_faces" data-number="4"></div>
      <div class="the_faces" data-number="5"></div>
      <div class="the_faces" data-number="6"></div>
      <div class="the_faces" data-number="7"></div>
    </div> 
  </div><!-- END #face-area -->
</div> <!-- END #faces -->

Warning sign #1: Repeated HTML structures without logical connections

This is a very common mistake I see when starting an effect like that. We create something that contains the content and then something that gets acted upon to show the effect. Thus we lose the benefit of already interacting with the parent element and using event handling to find out where we are. I guess historically this is based on “CSS only solutions” that needed that kind of separation as you could not calculate or detect mouse position in CSS.

As we create two separate sets of markup for an effect we need to find a way to connect the element that was interacted with to the one we want to show. This means adding IDs to all the elements, classes to all the elements we interact with and a data attribute to tell which of the images to show. This is bad for maintainability. If we add an image, we need to also add an element to interact with. Great code is catered to making maintenance as easy as possible. Here we have a lot of dependencies to deal with when adding or removing an image.

Warning sign #2: Lack of semantic markup

The other gripe I have with this HTML is that it means nothing at all. As the order of the images is important, the right HTML construct to use here is an ordered list.

Warning sign #3: Lack of alternative content

Another big HTML mistake is adding images without an alt attribute. This means that screenreader users would get the file path of the images read out to them. Either provide a sensible alternative text or add a alt=”” to hide the image from screenreaders.

Warning sign #4: Dependency on the number of elements

One big warning sign to me here is that our effect is dependent on the number of images in the widget and that we need to have the data attributes and the IDs maintained together although they are on different elements.

The more elements we add means the more IDs we need to maintain. This is not what coding is about. Computers are good at calculating things for us.

Warning sign #5: Empty elements and inline styles

Whenever I see inline styles I know something went wrong. There is no point in them and if ever they should only be generated by code, not by humans. The same with empty HTML elements: you probably did some extra work that is not needed. HTML is there to contain content or provide interaction. If you have a lot of empty DIVs without an obvious templating use case, something went wrong.

Giving no power to CSS

The CSS of the solution is not much, and it doesn’t do much either. This is a shame seeing how much easier it is for a visual maintainer to change CSS rather than changing JavaScript.

body {
  background: #333 
}
#faces {
  height: 333px;
  width: 500px;
  margin: 0 auto;
  border: 8px solid white;
}
#face-area {
  height: 500px;
  width: 333px;
  position: relative;
}
#the_faces_overlay {
  position: absolute;
  width: 500px;
  top: 0;
  left: 0;
}
#faces .the_faces {
  height: 333px;
  width: 14.2857143%;
  float: left;
  margin: 0;
  padding: 0;
}

Warning sign #6: CSS dependent on the amount of elements

The glaring issue here is the “width: 14.2857143%;” which is calculated by dividing 100% into seven parts. This means that if you delete an image from the HTML, you also need to change the CSS width here. You should never be dependent on the amount of elements in your CSS, as those are prone to change. In this case especially there is no logical way to find out why this is the width. CSS calc() can at least make that obvious but in general it is a bad idea to create look and feel that is tied to a certain amount of elements.

Goldfish jQuery

The jQuery code to make the effect work is very short:

// Reveal the "center" image
var centerImage = $("#image-4").show();
// Bind hovers to each column
$(".the_faces").each(function() {
  $(this).on("mouseover", function() {
    $("#image-" + $(this).attr("data-number")).show();
  }).on("mouseout",function() {
    $("#image-" + $(this).attr("data-number")).hide();
  });
});
// Reset center image
$("#face-area").on("mouseleave", function() {
  centerImage.show();
}).on("mouseenter", function() {
  centerImage.hide();
});

It is, however, very demanding to the browser. Slowing down a browser can be done in many ways – the most damaging ones are heavy computation, accessing the DOM and lots and lots of event handling. The latter two is what we do here.

Warning sign #7: lack of data caching

We loop over all the elements with the class “.the_faces” and add a mouseover and mouseout handler to each of them. Every time these get fired, we read an attribute, create a string with it and access an element that has the ID of the string and show or hide it. Showing and hiding using the jQuery methods is another access to the DOM as it manipulates the display style property. We show and hide the “center image” upfront and also on another event hander on the overall parent element.

If we were to add touch and keyboard handlers we’d triple the amount of assigned event handlers as we apply them on each image.

I call this Goldfish code – we keep asking the browser for things we should already know. The widget interface we have here is static HTML - there is no loading of content, no changes in it. Therefore there is no point in continuously reading out what the data-number attribute is and ask the browser to find the element with a certain ID. Caching results is a very simple thing to do and the performance benefits are amazing.

Rethinking the solution the vanilla web diet way

I approached the solution by looking at what we need to do here:

We have a widget of a certain size with images in it
We have an unknown amount of images – it should be dead easy to remove or add one or replace them all
Moving the mouse over the widget should loop through the images, also touching the widget should do so and it would be nice to be able to flip forward and backward with the keyboard
Should things not work out it would be nice to have an display that still makes sense

The HTML - we don’t need IDs or data attributes

<a href="/productpage" id="rollover">
<ol>
  <li><img src="pics/IMG_0518.jpg" alt""></li>
  <li><img src="pics/IMG_0517.jpg" alt=""></li>
  <li><img src="pics/IMG_0516.jpg" alt=""></li>
  <li><img class="current" src="pics/IMG_0515.jpg" alt=""></li>
  <li><img src="pics/IMG_0519.jpg" alt=""></li>
  <li><img src="pics/IMG_0520.jpg" alt=""></li>
  <li><img src="pics/IMG_0521.jpg" alt=""></li>
</ol>
</a>

We should link this to something – after all a beautiful effect like that should react in some sale or deep-dive, right? Good thing is that in HTML5 links can contain other elements. So all we add is a link. This automatically gives us keyboard access to the widget – something otherwise we’d have to create by using roaming tabIndex.

As the order of the images is important, an OL is the right element to use. Each image has an empty alt attribute to ensure there is no hassle with screenreaders. Instead of defining which image to show as the first one in our JavaScript we keep this maintained in HTML, too, by adding a class of “current” to the image.

Showing and hiding with CSS

body {
  font-family: arial, sans-serif;
}
#rollover.js {
  display: block;
  margin: 2em;
  z-index: 3;
  position: relative;
  height: 270px;
  width: 200px;
  cursor: none;
}
#rollover.js img {
  width: 100%;
  position: absolute;
  top: 0;
  left: 0;
  visibility: hidden;
}
#rollover.js img.current {
  visibility: visible;
}

As our functionality is dependent on JavaScript, our styling should be, too. This we can achieve by adding a “js” class to the element when JavaScript is available and only apply the styles when needed. This CSS gives the widget a fixed size and positions all the images stacked inside it.

Instead of doing a hide() and show() in JavaScript, all we need to do is to apply a class of “current” to the element we need to show. All the hiding and showing is thus done in CSS which means that in the future we’d want to do other visual things with the “shown” and “hidden” images, all we need to change is the CSS.

(function(){
 
  if (document.querySelector) {
 
    var rollover = document.querySelector('#rollover');
    rollover.className = 'js';
 
    var images = rollover.querySelectorAll('img');
    var all = images.length;
    var width = rollover.offsetWidth;
    var ox = rollover.offsetLeft;
    var boundarywidth = width / all;
    var current = 0;
    var x = 0;
    var index = 0;
    var touched = false;
 
    var setcurrent = function(index) {
      if (images[index]) {
        images[current].className = '';
        images[index].className = 'current';
        current = index;
      }
    };
 
    var findindex = function(x) {
      index = parseInt((x - ox) / boundarywidth, 10);
      if (index !== current) {
        setcurrent(index);
      }
    };
 
    rollover.addEventListener('mousemove', function(ev) {
      if (!touched) {
        findindex(ev.clientX);
      }
    }, false);
 
    rollover.addEventListener('touchstart', function(ev) {
      touched = true;
    }, false);
 
    rollover.addEventListener('touchend', function(ev) {
      touched = false;
    }, false);
 
    rollover.addEventListener('touchmove', function(ev) {
      if (touched) {
        findindex(ev.changedTouches[0].clientX);
        ev.preventDefault();
      }
    }, false);
 
    rollover.addEventListener('keydown', function(ev) {
      var key = ev.char || ev.key || ev.which;
      if (key === 37) { index = index - 1;}
      if (key === 39) { index = index + 1;}
      if (index < 0) {index = 0;}
      if (index > all - 1) {index = all - 1;}
      setcurrent(index);
    }, false);
 
    if (rollover.querySelector('.current')) {
      for (var i = 0; i < all; i++) {
        if (images[i].className === 'current') {
          current = i;
          break;
        }
      }
    } else {
      setcurrent(current);
    }
  }
})();

Quite longer than the jQuery, but bear with me as this does a lot more and in a much less demanding fashion. We wrap our code in a closure to makes sure we don’t leave any nasty globals behind. That should always be the first step.

Then we test if the browser supports document.querySelector. This is the standard answer to jQuery’s $() and is supported by lots and lots of great browsers. It is not supported by old and outdated browsers, which is why it is a good idea to test for it. This means that old Internet Explorer versions will not get the effect but instead they get the images as a numbered list (as we made the CSS dependent on a class applied with JavaScript). This is good, as it means we don’t need to test on these old browsers, which is hard to do and frankly a waste of our time.

We get a reference to our rollover widget using document.querySelector() and add the “js” class to it. This hides all the images and sets up the look and feel of the widget – all maintained in CSS. No need to loop through a lot of elements or use inline styles to hide them.

Next we get references to all the DOM elements we need and calculate what we need to find out what to show when the mouse cursor or the finger is on a certain part of the widget.

First we get all the images and store them in images. This will not change while the page is open, and querySelectorAll() gives us references to all of them. We store the amount of images in all to compare against later on.

Next we do the thing dynamically that the original solution did by hand – calculate the width of the different strips of the interaction that shows and hides images. We do this by reading out the offsetWidth of the widget as it is defined in CSS, so we don’t know and don’t want to hard-wire any widths in our JavaScript. We find out how far left the widget is in the browser by reading the offsetLeft property and store that in ox. Then we calculate the width of the interaction boundaries by dividing the width of the widget by the amount of images and store that in boundarywidth.

This makes the widget flexible to change any time the CSS changes or the amount of images does. If you remove or add an image, the width of the boundaries is calculated newly. No need to change the CSS to reflect that width. We made this now maintainable simply by adding or removing list items.

We define the current shown image index as 0, preset an x variable as 0, define the current index of the detected movement as 0 and set touched to false.

The index will be the index of the image to be shown at a certain point in the interaction. The index of the currently shown one is be stored in current, x will be detected position of the mouse or finger on the screen and touched defines if the screen is currently being touched or not.

The setcurrent() function hides the last shown image and shows a new one by shifting the “current” class from one to the other. It then stores the new image index in current. This is a very simple way to show a new state in a collection of things that can only show one at a time. No need to ask the browser which one is visible when we can store this in a variable like current.

The findindex() function converts the detected horizontal position of the mouse cursor or the finger of the user into an index of our image array. All you need to do is to subtract the left position of the widget itself and divide the value by the boundarywidth. Convert it to an integer and compare it to the current index and if it differs, call setcurrent().

All that is left is to assign the event handlers to make the magic happen. The first is a mousemove listener on the widget that calls findindex() when no touch happened. The current horizontal mouse position is stored in clientX of the mousemove event.

Touch interaction needs to be initiated (at least in Chrome in my testing here) so we set the Boolean of touched to true when a touchstart happened and to false when touchend was detected.

When a user moves a finger over the widget the browser fires the touchmove event and the current horizontal position is stored in the clientX property of the changedTouches array. We only detect the first finger in this case.

Keyboard detection doesn’t give us a position on the screen, so all we do is manipulate the image index directly. We listen for a keydown event and check the code of the key that was pressed. If it is the left arrow we subtract one from the current index and in the case of the right arrow we add one to it. We ensure that the index stays in the allowed limits and call setcurrent().

The last thing to do is to show the current image. If there is one with the right class in the HTML we need to find out its index and we do that by looping over them until we find the right one. If there isn’t any we just apply the current class to the first image (as defined at the start of the script).

Many solutions to the same idea

I hope this gives you an idea of how to approach an effect like that when you want to put it in production. There are of course other ways of doing it, but I wanted to ensure a few things that get very often forgotten:

The whole effect is now generated from the HTML, so all you need to do to create a new swivel is add other images
The whole look and feel is defined in CSS and you can resize the widget without having to worry about the size of the different boundaries
It can be used with a keyboard, the mouse or on touch devices
The DOM interaction is kept to an absolute minimum which means the performance on low spec devices is much, much better
There is no jQuery dependency

Robert Kaiser: Preserving Software - Feedback Requested!

KaiRo — Thu, 16 May 2013 22:08:13 +0000

As Digital Preservation is part of the agenda of the US Library of Congress, they're doing a workshop on Software Preservation next week, and Mozilla was invited as an expert group. Otto de Voogd and myself are in the delegation going there (I'll be roughly in the Washington, DC, area from Saturday until June 2) for Mozilla - and the text below is a guest post by Otto with questions that we would like some feedback on so we can represent the Mozilla community as well as possible:

On the 20th and 21st of May the Library of Congress holds a workshop on the topic of preserving software.
Otto de Voogd and Robert Kaiser will be representing Mozilla, putting forward our viewpoint as custodians of a codebase with a significant heritage and importance.

Many questions and thoughts arise. Here's an overview of ours; we look forward to feedback.

- Should archivists keep source codes or executables or both?

Executables and source code are both valuable. Executables are valuable because the source code is sometimes not available, or perhaps the build tools are not, and setting up a build environment for older code can be a difficult and complex thing.

Source is valuable to determine how a program works. It also makes it possible to reuse code and algorithms, especially, but not only, in the case of open source software.

- Preserving documentation.

Preserving documentation that goes with software, seems logical.
Would this need to go as far as preserving discussion threads and entries in bug trackers?

- Preserving environments/platforms.

It seems obvious that without preserving an environment in which the software can run, it is going to be impossible to experience the software.
Preserving such an environment should therefor be part of the software preservation effort.

To avoid the physical constraints imposed by preserving old hardware (which would be a preservation effort in its own right), a solution would be to build virtual machines and emulators.
As hardware capacity constantly grows, running virtual versions of older hardware should generally be feasible.

To fully recreate an environment we'd also need to preserve the operating systems and other software tools that the preserved software needs to run.
Those being software themselves would logical already be included in any software preservation effort.

Preserving documentation concerning environments, would also be required.
To build virtual machines and emulators it would be helpful for hardware makers to make technical specifications available. One could envision this to become a legal requirement at least for older hardware.

Can we imagine a world where web based emulators would allow an online digital library to serve users worldwide? Users who would be able to run old software in emulators running in their browsers...

- Is everything worth preserving, if not how does one go about selecting what is worth preserving?

Does one need to preserve every version of software, just the last version or all major releases? What about preserving software that has not spread widely. Would there be some threshold, or some other criteria?

- How does one index software and search the library?

There will be a need to gather meta data about software and the preservation of documentation as we already mentioned. This meta data and documentation could serve to populate an index enabling for instance the search for particular features.

- Can software preservation help in making code reusable?

If there are good ways to actually find relevant and useful code, this could lead to more reuse not only of actual code, but also of algorithms and concepts.
It may also become a valuable source for students who wish to learn about actual implementations of software solutions.

At the very least a minimum of meta data, such publication dates, copyright owners and licenses should be available to determine how certain code can be reused.
In particular for open source software we believe that software libraries should strive make it available without restrictions.

- Preserving data formats.

The software preservation effort should also include an effort to preserve data formats. Including technical descriptions of those formats and the tools to read, write and edit those formats.

- Can software preservation help in the discovery of prior art?

We believe it can, and as such preserving old code could be a great tool in preventing the repatenting of existing software concepts.

Of course we believe that software patents shouldn't exist in the first place, as software is already covered by copyrights, but at the very least prior art is a good avenue to prevent some of the worst abuse of software patents.

- How do copyrights affect software libraries?

A lot of software is licensed to be used on a particular piece of hardware or only available via subscription. How does this affect software libraries? Should there be exceptions like there are for traditional libraries?

In the life cycle of software, the commercially exploitable time is limited, likely anything older than 10 years no longer has any commercial value.
Maybe copyrights on software should be significantly reduced to something like 10 years, which is more than enough to cover the commercially exploitable timeframe of the software life cycle.

Such a limit would greatly enhance the work of software libraries, increasing availability and ease of access as well as removing a lot of the red tape involving requests for permission to keep copies.

- What about software as a service?

And what about software as a service, where neither the source code nor the executables are ever published? How can something like Gmail be preserved, when neither the service's code nor the environment is available to the public?

- Preserving "illegal" or cracked copies?

What if a copy of a piece of software comes from an illegal source? A cracked version with modifications maybe? They have value in themselves as they are a cultural expression.

What if such an illegal copy is the only copy still available? Would it make sense to preserve that too?

Rob Campbell: #io13

rob — Thu, 16 May 2013 14:45:49 +0000

I sat in on three sessions at Google IO 2013 yesterday.

Memory Lane with Chrome Devtools and GMail was the first.

The presenters showed off their Heap Tracking Profiler and Memory Tracking tool in the Timeline and explained how to use them to track down a leaky DOM node. It was a practical application of how to use a developer tool to solve a particular problem.

One interesting takeaway that surprised the presenters during their research: Always allocating more memory (caching) as a way to improve performance in a large application like Gmail is not a panacea for slow performance. Having a large heap space actually slows down the garbage collector and your performance suffers. It’s a fine balance.

What surprised me was how they analyzed their problem by tracking a user with a known high memory problem for three days. The Google team constantly monitors their apps’ performance via the window.performance API and can single out hotspots in the population.

The next talk I sat in on was about Chrome Apps. The presenter, Erik Kay showed off some of the “Immersive” experiences of Chrome Apps and the different ways they could interact with the hardware on the Chrome Book. The talk included a demo of a small thermal printer being hooked up and controlled over USB which garnered some applause.

The Chrome Web Store lets you buy apps for Chrome.

The only real mention of Android was that they were using PhoneGap and Cordova to provide their compatibility layer. Same for IOS. There will be compatibility issues with deploying on iOS but it seems surprising that they would pursue this completely separate technology for Android. Surely they could ship a full version of the Chrome Runtime and deal with hardware incompatibilities directly.

The questions from the room were interesting. One man (not a Mozillian) asked about WebRTC compatibility across the different platforms, pointedly repeating the question of whether or not he’d be able to use WebRTC in an app on iOS. Only when their WebView supports it.

Another man asked something about interoperability between B2G and ChromeRT. Erik said that there is “no forcing function yet to drive standardization”.

I think my biggest takeaway from this talk was that people wearing Google Glass look like dorks.

My second biggest takeaway was that I was very surprised that there was zero mention of the Google Play Store for Chrome Apps.

Last talk I attended was a Fireside Chat with the Blink Team. While I was expecting an actual fire and was disappointed there wasn’t one, the team bravely took questions from an audience confused about feature-detection, unprefixed CSS and market fragmentation.

Dan Buchner asked the panel something about standardization and I felt a little badly for the Blink team who had a whole chunk of slides talking about how they’re going to be good citizens. (If you want to participate, you should join blink-dev@chromium.org.)

I was interested in ChromeStatus.com/features which shows a spreadsheet of features in-progress. Time will tell how their Intent to Implement and Intent to Ship broadcasting will work from an Open Source point-of-view, but they are currently claiming that a third of their intents to implement are coming from outside of Google.

I wanted to meet Paul Irish after the talk but Steven Shankland showed up and pushed me out of the way. When he was done I did get to meet him, but I think Buchner had made him angry or something. Maybe he was just tired. I dunno.

Joey Armstrong: Transitioning to moz.build

njaph66 — Thu, 16 May 2013 14:04:47 +0000

If you have been wondering about status for the transition to mozbuild as a replacement for makefile infrastructure, a source is becoming available. People involved with the conversion have been meeting bi-weekly on the topic – details and content can … Continue reading →

Michael Kaply: Disabling Safe Mode (Again)

Mike Kaply — Thu, 16 May 2013 13:49:51 +0000

So lots of people are having trouble disabling safe mode using my earlier instructions and I discovered it is because of problems overlaying the dialog. So here are some new instructions that should work for everyone.

First, create a disablesafemode directory in distribution/bundles where the Firefox executable is located (you'll probable have to create the distribution and bundles directories as well). Then create a file called chrome.manifest that looks like this:

content disablesafemode content/
override chrome://browser/content/safeMode.xul chrome://disablesafemode/content/safeMode.xul

Then create a subdirectory called content. In that directory, create a file called safeMode.xul that looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet href="chrome://global/skin/"?>
<dialog xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul"
        id="safeModeDialog"
        buttons=","
        ondialogcancel="closeFirefox();">
  <script type="application/x-javascript;version=1.7">
  <![CDATA[
  function closeFirefox() {
    Components.utils.import("resource://gre/modules/Services.jsm");
    Services.startup.quit(Services.startup.eForceQuit);
  }
  ]]>
  </script>
  <description>
  Safe Mode has been disabled.
  </description>
</dialog>

This will cause a dialog to be shown instead of the safe mode dialog that when the user closes, will simply close Firefox. If you'd prefer to show no dialog at all, change ondialogcancel to onload.

I've tested this a lot, and it is working for me. Please let me know if you have problems.

Nagappan: Announce: LDTP 3.5 - Linux GUI test automation tool

noreply@blogger.com (Nagappan) — Thu, 16 May 2013 08:04:18 +0000

Highlights:
New API:
* inserttext, objtimeout, guitimeout, getcellsize, getcellvalue,
getobjectnameatcoords, getcombovalue, getaccesskey in Python client
* doubleClick, doubleClickRow, onWindowCreate, getCellSize, getComboValue,
appUnderTest, getAccessKey in Java client
* getcellsize, getcellvalue in Ruby client
* GetCellSize, GetComboValue, AppUnderTest, GetAccessKey, MouseRightClick,
DoubleClick, DoubleClickRow, RightClick in C# client

New control type:
* POPUP MENU for Ubuntu environment

Bugs fixed:
Ruby client:
* Fixed optional arguments to imagecapture
* Check window_name parameter, if empty then use @window_name passed in
constructor

Python client:
* Fixed optional argument APIs to work on both Windows and Linux
* imagecapture x, y offset, height and width parameters are disregarded if
window parameter is provided - Bug#685548
* Return unicode string all the time on gettextvalue
* Fix partial match argument in selectrow, compatible with Windows
* Patch by ebass to support Python 2.6
* Added Errno 101 as we see in ebass Ubuntu 10.04 environment

Core LDTP2
* Include label type on gettextvalue
* Don't include separators in the list

Perl client:
* Added perl client

Credit:
* Sawyer X for the Perl interface
* ebass (IRC nick name)
* Marek Rosa
* Thanks to all others who have reported bugs through forum / email /
in-person / IRC

About LDTP:
Cross Platform GUI Automation tool Linux version is LDTP, Windows version
is Cobra and Mac version is PyATOM.

* Linux version is known to work on GNOME / KDE (QT >= 4.8) / Java Swing /
LibreOffice / Mozilla application on all major Linux distribution.
* Windows version is known to work on application written in .NET / C++ /
Java / QT on Windows XP SP3 / Windows 7 / Windows 8 development version.
* Mac GUI testing is known to work on OS X Snow Leopard/Lion/Mountain Lion.
Where ever PyATOM runs, LDTP should work on it.

Download source / binary (RPM/DEB)

Documentation references: API / JavaDoc

For detailed information on LDTP framework and latest updates visit
http://ldtp.freedesktop.org

Report bugs

To subscribe to LDTP mailing lists

IRC Channel - #ldtp on irc.freenode.net

How can you help: Spread the news and send back your feedback to us

Brendan Eich: C is for Cookie

Brendan Eich — Thu, 16 May 2013 07:11:50 +0000

Mozilla is engaged in a broad, deep conversation about Internet privacy. We believe in putting users in control of their online experience, and we want a healthy, thriving web ecosystem — we do not see a contradiction. However, sometimes a crucial experiment is required to prove it.

To this end, we are testing a patch from Jonathan Mayer. Jonathan’s patch matches how Safari has worked for years, and does the following:

Allows cookies from sites you have already visited.
Blocks cookies from sites you have not visited yet.

The idea is that if you have not visited a site (including the one to which you are navigating currently) and it wants to put a cookie on your computer, the site is likely not one you have heard of or have any relationship with. But this is only likely, not always true. Two problems arise:

False positives. For example, say you visit a site named foo.com, which embeds cookie-setting content from a site named foocdn.com. With the patch, Firefox sets cookies from foo.com because you visited it, yet blocks cookies from foocdn.com because you never visited foocdn.com directly, even though there is actually just one company behind both sites.

False negatives. Meanwhile, in the other direction, just because you visit a site once does not mean you are ok with it tracking you all over the Internet on unrelated sites, forever more. Suppose you click on an ad by accident, for example. Or a site you trust directly starts setting third-party cookies you do not want.

Our challenge is to find a way to address these sorts of cases. We are looking for more granularity than deciding automatically and exclusively based upon whether you visit a site or not, although that is often a good place to start the decision process.

We plan to ship an evolution of the patch “on” by default, but we want to make refinements first. To make sure we get this right we need more data. Our next engineering task is to add privacy-preserving code to measure how the patch affects real websites. We will also ask some of our Aurora and Beta users to opt-in to a study with deeper data collection.

There are many conflicting claims about how this patch will affect the Internet. Why debate in theory what we can measure in practice? We are going to find out more and adjust course as needed. This is the essence of the release test cycle.

On Tuesday we did two things:

The patch has progressed to the Beta release channel for Firefox 22, but it is not “on” by default there. This allows more people to test the patch via Firefox’s “preferences” (AKA “options”) user interface, and avoids an abrupt change for site owners while we work on handling the hard cases.
The patch remains in the Aurora channel for Firefox, where it is “on” by default. This gives the patch better ongoing test coverage and facilitates A/B testing.

We have heard important feedback from concerned site owners. We are always committed to user privacy, and remain committed to shipping a version of the patch that is “on” by default. We are mindful that this is an important change; we always knew it would take a little longer than most patches as we put it through its paces.

For those who read this as Mozilla softening our stance on protecting privacy and putting users first, in a word: no. False positives break sites that users intentionally visit. (Fortunately, we haven’t seen too many such problems, but greater testing scale is needed.) False negatives enable tracking where it is not wanted. The patch as-is needs more work.

We look forward to continued dialog with colleagues, contributors, fans, and detractors. We will update all of you within six weeks so you can understand our thinking and how we will proceed. Comments welcome.

/be

P.S. Cookies (name history) were originally intended to be ephemeral (Windows 3.1 had so little usable memory with its 64K memory segments that Netscape’s founders had no choice). At first, they held only session state that could be recovered from the server by logging in again.

(Remind me to tell the story some day of Montulli’s aborted “twinkies” idea from the Netscape 2 era. UPDATE: Lou has published a new blog post about cookies.)

How far we have come in the amazing, living system that is the Web! No one planned for what actually happened, but with more work on the cookie policy in Firefox and (I hope) other browsers, I believe that we can evolve to a better space.

Erik Vold: Old School To Jetpack Part 2 - XUL Bashing

evold@mozilla.com (Erik Vold) — Thu, 16 May 2013 07:00:00 +0000

In Old School to Jetpack Part 1 I briefly mentioned that xul overlays are not supported with Jetpacks, so today I want to talk about how to convert old school XUL overlays.

A typical overlay would look something like lke this addon-browser-overlay.xul:

<?xml version="1.0"?>
<overlay xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul">
  <toolbar id="addon-bar">
    <textbox id="test-status-bar-message" insertbefore="addonbar-closebutton" value="Hi!"></textbox>
  </toolbar>
</overlay>

This overlay, would then be associated to browser.xul in the chrome.manifest file with an entry like this:

overlay    chrome://browser/content/browser.xul chrome://{add-on name here}/content/addon-browser-overlay.xul

Finally, the overlay would produce this in Firefox’s addon-bar:

With Jetpacks however the latter step cannot be done, because these overlay entries are ignored for restartless add-ons. One could manually force the overlay with a call to document.loadOverlay, but there is no way to undo this, that is why overlay’s are not supported for restartless add-ons at the moment.

So what is needed here is a module that will add the desired XUL elements to the desired XUL documents.

This is why I wrote a XUL package for Jetpack, it is very raw at the moment, and very simple, so you’ve been warned!

XUL Package for Jetpack

About

The source code can be found on github here.

Pros

Quick easy solution for coverting XUL overlays for use with Jetpacks
Handles unloads

Cons

Only works on Firefox (or other applications made with XUL)
At the moment this package only works for adding xul to browser.xul (although it won’t be hard to extend)

Alternatives

Before I go in to this module I want to take a moment to point out that there are many third party packages which handle most of the use cases that overlays for browser.xul support; they don’t cover everything though, like the example above, so this is why I wrote this general purpose package.

Usage

To implement the example above using this module, one would simply need to do the following:

const { XUL, getXULById } = require('xul/browser');
getXULById('addon-bar').insertBefore(XUL('textbox', {
  id: 'test-status-bar-message',
  value: 'Hi!'
}), 'addonbar-closebutton');

And this code is restartless!

Byron Jones: happy bmo push day!

glob — Thu, 16 May 2013 06:54:44 +0000

the following changes have been pushed to bugzilla.mozilla.org:

[868044] Bugzilla should be automagically aware of which product channel a bug-filer is using.
[815531] splinter fails to display attachment 681995 correctly
[869077] Component isn’t preselected from the query_string when cloning a bug
[850135] hide the textarea custom fields by default with an (edit) link
[866447] Make form.doc set up whiteboard for scrumbu.gs
[871436] release tracking flag refresh (24)
[870907] Project Kickoff Form: javascript responsible for checking for required values not working
[828344] “contains all of the words” no longer looks for all words within the same comment or flag
[872022] don’t link a review flag to splinter unless the attachment is a patch
[841559] Project Kickoff Form: Simplify finance question regrading budget
[850920] Project Kickoff Form: Add new question to Legal subsection area
[841449] Project Kickoff Form: New Question within Finance Sub Questions
[850934] Project Kickoff Form: Make Release Date a Required Field
[850932] Project Kickoff Form: Rework Privacy Policy/Project sub questions
[826214] New file with one line isn’t shown
[797840] Replying to a comment on Splinter always replies to the first comment
[821889] Make it so that Splinter shouts loudly when a patch introduces Windows line endings

Filed under: bmo, mozilla

Alexander Surkov: Accessible Mozilla: Tech overview of Firefox 22

noreply@blogger.com (Alexander Surkov) — Thu, 16 May 2013 05:45:40 +0000

Firefox 22 reached beta status (it will be released June 24). It's time to list accessibility improvements we made for this version.

ARIA

ARIA role="note" doesn't allow name from subtree (bug) anymore. The bug caused JAWS, for example, to announce role="note" content twice.

HTML

* HTML radio group position doesn't count hidden radio elements (bug). So if the page contains hidden input@type="radio" then a screen reader doesn't take them into account announcing the number of radios.

* HTML input@type="file" changed its hierachy. Now it contains a push button and a label. Be careful if you have dependences on this hierarchy (see bug).

* HTML5 header and footer has changed their mapping according to HTML spec:

footer element that is not a descendant of an article or section element. contentinfo role;

header element that is not a descendant of an article or section element. banner role.

XUL

XUL label@value element now implements text interface (partially). You can obtain a text between offsets but you can't get it by words for example (refer to bug). XUL label is used wide in Firefox user interface (for example, in Options dialog). Let us know if you have problems with new implementation.

ATK

RELATION_NODE_PARENT_OF has been implemented. It's exposed for aria-owns markup and XUL trees (used in Thunderbird and Firefox bookmarks).

Text interface

As I wrote before we started text interface reimplementation. Firefox 22 got improved getTextAt and getTextBefore offsets at word boundaries. Note, in case of getTextAt we had to mimic WebKit behavior rather than follow the ATK spec to keep Orca working.

Attention. It might be important

* Document load event may be fired a bit later than we used to do that, it will be fired right after all events contained in the queue at the time when document got loaded (see bug).

* IServiceProvider interface is implemented as a tear off (bug).

Dave Townsend: Firefox now ships with the add-on SDK

Dave Townsend — Wed, 15 May 2013 22:31:59 +0000

It’s been a long ride but we can finally say it. This week Firefox 21 shipped and it includes the add-on SDK modules.

What does this mean? Well for users it means two important things:

Smaller add-ons. Since they no longer need to ship the APIs themselves add-ons only have to include the unique code that makes them special. That’s something like a 65% file-size saving for the most popular SDK based add-ons, probably more for simpler add-ons.
Add-ons will stay compatible with Firefox for longer. We can evolve the modules in Firefox that add-ons use so that most of the time when changes happen to Firefox the modules seamlessly shift to keep working. There are still some cases where that might be impossible (when a core feature is dropped from Firefox for example) but hopefully those should be rare.

To take advantage of these benefits add-ons have to be repacked with a recent version of the SDK. We’re working on a plan to do that automatically for existing add-ons where possible but developers who want to get the benefits right now can just repack their add-ons themselves using SDK 1.14 and using cfx xpi --strip-sdk, or using the next release of the SDK, 1.15 which will do that by default.

Margaret Leibovic: New Coding Stewards Mailing List

Wed, 15 May 2013 18:18:00 +0000

As part of the Grow Mozilla effort, the coding stewards have been working to grow our coding community. Our main focus has been to increase the number of contributions to the core mozilla-central codebase, which includes making it easier for newcomers to get started, as well as keeping existing contributors engaged. Recent work has been focused on things like mentored bugs and recognizing contributors.

We hold open weekly meetings every Wednesday to discuss progress on these goals, but we want to give more developers the opportunity to get involved with this effort, so we created the coding-stewards@mozilla.org mailing list for broader discussion. If you’re interested in growing our coding community, but you might not be interested in another weekly meeting, please join the list!

Michael Kaply: Setting Default Application Handlers

Mike Kaply — Wed, 15 May 2013 15:16:43 +0000

One of the questions I get asked a lot is how to set default application handlers in Firefox (the Applications page in preferences). Most companies have resorted to creating a default mimeTypes.rdf file and loading this into their default profile. I finally took the time to understand how these handlers work and can give you some code to use.

   var handlerSvc = Components.classes["@mozilla.org/uriloader/handler-service;1"].getService(Components.interfaces.nsIHandlerService)
  // Change "image/tiff" the mime type you want to set the preference for
  var realMIMEInfo = mimeService.getFromTypeAndExtension("image/tiff", "");

  var file = Components.classes["@mozilla.org/file/local;1"].createInstance(Components.interfaces.nsIFile);
  // This should be the path to the .app file on Mac or the EXE on Windows
  file.initWithPath("/Applications/Preview.app");
  var localHandlerApp = Components.classes["@mozilla.org/uriloader/local-handler-app;1"].createInstance(Components.interfaces.nsILocalHandlerApp);
  localHandlerApp.executable = file;
  // The name that will be shown in preferences.
  // Not used on Mac
  localHandlerApp.name = "Preview";
 
  realMIMEInfo.preferredApplicationHandler = localHandlerApp;

  // This says to always use the helper app 
  realMIMEInfo.preferredAction = 2; // useHelperApp
  // This says not to ask
  realMIMEInfo.alwaysAskBeforeHandling = false;

  handlerSvc.store(realMIMEInfo);

You can add this code to your autoconfig file.

Robert O'Callahan: Travel

noreply@blogger.com (Robert) — Wed, 15 May 2013 11:58:19 +0000

Last week I was in California. It was my first time in the Mozilla SF office --- lovely view of the Bay from the roof. I always enjoy the free snacks and I'm always glad we don't have them in Auckland. I spent quality time with some of the people I know and love at Mozilla, and that's always exciting.

On Wednesday and Thursday I was at Half Moon Bay doing LEAD training. It was fun, but thinking about "soft skills" for two days straight is quite draining for me; my social skills are learned, not innate.

This cohort is different from previous cohorts --- most members are relatively new to Mozilla; of our cohort, Vlad and I have been at Mozilla the longest, by far. This gives me the honor and duty of representing the Mozilla old guard. I feel the power of the narrative that has me in the "crusty old engineer, harping about the old days and resisting change" role ... and I do my best to reject it :-).

One of the results of LEAD so far is that I perceive my relationships with other Mozilla staff to be warmer and stronger than they perceive them, on average. I suspect this may be related to the difficulty of maintaining deep relationships with remote employees I see a few times a year at best. I'm still trying to figure that out.

Plane movies:

Gangster Squad: Genre flick. OK.
Zero Dark Thirty: Pretty good. Not exactly entertaining, but interesting.
Live And Let Die: Some kind of cross between a Bond movie, a blaxploitation flick, and the Dukes Of Hazzard. Odd.
The Town: Genre flick. Slightly better than average.
I, Anna: Sort of noir-ish psychological thriller. OK.
Les Miserable: The movie of the musical. Pretty good. I need to read the book sometime.

Interestingly, Air New Zealand lets you see what movies they're showing on their routes. This Web interface is a pretty faithful mockup of the actual in-seat interface (which is pretty bad ... it would be great to be able to see more than one movie title at a time).

On Friday I leave for Taiwan for a week at the Mozilla office, a "Web rendering" work week. This should be even more fun than last week.

Robert O'Callahan: The Direct Route

noreply@blogger.com (Robert) — Wed, 15 May 2013 11:20:13 +0000

Over time I've become increasingly impressed with the broad applicability of Matthew 18:15-17:

If your brother or sister sins, go and point out their fault, just between the two of you. If they listen to you, you have won them over. But if they will not listen, take one or two others along, so that ‘every matter may be established by the testimony of two or three witnesses.’ If they still refuse to listen, tell it to the church; and if they refuse to listen even to the church, treat them as you would a pagan or a tax collector.

The first step is often difficult but crucial. The path of least resistance can be to go behind your antagonist's back --- to your friends, or their friends, or their manager. I've seen all kinds of negative consequences from following that path --- hurt, distrust, unnecessary escalation, confusion and fear. I feel my integrity depends on people knowing that whatever I say about them to others, they will not be surprised by because they've already heard it from me.

This applies in the other direction too, when people complain about third parties to me. If the third party is unaware of the issue, I don't want to know --- go away and talk to them first.

There are rare exceptions, usually involving time-critical emergencies or complex secrecy requirements.

Brandon Savage: Dealing with duplicated code

Brandon Savage — Wed, 15 May 2013 11:00:31 +0000

We’ve all seen it: we’re working along, and we come across code that just has a feel to it. It’s like déjà vu. You’ve seen this code before. You open another file – sure enough, there it is. The same code. Almost line for line. In large code bases, it’s likely that there are dozens [...]

Nikhil Marathe: Your Jabber ID as your Persona identity

noreply@blogger.com (Nikhil) — Wed, 15 May 2013 09:45:38 +0000

(This is NOT an official Mozilla project and does not in any way reflect the views of my employer.)

Mozilla Persona is a way for users to use their e-mail ID as their identity on the web. While cool, it will only really take off when existing services that people use become Identity Providers. XMPP (Jabber) is a widely deployed IM protocol whose IDs look like e-mail and it is a secure, federated system in alignment with Persona’s goals. I thought it would be really cool if I could log in to Persona enabled sites using Jabber IDs. I’d like to announce browserid-xmpp which does just that.

It should work with any XMPP server that supports components and BOSH. That said I have only tested it on my VPS (with Prosody, ejabberd and Openfire), so any issues and pull requests are welcome, as is a quick comment if you deploy it on your server. You’ll also need a relatively sophisticated web server like Apache or nginx to serve the browserid file with the right Content-Type. CheckMyIdP is a great way to check if everything is setup properly.

browserid-xmpp is two things. The first is a XMPP component that can plug into any XMPP server and answer a certificate signing query. This is a fork of the “official” browserid-certifier with an Jabber-RPC front-end rather than a web service.

The second is the set of provisioning and sign in pages that can be re-used by any domain. The authentication is handled as a two stage process using BOSH. This was my first experience with BOSH and it is ridiculously cool how it works and supports session hand-off to another page, without which this would not be possible. On the sign in page, an XMPP stream is established and authentication is done using standard XMPP authentication. The established BOSH stream has a session ID and every message sent has an incrementing request ID. On successful sign in, the sign in page sticks these two, along with the JID into sessionStorage. The provisioning page reads these out and ‘attaches’ to the existing BOSH stream. Due to the unpredictable nature of the SID and RID, there is a reasonable guarantee that someone who attached to the stream successfully knew about the stream before. The provisioning page then makes a Jabber-RPC call over the same stream to the XMPP component. This call is performed on behalf of the JID and a certificate is sent back to the browser. You are now signed in!

P.S. I’d like to thank Cory Benfield for an excellent guide to writing an IdP.

P.P.S. This post was published right before a 12-hour plane ride, so I’ll be back for tech support in a while.

Benjamin Kerensa: We should keep Firefox as default browser in Ubuntu

Benjamin Kerensa — Wed, 15 May 2013 07:49:48 +0000

I think that Firefox should stay as the default browser in Ubuntu for the following reasons:

Mozilla has a vibrant open source community and many of our contributors are active in both Ubuntu and Mozilla.
Mozilla’s mission is to promote openness, innovation and opportunity and Firefox encompasses all of those goals.
Chromium as a package has fell behind in the past while Firefox has team of contributors in the Ubuntu Community who keep it up-to date every cycle.
Firefox is perhaps one of the most popular open source browsers.
And most importantly in my opinion Mozilla by default respects user privacy and choice.

Why fix something if it’s not broken? If others prefer Chromium well then “sudo apt-get install chromium-browser” and I guess that’s just my two-cents on the topic.

Jared Wein: Firefox OS – The Web is the Platform

msujaws — Wed, 15 May 2013 00:27:29 +0000

This past Monday I gave a presentation at the Mobile Monday Detroit meeting about Firefox OS. Thanks to some great screen recording software by TechSmith, I also recorded it

I’ve also made the slides available online for those who just want to read (press ‘s’ on your keyboard to see my speaker notes, you may have to allow pop-ups).

Mobile Monday Detroit is a monthly meetup that gets people together to talk about activity in the mobile space. Many topics cover Android and iOS, so it was really cool to snag a spot on the podium and give a talk about one of the ‘alternative’ operating systems. Also presenting this week was Randy Nunez from Ford who gave a talk about the various open source mobile operating systems and Chris Peplin from Ford who talked about the Open XC API for cars that Ford is working on.

If you live in southeast Michigan you should try to make it out to one of the meetings. More information can be found at the Mobile Monday Detroit Meetup page.

Tagged: detroit, firefox, firefoxos, mozilla, planet-mozilla, presentation

Lloyd Hilaiel: What Is Persona?

Wed, 15 May 2013 00:00:00 +0000

I've worked on the Persona service for over two years now. My involvement began on a fateful airplane ride with the father of the specification that would become Persona. Shortly after, on April 6th 2011 I made the first commit. On July 1st 2011 Mike Hanson and I had built a prototype, made some decisions, and I wrote a post on what it is and how it works. Shortly after, Ben Adida joined me and we began carefully hiring wonderful people.

We've come a long way: over 20 people directly contribute to persona on a regular basis, it's supported by a community of thousands, and it's got an extremely hopeful growth curve. The striking thing, the subject of this post, is how Persona has always been guided by a consistent and clear vision. The community passionately defends this vision, they know when a feature or change does not fit the vision, and despite being separated by continents and cultures - I believe this vision is what has held the project together and let it grow into what it is today: An audacious attempt to change the world that actually has legs - an attempt to kill the password, to keep people safer, and to make trying new things online fun again.

So what is this vision? How succinctly can it be expressed? Let's find out: this post is my personal attempt to express the Persona vision. I would love to know how others find this, whether those who really care about Persona would empasize the same points.

Christian Heilmann: A few questions and answers about “mobile web” and sites vs. apps

Chris Heilmann — Tue, 14 May 2013 23:00:28 +0000

I just got asked to provide a few answers for a survey amongst “mobile web experts” and thought it’d be good to re-use those here. So here goes:

What is the difference between a web site and a web application?

There are a few differences. On a very basic level applications are catered for doing things whereas web sites are more catered for offering content for consumption. Web sites started as structured, interlinked academical documents. Later on we added multimedia content to make them much more engaging but all in all they are fixed in their state.
Applications are more dynamic. They allow for customisation of the interface and store the state of what happens so that when you get back to it you can go on from where you left off.

The use case of an app should always be to do something with it. This could be as simple as voting on a how much you like a kitten photo and go as far as editing video content live in your browser or on your device. Basic examples would be a webmail client as an application and a pure image gallery to click through as a web site.

Web sites are static whereas web applications have atomic updates and in of themselves have a very small footprint as most of the content gets loaded subsequently and changes every time you use it.

All in all it is a sliding scale though as for example an image gallery can easily become an application if it allows you to upload your own images or edit and remix the existing ones in your browser. That is one of the main benefits of web technology – it is very flexible and allows for quick and simple changes to the final product without being hindered by a complex compilation, packaging and deployment process.

What kind of features should a web site have to be qualified as a web application?

Again, there are many things to consider. One main thing is that an app does one thing and one thing well. It is there to help you do something.

Technically it should behave like the fat client apps of old: it should retain my state and settings, allow me to customise the interface to my needs and it needs to work offline. The latter is not a technical necessity in terms of definition but to me crucial usability. Seeing how flaky our connections are – I am writing this on a plane – our apps should make people as effective as possible and this means we shouldn’t be dependent on a connection. The interface should be usable whilst we are off the grid and sync as soon as we go online.

Customisation and personalisation of the interface and interactivity to me make an application. This could just mean a game where I can change my character and get extras the more I play. A proper “web” application to me also should use the web whenever it can. For example I am very frustrated with playing a game on my phone and when I go to my tablet playing the same game my score and achievements aren’t synced although the device knows who I am. Why be online then?

A lot of badly designed web apps are just wrappers for content like a news feed. For example turning a blog into an app is a pointless exercise. It just adds the overhead of having to install the wrapper, update and uninstall it when I am fed up with the blog but doesn’t give me the “do something” part that defines an app.

If I don’t interact with it other than reading there is no point in making it an app. You lose a lot of the flexibility of the web when packaging HTML as native apps with the most important thing being opaque updates. An app that is hosted on the web can be patched and upgraded without the end user having to download megabytes of data. That is incredibly useful on slow or flaky connections. Instead of the whole app you only download the changes.

What is the difference between a mobile-friendly site and a mobile web app?

A mobile-friendly site is a web site that detects the capabilities of the device it is displayed on and doesn’t make assumptions about how big my screen is and that I have a mouse and keyboard or not. It runs in the browser of the device and is thus hindered by its limitations – which in the case of older Android devices are quite limited indeed. It caters to the mobile space in terms of display changes – single column display, larger fonts, larger buttons.

A mobile web app is an application built with web technologies running in an own, full-screen wrapper and not inside a browser. It takes advantage of all the things the current environment allows for. For example it stores content and itself locally instead of having to be online and requesting everything new every time you open it. It can access the special features of certain environments like swipe gestures, accelerometer for interaction or accessing the camera to get content in. Its purpose is for me to do things with it and not just visit it like I visit a web site.

What would you consider is the key feature that made HTML5 (as opposed to Flash and Java) the number 1 choice for developing mobile web apps?

Flexibility is the super power of HTML5. It is easy to make an HTML5 app adapt to a new environment and to make an interface that shows and hides content and functionality dependent on the capability of the device or environment. Both Java and Flash are not “web native”, meaning you need to initiate and execute an own code environment inside a browser before you see the results. This hinders interactivity of the containing document and the content of the Flash movie or Java Applet. Whilst both Java and Flash have incredibly good development tools and capabilities once they are available they are very demanding to the hardware they run on. That’s why iOS devices don’t have Flash. There is already a fully capable environment available – the browser – and executing another one inside it means using a lot of resources. On mobile devices this means shorter battery life and the device heating up quickly.

With HTML5 we have the opportunity to improve what mobile, web enabled devices already have to have – a browsing environment. These are available open source and for free (Firefox, Chromium and others) and developers can build apps without needing to sign up for one company and get their SDK to get started.

All environments have their uses and there are things that are better done in Flash than in HTML5. Betting on open technologies and browsers means though that you are very likely to be flexible enough to cater to the next environment around the corner. The web has always evolved and mutated around the needs of the market. That’s why multimedia in HTML5 is just another element of the document and not a black hole that can not interact with the rest of the browser or the document.

Where would you say the mobile web is heading? Do you see a future for the mobile web?

There is no mobile web. There is the web. Right now we start consuming it more and more on mobile devices. That is cool. And the web is totally ready for that as it is flexible enough to adapt.

If you use web standard technologies to build applications that expect a certain device, a fixed size of a screen, a special way of user interaction or expect a fast connection you are building a very limited and very quickly outdated piece of software.

Over the last years we should have learned that hardware is a commodity and susceptible to very sudden change. An amazing piece of hardware that is the hot new thing now can tomorrow be embarrassingly outdated.

When you build your web apps to only cater for that you try to be native code and that is a race you can not win. A lot of native apps are built to promote a new piece of hardware and to get people to upgrade. That is a very old technique of selling more products called in-built obsolescence.

Web apps should be beyond this. Our job is to give end users the best possible experience on the current devices but not make them a necessity. We did this mistake in the past which is why you see web applications that tell you that you need a “modern browser like Internet Explorer 6” and “at least 800×600 resolution”.

Native apps on high-end devices are doing really well right now, but I can foresee that people will get bored of having to buy a new phone every year just to get new functionality that is not that important when you consider the cost. The web will stay and always be the open alternative for those who want to consume and create on their own terms instead of being dependent on the success or goals of a certain company.

Justin Wood: SeaMonkey 2.18 – Where are you?

Callek — Tue, 14 May 2013 22:22:06 +0000

Hey Everyone,

So SeaMonkey 2.18 was supposed to be out today, so where is it?

We have had a hardware error in the systems that allow us to reliably generate the release — Without these systems anything we create will be of unknown quality/stability.

In order to meet our own quality and stability requirements we are NOT releasing SeaMonkey 2.18.

While there is a chance we could have these systems back up in time to do an intermediate release (say something corresponding to a possible Gecko 21.0.1) we can not promise nor plan for it at this time.

We are actively working on repairing the system and its data, once that is complete we will go forth with a new BETA based on the SeaMonkey 2.19 train, and we expect to release SeaMonkey 2.19 on time, on June 25′th.

We thank you for your understanding.

Aza Raskin: Psychological Pitfalls And Lessons of A Designer-Founder

Aza Raskin — Tue, 14 May 2013 19:23:43 +0000

It’s an exceptional time to be a product person and a founder: we are collectively responsible for—and a part of—inventing the future. In the last ten years, design has changed the face of consumer electronics. That change has impacted the way we live, from how we communicate to how we get around.

I started Massive Health as a designer and a founder. These are the most important lessons and psychological pitfalls I learned. They apply to any founder or manager who is also a creative.

Your Job Isn’t To Make A Great Product

As a founder, your job isn’t to make a great product. It’s to build a great team that makes great products. You are who you hire.

If you love doing something, under no condition should you start a VC-backed company to do more of it. You won’t. You are going to spend all of your time recruiting, fundraising, recruiting, aligning team vision, recruiting, and figuring out which fires you can safely ignore.

To maintain your psychological health, you’ll need to learn how to shift the fufillment you get from making to the fufillment of enabling a team to make. You’ll be making vicariously, not making directly. You’ll have to come to terms and internalize it or else your lack of emotional fufillment will trickle down to your team.

Very early on, you’ll be able to make stuff. Enjoy it. It won’t last. In fact, making stuff as your team grows is often more harmful than helpful.

The Dangers of “Helping” & The Power of Words

Because you will spend your time building a great team that builds great product, the moments that you directly work on product will be rare. The rarity will make it insanely tempting—and it will feel insanely good—to get your hands dirty by fixing an interaction flow, pushing around some pixels, doing an information re-architecture, or whatever you may be passionate about. Here’s the thing: that can be insanely disruptive unless handled carefully.

Here’s why:

1

You are a founder, which means each word you say lands like an anvil. Even in a very small company, and especially in a larger one, it takes fortitude and courage for a team member to honestly critique your work. The courage required isn’t a one-time cost. It’s incurred every single time. By nature of being a founder, you are used to saying things with charisma and force and you will undoubtedly be excited by your solution and argue for it. This just makes it worse. A final note: it doesn’t matter how nice you are, or how close you are to your team. As a founder, your words are always more powerful than you think.

2

It’s easy for a brainstorm to be taken as direction. I talked to Tony Fadell of Nest and iPod fame about this topic: To this day, in every conversation he has—every—he tries to reiterate whether he is brainstorming or setting direction. Putting this into practice, you’ll feel like you are repeating yourself. That everyone gets it. Don’t stop. Keep saying it. Get used to saying it.

3

When you swoop in to help with complex problems, the person working on the problem has all the context and knows the pitfalls. You’ve hired this person because they are amazing at their job. Jumping in and then advocating for your ideas is often interpreted as a lack of trust in their ability. This can be a huge morale drain. Not only that, but your suggestions may be shortsighted. You’re jumping in—they’re living it.

You are a founder because of your vision and ability. Your input is, in fact, critical.

So what should you do? Define the problem and its constraints. Framing the problem is the most important step in helping your team find the right solution. You can and should point out what’s wrong with a design or product (sandwiching it with praise), just don’t prescribe how to fix it. Never give a solution unless a team member comes to you for help.

Hype can be psychologically compromising

Massive Health launched with a fair amount of hype. We were glad for it—it gave us good brand recognition while hiring, which is the most valuable outcome of PR for a pre-launch startup. But, it’ll hurt in ways you aren’t expecting. Think back to Color and how the hype overshadowed their product; it set expectations so high that whatever they made probably wouldn’t be good enough. When their product launched, the market thrashed them. That’s the obvious way hype hurts, and it’s not even the one I’m talking about.

The more subtle (and more damaging) way that hype hurts is psychologically. The hype nestled in the back of your mind is constantly tugging, saying that your product is never good enough to launch. That it isn’t big enough. That it isn’t ground-breaking enough.

The single most important thing a founder has to know is where to focus his team’s energy: what is and isn’t safe to ignore, which shortcuts are helpful or harmful, and where and when to invest in absolute perfection.

Hype compromises that ability.

Looking back, this was a major factor in launching only one of the awesome products we were working on at Massive. As a founder, I let perfect be the enemy of the good.

Ego And Reputation

A corollary to the psychological pressure of hype is ego. All product people learn the painful process of divorcing ideas from ego. That’s how great products are made. As a founder, you’ll have to learn how to divorce your ego from your company’s success.

You’ll live with knowing that you are personally judged based upon your company’s performance. That your reputation is tied to your company’s reputation. You’ll no longer be judged on your own work, but your team’s work. It will make you want to get too directly involved in the design. And, as we know, that’s dangerous.

Early on at Massive Health, I did not articulate these worries clearly or directly. It was just a fuzzy haze of emotion that confused my decision-making. Once I was able to articulate what was going on, it helped me mitigate its effects.

Be aware that ego can and will get in the way and be ready to fight it.

So Being A Designer and A Founder Sucks?

Not at all. It gives you huge strategic advantages. The Designer Fund articulates that well.

I founded Massive Health because there was a storm brewing: just over the horizon health was about to become consumerized; that quantified self would escape its geeky beginings and morph to become mainstream; that the Apples and Googles of consumer health didn’t yet exist but were about to. At the center of it all was design; and that design-focus is enabling us, now as a part of Jawbone, to invent the future.

The more you know.

Alon Zakai: The Elusive Universal Web Bytecode

noreply@blogger.com (azakai) — Tue, 14 May 2013 18:09:16 +0000

It's often said that the web needs a bytecode. For example, the very first comment in a very recent article on video codecs on the web says

A proper standardized bytecode for browsers would (most likely) allow developers a broader range of languages to choose from as well as hiding the source code from the browser/viewer (if that's good or not is subjective of course).

And other comments continue with

Just to throw a random idea out there: LLVM bytecode. That infrastructure already exists, and you get to use the ton of languages that already have a frontend for it (and more in the future, I'm sure).
[..]
I also despise javascript as a language and wish someone would hurry up replacing it with a bytecode so we can use decent languages again.
[..]
Put a proper bytecode engine in the browser instead, and those people that love javascript for some unknowable reason could still use it, and the rest of us that use serious languages could use them too.
[..]
Honestly, .Net/Mono would probably be the best bet. It's mature, there are tons of languages targeting it, and it runs pretty much everywhere already as fast as native code

Ignoring the nonproductive JS-hating comments, basically the point is that people want to use various languages on the web, and they want those languages to run fast. Bytecode VMs have been very popular since Java in the 90's, and they show that multiple languages can run in a single VM while maintaining good performance, so asking for a bytecode for the web seems to make sense at first glance.

But already in the quotes above we see the first problem: Some people want one bytecode, others want another, for various reasons. Some people just like the languages on one VM more than another. Some bytecode VMs are proprietary or patented or tightly controlled by a single corporation, and some people don't like some of those things. So we don't actually have a candidate for a single universal bytecode for the web. What we have is a hope for an ideal bytecode - and multiple potential candidates.

Perhaps though not all of the candidates are relevant? We need to pin down the criteria for determining what is a "web bytecode". The requirements as mentioned by those requesting it include

Support all the languages

Run code at high speed

To those we can add two additional requirements that are not mentioned in the above quotations, but are often heard:

Be a convenient compiler target
Have a compact format for transfer

In addition we must add the requirements that anything that runs on the web must fulfill,

Be standardized

Be platform-independent

Be secure

JavaScript can already do the last three (it's already on the web, so it has to). Can it do the first four? I would say yes:

Support all the languages: A huge list of languages can compile into JavaScript, and that includes major ones like C, C++, Java, C#, LLVM bytecode, and so forth. There are some rough edges - often porting an app requires changing some amount of code - but nothing that can't be improved on with more work, if the relevant communities focus on it. C++ compilers into JavaScript like Emscripten and Mandreel have years of work put into them and are fairly mature (for example see the Emscripten list of demos). GWT (for Java) has likewise been used in production for many years; the situation for C# is perhaps not quite as good, but improving, and even things like Qt can be compiled into JavaScript. For C#, Qt, etc., it really just depends on the relevant community being focused on the web as one of its targets: We know how to do this stuff, and we know it can work.
Run code at high speed: It turns out that C++ compiled to JavaScript can run at about half the speed of native code, which in some cases outperforms Java, and is expected to get better still. Those numbers are when using the asm.js subset of JavaScript, which basically structures the compiler output into something that is easier for a JS engine to optimize. It's still JavaScript, so it runs everywhere and has full backwards compatibility, but it can be run at near-native speed already today.
Be a convenient compiler target: First of all, the long list of languages from before shows that many people have successfully targeted JavaScript. That's the best proof that JavaScript is a practical compiler target. Also, there are many languages that compile into either C or LLVM bytecode, and we have more than one compiler capable of compiling those to the web, and one of them is open source, so all those languages have an easy path. Finally, while compiling into a "high-level" language like JavaScript is quite convenient, there are downsides, in particular the lack of support for low-level control flow primitives like goto; however, this is addressed by reusable open source libraries like the Relooper.
Have a compact format for transfer: It seems intuitive that a high-level language like JavaScript cannot be compact - it's human-readable, after all. But it turns out though that JS as a compiler target is already quite small, in fact comparable to native code when both are gzipped. Also, even in the largest and most challenging examples, like Unreal Engine 3, the time spent to parse JS into an AST does not need to be high. For example, in that demo it takes just 10 seconds on my machine to both parse and fully optimize the output of over 1 million lines of C++ (remember that much of that optimization time would need to be done no matter what format the code is in, because it has to be a portable one).

So arguably JavaScript is already very close to providing what a bytecode VM is supposed to offer, as listed in the 7 requirements above. And of course this is not the first time that has been said, see here for a previous discussion from November 2010. In the 2.5 years since that link, the case for that approach has gotten significantly stronger, for example, JavaScript's performance on compiled code has improved substantially, and compilers to JavaScript can compile very large C++ applications like Unreal Engine 3, both as mentioned before. At this point the main missing pieces are, first (as already mentioned) improving language support for ones not yet fully mature, and second, a few platform limitations that affect performance, notably lack of SIMD and threads with shared state.

Can JavaScript fill the gaps of SIMD and mutable-memory threads? Time will tell, and I think these things would take significant effort, but I believe it is clear that to standardize them would be orders of magnitude simpler and more realistic than to standardize a completely new bytecode. So a bytecode has no advantage there.

Some of the motivation for a new bytecode appears to come from an elegance standpoint: "JavaScript is hackish", "asm.js is a hack", and so forth, but a new from-scratch bytecode would be (presumably) a thing of perfection. That's an understandable sentiment, but technology has plenty of such things, witness the persistence of x86, C++, and so forth (some would add imperative programming to that list). It's not only true of technology but human civilization as well, for example no natural language has the elegance of Esperanto, and our currently-standing legal and political systems are far from what a from-scratch redesign would arrive at. But large long-standing institutions are easier to improve continuously rather than to completely replace. I think it's not surprising that that's true for the web as well.

(Note that I'm not saying we shouldn't try. We should. But we shouldn't stop trying at the same time to also improve the current situation in a gradual way. My point is that the latter is more likely to succeed.)

Elegance aside, could a from-scratch VM be better than JavaScript? In some ways of course it could, like any redesign from scratch of anything. But I'm not sure that it could fundamentally be better in substantial ways. The main problem is that we just don't know how to create a perfect "one bytecode to rule them all" that is

Fast - runs all languages at their maximal speed
Portable - runs on all CPUs and OSes
Safe - sandboxable so it cannot be used to get control of users' machines

The elusive perfect universal bytecode would need to do all three, but it seems to me that we can only pick two.

Why is this so, when supposedly the CLR and JVM show that the trifecta is possible? The fact is that they do not, if you really take "fast" to mean what I wrote above, which is "runs all languages at their maximal speed" - that's what I mean by "perfect" in the context of the last paragraph. For example, you can run JavaScript on the JVM, but it won't come close to the speed of a modern JS VM. (There are examples of promising work like SPUR, but that was done before the leaps in JS performance that came with CrankShaft, TypeInference, IonMonkey, DFG, etc.).

The basic problem is that to run a dynamic language at full speed, you need to do the things that JavaScript engines, LuaJIT, etc. do, which includes self-modifying code (architecture-specific PICs), or even things like entire interpreters in handwritten optimized assembly. Making those things portable and safe is quite hard - when you make them portable and safe, you make them more generic pretty much by definition. But CPUs have significant-enough differences that doing generic things can lead to slower performance.

The problems don't stop there. A single "bytecode to rule them all" must make some decisions as to its basic types. LuaJIT and several JavaScript VMs represent numbers using a form of NaNboxing, which uses invalid values in doubles to store other types of values. Deciding to NaNbox (and in what way) or not NaNbox is typically a design desicion for an entire VM. NaNboxing might be well and good for JS and Lua, but it might slow down other languages. Another example is how strings are implemented: IronPython, Python on .NET, ran into issues with how Python expects strings to work as opposed to .NET.

Yet another area where decisions must be made is garbage collection. Different languages have different patterns of usage, both determined by the language itself and the culture around the language. For example, the new garbage collector planned for LuaJIT 3.0, a complete redesign from scratch, is not going to be a copying GC, but in other VMs there are copying GCs. Another concern is finalization: Some languages allow hooking into object destruction, either before or after the object is GC'd, while others disallow such things entirely. A design decision on that matter has implications for performance. So it is doubtful that a single GC could be truly optimal for all languages, in the sense of being "perfect" and letting everything run at maximal speed.

So any VM must make decisions and tradeoffs about fundamental features. There is no obvious optimal solution that is right for everything. If there were, all VMs would look the same, but they very much do not. Even relatively similar VMs like the JVM and CLR (which are similar for obvious historic reasons) have fundamental differences.

Perhaps a single VM could include all the possible basic types - both "normal" doubles and ints, and NaNboxed doubles? Both Pascal-type strings and C-type strings? Both asynchronous and synchronous APIs for everything? Of course all these things are possible, but they make things much more complicated. If you really want to squeeze every last ounce of performance out of your VM, you should keep it simple - that's what LuaJIT does, and very well. Trying to support all the things will lead to compromises, which goes against the goal of a VM that "runs all languages at their maximal speed".

(Of course there is one way to support all the things at maximal speed: Use a native platform as your VM. x86 can run Java, LuaJIT and JS all at maximal speed almost by definition. It can even be sandboxed in various ways. But it has lost the third property of being platform-independent.)

Could we perhaps just add another VM like the CLR alongside JavaScript, and get the best of both worlds that way, instead of putting everything we need in one VM? That sounds like an interesting idea at first, but it has technical difficulties and downsides, is complex, and would likely regress existing performance.

Do we actually need "maximal speed"? How about just "reasonable speed"? Definitely, we can't hold out for some perfect VM that can do it all. In the last few paragraphs I've been talking about a "perfect" bytecode VM that can run everything at maximal speed. My point is that it's important to realize that there is no such VM. But, with some compromise we definitely can have a VM that runs many things at very high speeds. Examples of such VMs are the JVM, CLR, and as mentioned before JavaScript VMs as well, since they can run one very popular dynamic language at maximal speed, and they can run statically typed code compiled from C++ about as well or even better than some bytecode VMs (with the already-discussed caveats of SIMD and shared-mutable threads).

For that reason, switching from JavaScript to another VM would not be a strictly better solution in all respects, but instead just shift us to another compromise. For example, JavaScript itself would be slower on the CLR, but C# would be faster, and I'm not sure which of the two can run C++ faster, but my bet is that both can run it at about the same speed.

So I don't think there is much to gain, technically speaking, from considering a new bytecode for the web. The only clear advantage such an approach could give is perhaps a more elegant solution, if we started from scratch and designed a new solution with less baggage. That's an appealing idea, and in general elegance often leads to better results, but as argued earlier there would likely be no significant technical advantages to elegance in this particular case - so it would be elegance for elegance's sake.

I purposefully said we don't need a new bytecode in the last paragraph. We already have JavaScript, which I have claimed is quite close to providing all the advantages that a bytecode VM could. Note that this wasn't entirely expected - not any language can in a straightforward way be transformed into a more general target for other languages. It just so happens though that JavaScript did have just enough low-level support (bitwise operations being 32-bit, for example) to make it a practical C/C++/LLVM IR compiler target, which made it worth investing in projects like the Relooper that work around some of its other limitations. Combined with the already ongoing speed competition among JavaScript engines, the result is that we now have JavaScript VMs that can run multiple languages at high speed.

In summary, we already have what practically amounts to a bytecode VM in our browsers. Work is not complete, though: While we can port many languages very well right now, support for other languages is not quite there yet. If you like a language that is not yet supported on the web, and you want it to run on the web, please contribute to the relevant open source project working on doing that (or start one if there isn't one already). There is no silver bullet here - no other bytecode VM that if only we decided on it, we would have all the languages and libraries we want on the web, "for free" - there is work that needs to be done. But in recent years we have made huge amounts of progress in this area, both in infrastructure for compiling code to JavaScript and in improvements to JavaScript VMs themselves. Let's work together to finish that for all languages.

Niko Matsakis: Procedures, continued

Tue, 14 May 2013 13:20:00 +0000

So, I didn’t actually mean to post that previous post, I had intended to think more on the idea. But oh well, cat’s out of the bag. In any case, I’ve been thinking about the “closures” vs “procedures” idea that I jotted down there and decided to try and elaborate on it a bit more, since I find it has a lot of appeal. In particular I think that the current collection of closure types is addressing too many distinct use cases and the result is confusing.

UPDATE 2013.05.14: Edited to tweak various errors and to add some variations at the end that I prefer.

Today: by-reference vs copying closures

Today we offer three different kinds of closures (&fn, @fn, and ~fn), but these closures can really be divided into two basic categories: by-reference and copying closures. A by-reference closure is the usual kind: it is allocated on the stack and has full access to the variables in the creating stack frame. It can read them, write them, and borrow them. These are used with for loops and the like.

Copying closures, on the other hand, are somewhat different. They are not tied to any particular stack frame. Instead, they copy the current values of the variables which they close over into their environment (like all the default Rust copies, this is a shallow copy, so if the value being closed over contains ~ pointers, it will no longer be accessible from the creator). These closures are used primarily as task bodies and for futures. There are some scattered uses of @fn closures in the compiler but as far as I can tell they are all legacy code that should eventually be purged and rewritten to use traits (i.e., the visitor, the AST folder).

Loosely speaking, a &fn closure is by-reference and @fn and ~fn closures are copying closures. But this is not strictly true. In fact, an the &fn type can be either a by-reference closure or a copying closure, because you are permitted to borrow a @fn or ~fn to a &fn. So the type in isolation does not tell you whether a closure is by-reference or not. In fact, there is no explicit indication at all—instead, when you create a closure today (i.e., with a |x, y| ...) expression, the compiler infers based on the expected types whether this should be a by-reference closure or a copying closure. Because the semantics of these two vary greatly, I find this potentially quite confusing and unfortunate.

Tomorrow (perhaps): closures and procedures

In general, I would prefer to draw a starker line between copying and by-reference closures. I propose to use the term closure to refer only to by-reference, stack-allocated closures. We could then use another term, perhaps procedure, to refer to the copying closures. This would mean that our type hierarchy would look like:

T = S               // sized types
  | U               // unsized types
S = fn(S*) -> S     // closures (*)
  | &'r T           // region ptr
  | @T              // managed ptr
  | ~T              // unique ptr
  | [S, ..N]        // fixed-length array
  | uint            // scalars
  | ...
U = [S]             // vectors
  | str             // string
  | Trait           // existential ("exists S:Trait.S")
  | proc(S*) -> S   // procedures (*)

This chart is basically the same as the one you will find in the dynamically sized types post from before with one crucial difference: closure types have been split from procedures, and closure types have moved into the category of sized types, meaning that you no longer write an explicit sigil when you use one. This is because the representation of a closure would always be a pair of a borrowed pointer into the stack and a function pointer: the type has a fixed size (two words) and requires no memory allocation.

I have chosen to leave procedures as unsized, since a procedure must allocate memory on the heap, and this allows the user to select which heap is used; in earlier drafts of this idea, I had modified procedures to implicit use the exchange heap, meaning that a type like proc() always represented an exchange heap allocation. But I think it’s more consistent to have that type be written ~proc, and it maintains the general Rust invariant “you don’t have allocation unless you see a sigil”.

UPDATE: bstrie on IRC asked about fn items, which never have any environment. As today, these would continue to be coercable to either a closure or a procedure.

Closure and procedure expressions

Closures would still be created with the form |x, y| expr. Procedures would be created using the keyword proc: proc(x, y) expr. If desired, we could integrate procedures into do using some syntax like one of the following, depending on whether we wish to make the sigil explicit:

do spawn proc { ... }   // sigil inferred

do spawn ~proc { ... }  // sigil explicit

Closure and procedure types in more detail

The full function or procedure type would look something like this ([] indicates optional content):

 [once] (fn|proc) [:['r] [Bounds]] <'a...> (S*) -> S
 ^~~~~^           ^~~~~~~~~~~~~~~^ ^~~~~~^ ^~~^    ^
   |                     |            |     |      |
   |                     |            |     |  Return type
   |                     |            |    Argument types
   |                     |          Bound lifetime names
   |               Lifetime and trait bounds
Onceness

Here the “onceness” indicates whether the closure/procedure can be called more than once. The “lifetime and trait bounds” indicate constraints on the environment. The lifetime bound 'r indicates the minimum lifetime of the variables that the closure/procedure closes over, and the “bounds” (if any) would give bounds on the types of those variables. Finally, you have the argument and return types.

If omitted, the default bounds for a closure would be a fresh lifetime and no type bounds. The default bounds for a procedure would be the static lifetime and Owned.

Use cases

Let’s look briefly at the use cases I listed before.

Higher-order and once functions

Typical uses for higher-order and once functions look much the same as before, but minus a sigil.

impl<T:Sized> for [T] {
    pub fn map<U:Sized>(f: fn(&T) -> U) -> ~[U] { ... }
                        // ^~~~~~~~~~~
}

impl<T:Sized> for Option<T> {
    // `each` on an option type can only execute at most once:
    pub fn each(f: once fn(&T) -> bool) -> bool { ... }
                // ^~~~~~~~~~~~~~~~~~~
    }
}

For contrast, these are &fn(&T) -> U and &once fn(&T) -> bool today.

Sendable functions and sendable once functions

Here is an example of a sendable once function:

fn spawn(f: ~once proc()) {...}
         // ^~~~~~~~~~

As we saw before, one would write one of the following to call this function:

do spawn proc { ... }
spawn(proc { ... })

Creating a future would look like future(proc expr) (vs future(|| expr) today).

Const closures

One could still use const closures to achieve lightweight parallelism:

impl<T:Sized> for [T] {
    pub fn par_map<U:Sized>(f: fn:Const(&T) -> U) -> bool { ... }
                            // ^~~~~~~~~~~~~~~~~
}

However, I have been thinking that we’ll have to be careful here, we need some way to guarantee that the closure does not move from its environment and then replace the moved value. Today this is illegal, but if we can prevent closures from recursing (which we must do anyhow) then we could make such moves legal, and it would be useful sometimes. On simple solution is to stay that if the closure type has a Const bound, moves are illegal, but it’s a bit…ad-hoc, since the bounds are only supposed to be constraining the types of the variables that are closed over. Still, it might be good enough.

Sendable const functions and combinators

As I argued before, I think these are not important use cases, but with procedures they actually work out fine (though not with variations 2 and 3 below). A sendable const function can be expressed with the type ~proc:Owned+Const(), which is complex, but then it is a complex idea. Combinator types would likely look like @proc or @proc:'r, in the case where the combinator closes over borrowed data.

Variation #1: Leaving procedures out of the core language

In fact, I think proc types need not be built into the language, you could model them with traits, though you’d probably want a macro like proc!(...) for defining the proc body. This would also mean the procedures can’t be used with do form.

Variation #2: Limit procedures to execute once

I don’t know of any (good) uses cases for non-once procedures. I think they should just always be once. This would mean that the only closure types that are commonly needed would be:

fn(T) – normal higher-order functions
once fn(T) – higher-order functions that execute at most once
~proc(T) – procedures

Because procedures can always be desugared into a struct and a trait, this would not lose no expressiveness.

Variation #3: Limit procedures to execute once and use exchange heap

For maximum streamlining, we could make proc implicitly use ~, in which case it would be written:

fn(T) – normal higher-order functions
once fn(T) – higher-order functions that execute at most once
proc(T) – procedures

These types read pretty well, I think.

Summary

I have long been unsatisfied with the implicit and confusing divide between “by reference” and “copying” closures. Splitting them into two concepts seems to address a lot of issues and be an overall win to me.

Tim Taubert: Scotland.JS 2013 in Edinburgh

Tue, 14 May 2013 06:00:00 +0000

I got to spend Wednesday through Friday in Edinburgh last week to attend Scotland.JS. Edinburgh is a lovely city and I will definitely return to get to know it better. It has great people, beers, food and even a castle - what could one want more?

I arrived on Wednesday, just in time for the TechMeetup. It took place in the Appleton Tower and had about 70-80 attendees with quite diverse technical backgrounds. I had a very interesting talk to a freelance patent attorney and also met a couple of people I knew from recent conferences and meetups in Berlin. After we were out of free pizza and beers we met most of the Scotland.JS attendees in a pub right next to the Appleton Tower. Cue more beers and tech conversations.

Thursday was the first day of Scotland.JS that started with a great keynote from Jan Lehnardt. The most memorable talk of the day was about Functional Reactive Programming using Bacon.js by Philip Roberts. There of course were a lot more great talks but the possibilities of creating and combining event streams kept my head spinning the most. At the end of the day we again went to a neat pub near the venue and I headed to bed rather early, still a little tired from travel and the day before.

Friday started with a short 20-minute walk through Edinburgh’s old town that I needed to pass on my way to the venue. My favorite talk of the second and last day was held by Dominic Tarr that tought us about the internals of leveldb. I also really enjoyed Mark Boas talking about making audio a first-class citizen of the web - a very entertaining and impressive mix of technologies. A GitHub drinkup in the venue’s own pub was a great way to end the conference - lots of good conversations with Scotland.JS attendees and speakers as well as people from the local tech community and visitors of the Scottish Ruby Conf.

I flew home the very next day and already miss Edinburgh with its neat little alleys and pubs everywhere. The JavaScript community has once again shown its value as a very interesting mix of nice welcoming people coming from so many different technical backgrounds. Like after most conferences I feel a little exhausted and my head is full of ideas for future or current (side-)projects. I will try my best to return next year, it has been a lot of fun!

Niko Matsakis: Mutable fn alternatives

Mon, 13 May 2013 21:31:00 +0000

I’ve been thinking about what I wrote in my last post regarding closures and I am beginning to change my opinion about the correct solution. fn~ just seems so unfortunate. So, besides writing fn~, what are the other options? I just thought I’d write down a few of the other ideas I’ve come up with for later reference. Not saying any of the ideas in this post are good yet.

Just write `&mut fn()`

Maybe it’s not so bad. It is advertising the possibility that the closure may mutate its environment. This would mean that while &fn() is a valid type, it is a type that does not permit the function to be called, much as &&mut (pointer to a mutable borrowed pointer) does not permit the mutable borrowed pointer to be used.

At first I was thinking that there is also a valid interpretation for &fn, meaning a function that does not mutate the variable in its environment, but then I realize that per the DST proposal any &mut fn could be borrowed to &fn, and so that would not be sound.

Remove everything but borrowed closures

We could just only have borrowed closures. The type would be written fn[:bounds]() or once fn[:bounds](). There’d be no need to notate the kind of environment pointer: it’s always a borrowed pointer. All other uses of closures would be expressed using traits and impls.

Mainly this means that code which spawns traits would get somewhat verbose, because you would need to create a struct or some other type to capture all of the upvars. For larger tasks, this is not a big deal, but for some code it could be rather annoying. I imagine futures in particular would become much more verbose; enough so as to be nearly unusable.

On the upside, there’d be no more confusion about whether a closure copies its environment or not (no, it never does). Closure types would be simpler (no need to worry about sigils). You’d write fn() or once fn() in all but the most esoteric cases. The code to manage closures would become much simpler.

Add a new keyword for what is now called an “owned closure”

This is basically the fn~ solution with another name. Rather than writing fn~ to indicate a closure value that owns its environment, we could write proc (for procedure) or something like that. This avoids the annoying “sigil after the name”, at the cost of a new keyword.

Procedures could probably always be single-shot (that is, once). Almost all use cases for them (futures, tasks, etc) are single-shot, and the others could probably be accommodated with traits instead. But we could also distinguish between a proc and a once proc if we wanted.

Procedures would probably be less interoperable with functions, since the name does not particularly suggest interoperability. For example, I imagine you could not use a proc where a fn is expected. I don’t know of any time that this is actually important.

Using a different name also helps to draw a clear line between between “closures” (which reference the variables in the stack frame that created them) and “procedures” (which copy out from that stack frame). I personally would prefer to designate procedures with a different syntax, e.g., proc(x, y) { ... } in place of |x, y| ..., but this is not necessary (as an aside, I had hoped to write some today about why I think our current use of || to designate any kind of closure is troublesome and should be changed, before I realized that we’d have to address this problem I’m thinking over instead).

More ideas?

Ok, that’s most of the more radical ideas I’ve had so far. I’ll have to keep thinking on it.

Gregory Szorc: Build System Status Update 2013-05-14

Mon, 13 May 2013 19:35:00 +0000

I'd like to make an attempt at delivering regular status updates on the Gecko/Firefox build system and related topics. Here we go with the first instance. I'm sure I missed awesomeness. Ping me and I'll add it to the next update.

MozillaBuild Windows build environment updated

Kyle Huey released version 1.7 of our Windows build environment. It contains a newer version of Python and a modern version of Mercurial among other features.

I highly recommend every Windows developer update ASAP. Please note that you will likely encounter Python errors unless you clobber your build.

New submodule and peers

I used my power as module owner to create a submodule of the build config module whose scope is the (largely mechanical) transition of content from Makefile.in to moz.build files. I granted Joey Armstrong and Mike Shal peer status for this module. I would like to eventually see both elevated to build peers of the main build module.

moz.build transition

The following progress has been made:

Mike Shal has converted variables related to defining XPIDL files in bug 818246.
Mike Shal converted MODULE in bug 844654.
Mike Shal converted EXPORTS in bug 846634.
Joey Armstrong converted xpcshell test manifests in bug 844655.
Brian O'Keefe converted PROGRAM in bug 862986.
Mike Shal is about to land conversion of CPPSRCS in bug 864774.

Non-recursive XPIDL generation

In bug 850380 I'm trying to land non-recursive building of XPIDL files. As part of this I'm trying to combine the generation of .xpt and .h for each input .idl file into a single process call because profiling revealed that parsing the IDL consumes most of the CPU time. This shaves a few dozen seconds off of build times.

I have encounterd multiple pymake bugs when developing this patch, which is the primary reason it hasn't landed yet.

WebIDL refactoring

I was looking at my build logs and noticed WebIDL generation was taking longer than I thought it should. I filed bug 861587 to investigate making it faster. While my initial profiling turned out to be wrong, Boris Zbarsky looked into things and discovered that the serialization and deserialization of the parser output was extremely slow. He is currently trying to land a refactor of how WebIDL bindings are handled. The early results look very promising.

I think the bug is a good example of the challenges we face improving the build system, as Boris can surely attest.

Test directory reorganization

Joel Maher is injecting sanity into the naming scheme of test directories in bug 852065.

Manifests for mochitests

Jeff Hammel, Joel Maher, Ted Mielczarek, and I are working out using manifests for mochitests (like xpcshell tests) in bug 852416.

Mach core is now a standalone package

I extracted the mach core to a standalone repository and added it to PyPI.

Mach now categorizes commands in its help output.

Requiring Python 2.7.3

Now that the Windows build environment ships with Python 2.7.4, I've filed bug 870420 to require Python 2.7.3+ to build the tree. We already require Python 2.7.0+. I want to bump the point release because there are many small bug fixes in 2.7.3, especially around Python 3 compatibility.

This is currently blocked on RelEng rolling out 2.7.3 to all the builders.

Eliminating master xpcshell manifest

Now that xpcshell test manifests are defined in moz.build files, we theoretically don't need the master manifest. Joshua Cranmer is working on removing them in bug 869635.

Enabling GTests and dual linking libxul

Benoit Gerard and Mike Hommey are working in bug 844288 to dual link libxul so GTests can eventually be enabled and executed as part of our automation.

This will regress build times since we need to link libxul twice. But, giving C++ developers the ability to write unit tests with a real testing framework is worth it, in my opinion.

ICU landing

ICU was briefly enabled in bug 853301 but then backed out because it broke cross-compiling. It should be on track for enabling in Firefox 24.

Resource monitoring in mozbase

I gave mozbase a class to record system resource usage. I plan to eventually hook this up to the build system so the build system records how long it took to perform key events. This will give us better insight into slow and inefficient parts of the build and will help us track build system speed improvements over time.

Sorted lists in moz.build files

I'm working on requiring lists in moz.build be sorted. Work is happening in bug 863069.

This idea started as a suggestion on the dev-platform list. If anyone has more great ideas, don't hold them back!

Smartmake added to mach

Nicholas Alexander taught mach how to build intelligently by importing some of Josh Matthews' smartmake tool's functionality into the tree.

Source server fixed

Kyle Huey and Ted Mielczarek collaborated to fix the source server.

Auto clobber functionality

Auto clobber functionality was added to the tree. After flirting briefly with on-by-default, we changed it to opt-in. When you encounter it, it will tell you how to enable it.

Faster clobbers on automation

I was looking at build logs and identified we were inefficiently performing clobber.

Massimo Gervasini and Chris AtLee deployed changes to automation to make it more efficient. My measurements showed a Windows try build that took 15 fewer minutes to start - a huge improvement.

Upgrading to Mercurial 2.5.4

RelEng is tracking the global deployment of Mercurial 2.5.4. hg.mozilla.org is currently running 2.0.2 and automation is all over the map. The upgrade should make Mercurial operations faster and more robust across the board.

I'm considering adding code to mach or the build system that prompts the user when her Mercurial is out of date (since an out of date Mercurial can result in a sub-par user experience).

Parallelize reftests

Nathan Froyd is leading an effort to parallelize reftest execution. If he pulls this off, it could shave hours off of the total automation load per checkin. Go Nathan!

Overhaul of MozillaBuild in the works

I am mentoring a pair of interns this summer. I'm still working out the final set of goals, but I'm keen to have one of them overhaul the MozillaBuild Windows development environment. Cross your fingers.

J. Paul Reed: Eulogy for a Founding Father

preed — Mon, 13 May 2013 18:52:48 +0000

About a month ago, I noticed a tweet from Coop:

Pouring out a little liquor for tinderbox today. Drinking the rest, because, you know, tinderbox.

It linked to a mozilla.dev.planning post describing the plan to end-of-life Tinderbox¹.

As one of a handful of people who was required in an employment-capacity to support Tinderbox in production^2,3, I can certainly understand the elation at getting rid of the aged continuous integration system. It hasn’t changed much (or seen much maintenance for that matter) since its original open source release fifteen years ago and certainly had plenty of warts⁴.

Having said that, part of me is sad at the… glee, for lack of a better word, at its demise.

Tinderbox is certainly antiquated by any modern standard, but it should not be forgotten that, having been released in 1998, it is very much the grandfather of continuous integration systems.

It may have “sucked,” but it facilitated a workflow and nurtured an ethos that is not only extremely important, but taken for granted today: namely the notion that individual developers should be “on the hook” when checking in, and have a responsibility to their peers to monitor the build and make sure the tree “stays green.”

It was Tinderbox that was largely responsible for introducing a generation of software engineers to this now-commonplace concept, and helping to get a previous generation of engineers to care about such things. Mozilla was the poster-child user for Tinderbox, but I know of at least VMware and Yahoo who used it years before Hudson/Jenkins and Buildbot existed.

Beyond that, it sports features that those systems TO THIS DAY do not:

Tinderbox put its build logic in the clients, and had them report to the server via email; this may seem odd now, but the asynchronous nature of that data flow meant that Tinderbox was surprisingly tolerant to network failures, something Jenkins and (especially) Buildbot both continue to handle horribly.
Tinderbox supported JSON output⁵ that allowed the development of its successor, tbpl, and other tools; it was one of the first CI systems to make its collection of data consumable in such a transparent format.
As mentioned above, Tinderbox was modular, separating out the logic of the client from the server, and using a simple API to communicate between them. This meant it was trivial to write a Tinderbox client in whatever language you preferred to write one in, as long as it output something the Tinderbox server expected; this is huge, and a major distinction between today’s systems, which follow a command-and-control model. I personally know Jenkins users who tell horror stories about 40+ minute startup times when tens-of-thousands of clients were involved. Tinderbox, by design, never had this problem.
Tinderbox facilitated continuous integration for open source projects in a way that still has yet to be replicated; say what you will about using email as a communication mechanism, but it allowed outside parties to set up and maintain “weird” platforms corporate sponsors of an open source project didn’t feel like investing in⁶, and yet still provide those CI results to the community in a highly-visible, inclusive way⁷. Good luck getting those Jenkins and Buildbot ports open through the corporate firewall today. And for all the hype about Git’s decentralization, there isn’t a continuous integration tool today built around a decentralized model like Tinderbox was.

It’s easy to hate on tools like Tinderbox when you purposefully ignore the context in which they were created⁸: Richard Stallman’s venerable GCC compiler was “a pile of crap”… if you forget it was the most widely used compiler before the EGCS fork, over a decade before “Clang” was more than a Klingon General⁹. “CVS sucks!” … yeah, compared to Git, Subversion, and Perforce. But not compared to RCS or Microsoft Source Safe. “Shell scripts are old-school garbage!”… except if you were trying to automate something in the 80s and early 90s, and didn’t want to have to learn about C pointers to do it.

It’s unfortunate that it seems unlikely Tinderbox will ever get the respect it truly deserves as one of the founding fathers of this idea of continuous integration that most of us take for granted today.

But perhaps that’s merely another example of Alan Kay’s indictment of our industry for its “disdain of history.”

In any event, to paraphrase Apollo 13 Mission control: “Farewell Tinderbox… and we thank you.”

_______________
¹ And the requisite bug, which you can conveniently find find by searching for keyword “tinderbox-death”↵
² Coop also holds this distinction↵
³ And as one who also has a number of (now embarrassing) checkins into that code base↵
⁴ Including some pretty bad ones ↵
⁵ And possibly others↵
⁶ Firefox on AIX, anyone?↵
⁷ It’s telling that the last few projects still using Mozilla’s Tinderbox instance fall pretty squarely into this category: Camino and Bugzilla↵
⁸ Especially when they’re (merely) broadly used for fifteen years, but not actively maintained↵
⁹ Not really ↵

Eric Shepherd: Documentation reviews in a snap on MDN

sheppy — Mon, 13 May 2013 17:42:53 +0000

A key step in the documentation process is the review. The Mozilla Developer Network (MDN) developer documentation team aspires to perform two types of review on each article on the wiki:

A technical review involves having someone with technical expertise in the article’s subject area read the article and ensure that it’s factually and technically accurate. This means making sure our explanations of the subject matter are right and make sense to the developers, as well as ensuring that any code samples are implemented correctly.

An editorial review is where someone reads the article and corrects any typos, grammatical mistakes, or stylistic or layout problems they can find.

When writing an article (or editing an existing article), there are checkboxes that let you request each of these types of review on your article; indeed, both of these flags are enabled by default for any newly-created page. Yellow banners are displayed at the top of the page to let readers know that these flags are set, as appropriate.

In the past, the only way to remove the review requests (for example, if you’ve read the article and didn’t see any factual errors) was to click the Edit button, scroll to the checkbox for the editorial review, toggle it off, and save the article.

Now there’s a better way!

Now, on pages with one or both of these review flags set, you’ll see a new “Quick review” box at the bottom of the article. This box looks like this:

The MDN Quick Review box.

All you have to do if you’ve completed a review is to turn on the appropriate checkbox(es) and click “Confirm Reviews.” The review is recorded and you have our gratitude for your help!

If you’d like to actually hunt down and take care of articles in need of review, there’s an easy way to find them:

I’m hopeful that this streamlined approach to reviews will help encourage subject-matter experts to review content, and will help casual users of MDN get involved by reviewing content as well. We have some improvements planned for future updates to MDN that will make this even better, but this is a great start, and I’m grateful to our development team for putting this together!

Gregory Szorc: Mozilla Build System Brain Dump

Mon, 13 May 2013 17:25:00 +0000

I hold a lot of context in my head when it comes to the future of Mozilla's build system and the interaction with it. I wanted to perform a brain dump of sorts so people have an idea of where I'm coming from when I inevitably propose radical changes.

The sad state of build system interaction and the history of mach

I believe that Mozilla's build system has had a poor developer experience for as long as there has been a Mozilla build system. Getting started with Firefox development was a rite of passage. It required following (often out-of-date) directions on MDN. It required finding pages through MDN search or asking other people for info over IRC. It was the kind of process that turned away potential contributors because it was just too damn hard.

mach - while born out of my initial efforts to radically change the build system proper - morphed into a generic command dispatching framework by the time it landed in mozilla-central. It has one overarching purpose: provide a single gateway point for performing common developer tasks (such as building the tree and running tests). The concept was nothing new - individual developers had long coded up scripts and tools to streamline workflows. Some even published these for others to use. What set mach apart was a unified interface for these commands (the mach script in the top directory of a checkout) and that these productivity gains were in the tree and thus easily discoverable and usable by everybody without significant effort (just run mach help).

While mach doesn't yet satisfy everyone's needs, it's slowly growing new features and making developers' lives easier with every one. All of this is happening despite that there is not a single person tasked with working on mach full time. Until a few months ago, mach was largely my work. Recently, Matt Brubeck has been contributing a flurry of enhancements - thanks Matt! Ehsan Akhgari and Nicholas Alexander have contributed a few commands as well! There are also a few people with a single command to their name. This is fulfilling my original vision of facilitating developers to scratch their own itches by contributing mach commands.

I've noticed more people referencing mach in IRC channels. And, more people get angry when a mach command breaks or changes behavior. So, I consider the mach experiment a success. Is it perfect, no. If it's not good enough for you, please file a bug and/or code up a patch. If nothing else, please tell me: I love to know about everyone's subtle requirements so I can keep them in mind when refactoring the build system and hacking on mach.

The object directory is a black box

One of the ideas I'm trying to advance is that the object directory should be considered a black box for the majority of developers. In my ideal world, developers don't need to look inside the object directory. Instead, they interact with it through condoned and supported tools (like mach).

I say this for a few reasons. First, as the build config module owner I would like the ability to massively refactor the internals of the object directory without disrupting workflows. If people are interacting directly with the object directory, I get significant push back if things change. This inevitably holds back much-needed improvements and triggers resentment towards me, build peers, and the build system. Not a good situation. Whereas if people are indirectly interacting with the object directory, we simply need to maintain a consistent interface (like mach) and nobody should care if things change.

Second, I believe that the methods used when directly interacting with the object directory are often sub-par compared with going through a more intelligent tool and that productivity suffers as a result. For example, when you type make in inside the object directory you need to know to pass -j8, use make vs pymake, and that you also need to build toolkit/library, etc. Also, by invoking make directly, you bypass other handy features, such as automatic compiler warning aggregation (which only happens if you invoke the build system through mach). If you go through a tool like mach, you should automatically get the most ideal experience possible.

In order for this vision to be realized, we need massive improvements to tools like mach to cover the missing workflows that still require direct object directory interaction. We also need people to start using mach. I think increased mach usage comes after mach has established itself as obviously superior to the alternatives (I already believe it offers this for tasks like running tests).

I don't want to force mach upon people but...

Nobody likes when they are forced to change a process that has been familiar for years. Developers especially. I get it. That's why I've always attempted to position mach as an alternative to existing workflows. If you don't like mach, you can always fall back to the previous workflow. Or, you can improve mach (patches more than welcome!). Having gone down the please-use-this-tool-it's-better road before at other organizations, I strongly believe that the best method to incur adoption of a new tool is to gradually sway people through obvious superiority and praise (as opposed to a mandate to switch). I've been trying this approach with mach.

Lately, more and more people have been saying things like we should have the build infrastructure build through mach instead of client.mk and why do we need testsuite-targets.mk when we have mach commands. While I personally feel that client.mk and testsuite-targets.mk are antiquated as a developer-facing interface compared to mach, I'm reluctant to eliminate them because I don't like forcing change on others. That being said, there are compelling reasons to eliminate or at least refactor how they work.

Let's take testsuite-targets.mk as an example. This is the make file that provides the targets to run tests (like make xpcshell-test and make mochitest-browser-chrome). What's interesting about this file is that it's only used in local builds: our automation infrastructure does not use testsuite-targets.mk! Instead, mozharness and the old buildbot configs manually build up the command used to invoke the test harnesses. Initially, the mach commands for running tests simply invoked make targets defined in testsuite-targets.mk. Lately, we've been converting the mach commands to invoke the Python test runners directly. I'd argue that the logic for invoke the test runner only needs to live in one place in the tree. Furthermore as a build module peer, I have little desire to support multiple implementations. Especially considering how fragile they can be.

I think we're trending towards an outcome where mach (or the code behind mach commands) transitions into the authoratitive invocation method and legacy interfaces like client.mk and testsuite-targets.mk are reimplemented to either call mach commands or the same routine that powers them. Hopefully this will be completely transparent to developers.

The future of mozconfigs and environment configuration

mozconfig files are shell scripts used to define variables consumed by the build system. They are the only officially supported mechanism for configuring how the build system works.

I'd argue mozconfig files are a mediocre solution at best. First, there's the issue of mozconfig statements that don't actually do anything. I've seen no-op mozconfig content cargo culted into the in-tree mozconfigs (used for the builder configurations)! Oops. Second, doing things in mozconfig files is just awkward. Defining the object directory requires mk_add_options MOZ_OBJDIR=some-path. What's mk_add_options? If some-path is relative, what is it relative to? While certainly addressable, the documentation on how mozconfig files work is not terrific and fails to explain many pitfalls. Even with proper documentation, there's still the issue of the file format allowing no-op variable assignments to persist.

I'm very tempted to reinvent build configuration as something not mozconfigs. What exactly, I don't know. mach has support for ini-like configuration files. We could certainly have mach and the build system pull configs from the same file.

I'm not sure what's going to happen here. But deprecating mozconfig files as they are today is part of many of the options.

Handling multiple mozconfig files

A lot of developers only have a single mozconfig file (per source tree at least). For these developers, life is easy. You simply install your mozconfig in one of the default locations and it's automagically used when you use mach or client.mk. Easy peasy.

I'm not sure what the relative numbers are, but many developers maintain multiple mozconfig files per source tree. e.g. they'll have one mozconfig to build desktop Firefox and another one for Android. They may have debug variations of each.

Some developers even have a single mozconfig file but leverage the fact that mozconfig files are shell scripts and have their mozconfig dynamically do things depending on the current working directory, value of an environment variable, etc.

I've also seen wrapper scripts that glorify setting environment variables, changing directory, etc and invoke a command.

I've been thinking a lot about providing a common and well-supported solution for switching between active build configurations. Installing mach on $PATH goes a long way to facilitate this. If you are in an object directory, the mozconfig used when that object directory was created is automatically applied. Simple enough. However, I want people to start treating object directories as black boxes. So, I'd rather not see people have their shell inside the object directory.

Whenever I think about solutions, I keep arriving at a virtualenv-like solution. Developers would potentially need to activate a Mozilla build environment (similar to how Windows developers need to launch MozillaBuild). Inside this environment, the shell prompt would contain the name of the current build configuration. Users could switch between configurations using mach switch or some other magic command on the $PATH.

Truth be told, I'm skeptical if people would find this useful. I'm not sure it's that much better than exporting the MOZCONFIG environment variable to define the active config. This one requires more thought.

The integration between the build environment and Python

We use Python extensively in the build system and for common developer tasks. mach is written in Python. moz.build processing is implemented in Python. Most of the test harnesses are written in Python.

Doing practically anything in the tree requires a Python interpreter that knows about all the Python code in the tree and how to load it.

Currently, we have two very similar Python environments. One is a virtualenv created while running configure at the beginning of a build. The other is essentially a cheap knock-off that mach creates when it is launched.

At some point I'd like to consolidate these Python environments. From any Python process we should have a way to automatically bootstrap/activate into a well-defined Python environment. This certainly sounds like establishing a unified Python virtualenv used by both the build system and mach.

Unfortunately, things aren't straightforward. The virtualenv today is constructed in the object directory. How do we determine the current object directory? By loading the mozconfig file. How do we do that? Well, if you are mach, we use Python. And, how does mach know where to find the code to load the mozconfig file? You can see the dilemma here.

A related issue is that of portable build environments. Currently, a lot of our automation recreates the build system's virtualenv from its own configuration (not that from the source tree). This has and will continue to bite us. We'd really like to package up the virtualenv (or at least its config) with tests so there is no potential for discrepancy.

The inner workings of how we integrate with Python should be invisible to most developers. But, I figured I'd capture it here because it's an annoying problem. And, it's also related to an activated build environment. What if we required all developers to activate their shell with a Mozilla build environment (like we do on Windows)? Not only would this solve Python issues, but it would also facilitate simpler config switching (outlined above). Hmmm...

Direct interaction with the build system considered harmful

Ever since there was a build system developers have been typing make (or make.py) to build the tree. One of the goals of the transition to moz.build files is to facilitate building the tree with Tup. make will do nothing when you're not using Makefiles! Another goal of the moz.build transition is to start derecursifying the make build system such that we build things in parallel. It's likely we'll produce monolithic make files and then process all targets for a related class IDLs, C++ compilation, etc in one invocation of make. So, uh, what happens during a partial tree build? If a .cpp file from /dom/src/storage is being handled by a monolithic make file invoked by the Makefile at the top of the tree, how does a partial tree build pick that up? Does it build just that target or every target in the monolithic/non-recursive make file?

Unless the build peers go out of our way to install redundant targets in leaf Makefiles, directly invoking make from a subdirectory of the tree won't do what it's done for years.

As I said above, I'm sympathetic to forced changes in procedure, so it's likely we'll provide backwards-compatibile behavior. But, I'd prefer to not do it. I'd first prefer partial-tree builds are not necessary and a full tree build finishes quickly. But, we're not going to get there for a bit. As an alternative, I'll take people building through mach build. That way, we have an easily extensible interface on which to build partial tree logic. We saw this recently when dumbmake/smartmake landed. And, going through mach also reinforces my ideal that the object directory is a black box.

Semi-persistent state

Currently, most state as it pertains to a checkout or build is in the object directory. This is fine for artifacts from the build system. However, there is a whole class of state that arguably shouldn't be in the object directory. Specifically, it shouldn't be clobbered when you rebuild. This includes logs from previous builds, the warnings database, previously failing tests, etc. The list is only going to grow over time.

I'd like to establish a location for semi-persistant state related to the tree and builds. Perhaps we change the clobber logic to ignore a specific directory. Perhaps we start storing things in the user's home directory. Perhaps we could establish a second object directory named the state directory? How would this interact with build environments?

This will probably sit on the backburner until there is a compelling use case for it.

The battle against C++

Compiling C++ consumes the bulk of our build time. Anything we can do to speed up C++ compilation will work wonders for our build times.

I'm optimistic things like precompiled headers and compiling multiple .cpp files with a single process invocation will drastically decrease build times. However, no matter how much work we put in to make C++ compilation faster, we still have a giant issue: dependency hell.

As shown in my build system presentation a few months back, we have dozens of header files included by hundreds if not thousands of C++ files. If you change one file: you invalidate build dependencies and trigger a rebuild. This is why whenever files like mozilla-config.h change you are essentially confronted with a full rebuild. ccache may help if you are lucky. But, I fear that as long as headers proliferate the way they do, there is little the build system by itself can do.

My attitude towards this is to wait and see what we can get out of precompiled headers and the like. Maybe that makes it good enough. If not, I'll likely be making a lot of noise at Platform meetings requesting that C++ gurus brainstorm on a solution for reducing header proliferation.

Conclusion

Belive it or not, these are only some of the topics floating around in my head! But I've probably managed to bore everyone enough so I'll call it a day.

I'm always interested in opinions and ideas, especially if they are different from mine. I encourage you to leave a comment if you have something to say.

Rob Campbell: Firefox 23 Developer Tools Fixes

rob — Mon, 13 May 2013 16:40:51 +0000

Bug ID	Summary
581352	Console HTTP output should make JSON viewable as objects
586125	“””Copy”” “”Select All”” etc. popup menu item in the Web Console displays keyboard shortcuts
587757	Implement Browser Console
638953	Cannot copy keys or values in web console object inspector
640225	Make Property Panels Editable
688400	Web Console throws a TypeError when trying to display objects with no toString method
688401	Web Console displays “”[null]“” for [Object.create(null)]
711846	Intermittent browser_webconsole_bug_595934_message_categories.js \| Test timed out (mostly on WinXP PGO)
725771	xpath use in console returns null elements
744741	console.dir tree is empty with cached messages
748851	Web Developer Console incorrectly displays escaped hex/unicode characters
757897	All code in the web console is considered to be at line 1
774365	Web Console setTimeout can’t find declared functions by name
783499	Web Console should use the debugger API
808370	Use the VariablesView in webconsole
809424	Intermittent browser_webconsole_bug_611795.js \| The unknown CSS property warning is displayed only once – Got 1, expected 2
831400	Intermittent devtools/webconsole/test/browser_webconsole_bug_764572_output_open_url.js\| Timed out while waiting for: complete to initialize the opening tab
836720	“”InternalError: too much recursion”” not correctly caught
840093	Text in object inspector can’t be copied
841129	Intermittent browser_repeated_messages_accuracy.js \| two repeats for the second css warning – Got 1, expected 2
843287	UI improvements for the variables view and sidebar
847349	Variables View should have left arrow do collapse and navigate in separate steps
853568	Intermittentbrowser_webconsole_bug_599725_response_headers.js \| we do not have the Content-Type header \| Content-Length != 60 – Didn’t expect 60, but got it
855058	Intermittent browser/dom/tests/browser/browser_ConsoleAPITests.js \| timer duration is positive
857441	Autocompletion popup is very jumpy while typing text
859170	Trying to select [...] for a very long message printed to the web console will hang Firefox
859818	ConsoleAPIStorage.jsm tries to delete var-declared _consoleStorage
859858	Intermittent browser_bug664688_sandbox_update_after_navigation.js \| Timed out while waiting for: window.location.href result is displayed after goBack(), browser_repeated_messages_accuracy.js \| Timed out while waiting for: messages displayed
862024	Warning about replaced window.console API shows when content scripts do not change the object
863943	Intermittentbrowser_webconsole_bug_602572_log_bodies_checkbox.js \| Test timed out, browser_webconsole_bug_603750_websocket.js \| uncaught exception – ReferenceError: ok is not defined
865288	console.log with different object logged from same line number are grouped together
865792	Firefox for Android remote console doesn’t evaluate JS expressions
865871	Object inspector can’t be closed
677930	Style Inspector: make URLs clickable
777681	[rule view] Abbreviate long CSS file names source links
777877	Switch CssRuleView.jsm back to WeakMaps
854948	GCLI context.environment.contentDocument points to doc from when GCLI first opened
855502	we need a proper design for the new checkboxes
855520	“”No element selected”” in the inspector sidebar could use some margins
861908	Destroying Inspector without destroying Toolbox fails.
866381	Inspector shouldn’t break if target is a chrome document
679364	Use either content/devtools/* or content/browser/devtools/* for aliases
851231	Output console.jsm API calls to the browser console
855544	Implement a network monitor
855763	[ResponsiveMode] Intermittent browser_responsiveuiaddcustompreset.js \| Test timed out, \| Found a tab after previous test timed out: data:text/html,foo,browser_Services.js \| Services.prompt is an nsIPromptService
855914	Start using the jetpack loader in devtools
859089	[netmonitor] Under very constrained sizes text in the timeline view is sometimes not rendered
859221	[netmonitor] Sizes are displayed in KB not kb
861197	chrome.manifest files created by the devtools loader should be ignored
723923	Debugger ‘breakpoint list’ GCLI command should have extra nice output
792815	Update promise usage in gcli commands
831720	GCLI needs an appcache command
861192	gcli addon command shows [object Object] instead of a list of addons
865680	devtools.gcli.allowSet is in firefox.js twice
869233	TEST-UNEXPECTED-FAIL \| browser_cmd_appcache_valid.js \| html output for appcache validate should match successfully. Actual textContent: “”Manifest URI…””
739031	Check for unused strings not used in translation (e.g. scoped.label) and remove them
740541	Style sheets in iframes are not listed in the style editor
816967	Remotable Style Editor
866264	Intermittent devtools/styleeditor/test/browser_styleeditor_new.js \| new editor shows 1 rule after modification – Got 2, expected 1
772119	expose source mapped sources over the remote debugging protocol
818134	Allow multiple debuggers in toolboxes to debug separate tabs
832231	After a reload, breakpoints require multiple resumes to allow execution to continue
837723	[jsdbg2] Debugger should provide a way to get Debugger.Object referents directly
840292	The debugger’s xpcshell tests should use listTabs/attachTab instead of listContexts
843187	Variables view: going down through the properties via keyboard is really broken
849069	relative source map URLs should be resolved according to the spec’s rules
849071	Create some kind of chrome to turn source mapping on/off in the debugger
850738	Add a crap load of telemetry tests
851836	breakpoints[aLocation.line] is undefined after bug 820012
853787	Report errors in nested event loops in xpcshell tests
856232	initialize globalActorFactories upon declaring DebuggerServer
859181	Variables view closes immediately on first open
860349	Intermittent browser_dbg_createChrome.js \| application timed out after 330 seconds with no output
861783	Removing watch expression of large object not possible via mouse
861817	Stop using Ctrl+Shift+O shortcut for function searching.
861852	Telemetry data from the remote protocol always appear in the remote buckets
862142	remote debugger doesn’t find any sources for simulator on nightly
862186	remove unused FileUtils import in DebuggerUI.jsm
862360	Use the exported Promise.all instead of reimplementing it
863571	Enable source map xpcshell tests on B2G
864802	The _store on each variables view instance is redundant
864900	The listTabs protocol request can fail if made immediately after a tab is closed
865073	remote debugging protocol: handlers should systematically catch and log exceptions
865328	JS debugger: clean up DebuggerClient.prototype.close
865509	JS debugger: UnsolicitedPauses in dbg-client.jsm should mention ‘exception’
866278	remote debugging protocol: number packets when dumping
867481	Remote debugging connection screen is broken
839515	Tiny fonts in the remote connection screen
849500	navigate and will-navigate events for remoted targets carry payload that is incompatible with the non-remoted case
851546	Create an Options panel for the toolbox
857082	TabTarget.makeRemote doesn’t need any arguments
859569	Target.makeRemote should attachTab
860101	payload._navPayload is a Window in ‘will-navigate’ event handler
862294	Sidebar should be responsive in docked Toolbox.
862363	Sync the killswitch upon (un)registering of a tool and respect it in Options Panel
862398	Maintain the order of tools in the tab and the options panel list
863600	Intermittent browser_toolbox_options.js on Windows PGO builds \| Test timed out
851381	Make the Scratchpad evaluate asynchronously
865803	Unable to save Scratchpads, Style Editor, about:memory, … on OS X
852777	Make Tilt more extensible
862435	Tilt doesn’t open if the Inspector was opened beforehand
855244	Profiler hangs after pressing “”Stop”” if it was started in multiple profiles in different tabs
856797	Typo in “”profilerNotStarted”” in gclicommands.properties
858759	profiler.css in browser/devtools/ should actually be in browser/themes instead
859062	Hide right arrow in the SideMenuWidget
859085	Add tooltips to all labels in the UI that may have cropped values
859308	Hide Blocked in Timings tab until we know what that’s doing
859968	JSONP requests are not parsed into a tree
859980	JSON request not parsed
860055	SideMenuWidget shouldn’t scroll back to the selected item when new requests are coming in
860175	On netmonitor examining the specific details of a network action leads to bad overlapping display
860232	Function callbacks in jsonp responses should be displayed in the Response tab
861487	Sizes are displayed with non-localized decimal separator
862198	Intermittent browser_net_json-long.js \| Test timed out, found a tab
864176	Details pane should remember its height when docked on the right
864519	Several Netmonitor tests fail when ECMAScript Internationalization API is enabled
866091	Intermittentdevtools/netmonitor/test/browser_net_prefs-reload.js \| Test timed out \| Found a tab after previous test timed out: http://example.com/browser/browser/devtools/netmonitor/test/html_simple-test-page.html and tons of fallout

Source: Firefox 22 – 23.

Jared Wein: We’re Jamun!

msujaws — Mon, 13 May 2013 16:00:02 +0000

Well, we were “jamun”. This past week we shut off the Jamun project branch of Australis and moved our focus to the UX branch.

What does all this mean?

In a nutshell, this means that the Australis customization rewrite has progressed enough to the point that we feel it is ready to start getting much broader testing. We’re in the final stretch of the project, and we want our changes to reach testers at a faster pace.

Here’s where we stand today:
* Most customization workflows are possible. Not all UI migrations are complete.
* Most polish on the edges isn’t there. It does however look pretty close on Windows and Mac now.
* Some final graphics are missing, but they’re not far away.

Wait, wait, what are we talking about here?

Ah, okay, I’ll take a step back. A while back, in fact, a looooong time ago… there was a presentation from the Firefox UX team about a new browser UI refresh and rewrite of our customization workflows. It turns out that many users don’t know that Firefox is customizable. Some users accidentally customize their browser and don’t know what went wrong. Then, there’s this super-tiny super-expert user group that has figured out how to customize Firefox and they *adore* it.

So, what are we doing about it?

Well, the first thing that we’re doing is making entering and exiting customization of Firefox much easier. No longer will a user have to right-click on a special portion of a toolbar and choose “Customize…”. This was way too hard to find for the vast majority of users. We’ve left that same entry point there, but we’ve also created a very visible “Customize” button.

Sounds good, where is this Customize button though?

Great question! Another goal of Australis is to unify the user experience between Windows, OS X, and Linux. On Windows and Linux, Firefox has an “Application Menu” in the top-left corner of the browser. We’ve moved this menu to the right-side of the navigation toolbar and it will now be visible on all three platforms. We’ve also been hard at work trying to make this menu easy to use and navigate. The Customize button is located at the bottom of this menu.

Another really cool thing about this menu is that it will be customizable. When you enter Customization mode, you’ll be able to add, remove, and rearrange items in the menu as well as items on the toolbars.

Here’s a screenshot of what the Customization mode looks like today on Windows:

As I mentioned earlier, all of this is still very much a “work-in-progress”, so it’s expected that people will find bugs and rough edges. If you’d like to play with it today, you can download the UX Nightly builds and give it a run. The UX Nightly builds will update daily with new changes to the customization.

Please let us know about any bugs that you find by filing a bug in Bugzilla in the Firefox::Theme or Firefox::Toolbars component (and mark the bug as blocking bug 770135). If you don’t feel comfortable doing the above, then just leave a comment on this blog post and I or someone else will file the bug for you.

Tagged: australis, firefox, mozilla, planet-mozilla, usability

Niko Matsakis: Recurring closures and dynamically sized types

Mon, 13 May 2013 14:35:00 +0000

I realized today that there is an unfortunate interaction between the proposal for dynamically sized types and closure types. In particular, in the case of the recurring closure, I described the soundness issues that arise in our language when closures are able to recurse.

My solution for this was to make the type system treat a &fn() value the same way it treats &mut T pointers: they would be non-copyable, and when you invoke them, that would be effectively like a “mutable borrow”, meaning that for the duration of the call the original value would become inaccessible. So in short the type system would guarantee that when you call a closure, that same closure is not accessible from any other path in the system, just as we now guarantee that when you mutate a value, that same value is not accessible from any other path in the system.

This is all well and good, and I think this treatment would be largely invisible to the user under common access patterns. However, it does not play well with the proposal for dynamically sized types, because under this proposal all things written &T must behave the same, no matter what T is. This is in fact the whole point of the proposal! But here I want to treat &fn specially.

I’ve been pondering various solutions this morning. I have come up with two possible avenues:

Instead of writing &fn() you could write &mut fn(). This is perhaps the “principled” solution, but I consider it rather a non-starter. Writing &fn() for a closure is…tolerable, but &mut fn() is not. It’s verbose and it seems sort of nonsensical (although there is some logic to it, when you consider that calls to the function may mutate the environment and so forth).
We go back to the older notation and move sigils for closures after the fn. This actually has some notational perks. For example, rather than writing &fn() we can just write fn() (if there is no sigil, we can default to &). On the minus side, a sendable closure would be written fn~()—but, then again, under the dynamically sized types proposal, sendable closures were going to be written ~fn:Owned(), so is fn~() really so bad?

More details after the fold.

OK, let’s dig into the details a bit more. As anyone who has been following my blog posts probably knows by now, there are many, many use cases for closures. I want to dive into the use cases that are on my mind and elaborate on them. I also want to take this case to write up a bit more thoroughly how I think closures should work, including a few unrelated issues.

Syntax and use cases

Here is a list of use cases to be accommodated:

“Higher-order functions”: simple functions like map, fold and so forth. By far the most common use case.
“Once functions”: functions that can only execute once. This means that they can move values out of their environment.
“Sendable functions”: functions that can be sent between tasks. This means that they only close over “sendable” values (no garbage-collected data or borrowed pointers).
“Sendable once functions”: sendable functions that can only execute once. This is what a task body will be.
“Const functions”: functions that do not close over mutable state. We don’t make much use of this yet, but I plan to do so in order to achieve lightweight fork-join parallelism a la PJS.

The use cases above seem to me to be the “bread and butter” cases that will arise frequently. I will go over the syntax and give an example for each of those use cases shortly. Interestingly, I think that all of them actually read reasonably well if the sigils are moved after the fn keyword, and in some cases the examples read much better.

However, there are two additional use cases that I have considered in the past which I left out. These use cases become significantly harder to read under the new proposal (though they were always hard to read). Interestingly, I realized while writing this blog post that I think these use cases are no longer terribly important, since both of them can be expressed equally well using objects instead of closures, as I will explain shortly. The two use cases are:

“Sendable const functions”: functions that can be sent between tasks and do not close over mutable state. You could safely share such functions between tasks in an ARC (atomically referenced counted container) and execute them multiple times in parallel.
“Combinators”: combinator libraries create and return closures that closure over their arguments, which may include borrowed values.

Higher-order functions

Here is an example of a simple higher-order function (with the closure type highlighted):

impl<T:Sized> for [T] {
    pub fn map<U:Sized>(f: fn(&T) -> U) -> ~[U] { ... }
                        // ^~~~~~~~~~~
}

For contrast, this is &fn(&T) -> U today.

Once functions

Here is an example of a higher-order function that executes at most once:

impl<T:Sized> for Option<T> {
    pub fn each(f: once fn(&T) -> bool) -> bool { ... }
                // ^~~~~~~~~~~~~~~~~~~
    }
}

For contrast, this is &once fn(&T) -> U today.

Sendable functions and sendable once functions

Here is an example of a sendable once function:

fn spawn(f: once fn~()) {...}
         // ^~~~~~~~~~

The ~ after the fn tells the type system that the environment for this function is allocated using an owned pointer. It also implies a default bound of Owned. The once tells the type system that the function will only execute once.

For contrast, this is ~once fn() today.

Const functions

Here is an example of how I would use a const function to achieve lightweight parallelism:

impl<T:Sized> for [T] {
    pub fn par_map<U:Sized>(f: fn:Const(&T) -> U) -> bool { ... }
                            // ^~~~~~~~~~~~~~~~~
}

This is a parallel map function. It is similar to the regular map except that its iterations execute in parallel. As a consequence, it demands a fn:Const rather than a fn—the Const bound specifies that all the environmental state must be immutable. This is exactly the “patient parent” or “parallel closures” model that is used in PJS and described in this HotPar paper I wrote.

For contrast, this is &fn:Const() today.

Sendable const functions

Sendable const functions are one of the two cases that I said would become less attractive under the new proposal. They would look something like fn~:Const (vs ~fn:Const today). The newer syntax works and should be available, but it’s hard to read, due I think to the juxtaposition of ~ (which specifies the kind of pointer used for the environment) and the : that begins the bound specifier :Const. If this use case were important, I might be worried that the syntax is too ugly, but when I tried to come up with an example for where this use case would be needed, I realize that time has left the use case behind to some extent.

The primary use case for a sendable const function initially was to allow hashtables to be placed in ARCs—the reason for this was that a HashMap requires closures for for computing the hash function of its argument, and those to share the hashmap (and perform parallel lookups) we had to be sure that the closures would not mutate any state. However, this is somewhat outdated, because hashing and equality comparison today is based on traits rather than closures.

Now, using traits is somewhat limited, because due to coherence it means that any one type can only be hashed in one way, and sometimes you would like to have specialized hashing for specific circumstances. But these use cases can easily be accommodated in three ways:

Using newtyped keys (struct MyKey(key)) and defining different implementations for the hashing and equality traits on MyKey.
If a newtyped key is not acceptable, you can write a hash table that takes a simple function pointer (extern "Rust" fn) rather than a closure. Function pointers carry no state, but state is rarely needed for equality comparisons.
If you really need state, then you can write a specialized trait in lieu of a closure:
```
trait HashFuncs<K> {
    fn hash(&self, k: &K) -> uint;
    fn eq(&self, k1: &K, k2: &K) -> bool;
}
```
Now your hashtable can either take a ~HashFuncs object to use for hashing and equality comparison or, if you wish to avoid dynamic dispatch for performance reasons, you can parameterize your hashtable type by the instance of HashFuncs that it should use:
```
struct MyHashMap<K,V,F:HashFuncs<K>> {
    f: F,
    ...
}
```

Combinators

General purpose combinators are the other case that (might) get less attractive. This is less clear cut. The idea of a combinator library is that you have functions that return functions, and then you can compose these functions into bigger functions. The most common example is a parser combinator, which is a simple way to create inefficient and buggy parsers (ok, that’s unfair, but I couldn’t resist; I’ve had some bad experiences trying to scale up parser combinators—truth is, they are super nice to work with, at least until things go wrong).

Anyway, a typical parser combinator library would begin with a primitive like the following:

fn expect(c: char) -> fn@(&mut ParseState) -> Result<(), Err> { ... }
                   // ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Note that the function returns a closure. We used fn@ because this closure must be allocated on some heap in order for us to return it, and because using the type fn@ (vs say fn~) would allow us to close over managed and other task-local data. So far, I think this example works out fine.

Where things get more complex is if we want to close over borrowed pointers. For example, imagine an expect function that takes a slice:

fn expect_string<'a>(s: &'a str)
                     -> fn@:'a(&mut ParseState) -> Result<(), Err> {...}
                     // ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Here the type system will require that the lifetime 'a of the input slice s appear in the resulting function type, so that it can be sure that the function is not used after the slice is no longer valid. This makes the type more complicated: fn@:'a (vs the also-not-especially-intuitive notation of @'a fn today).

Of course, one could address this problem by having expect_string take a ~str or @str instead of a borrowed string, but in some use cases borrowed pointers may perfect sense. For example, I had once thought to use this pattern to create a combinator library for expressing iteration primitives like enumerate and so forth (similar, experimental work is now underway in the iter module).

Interestingly, just as with sendable const closures, objects and traits can provide an alternative that is ultimately (I think) a better and more readable design anyway. We could rewrite the return type from a closure into a trait:

trait Parser<R> {
    fn parse(&mut ParseState) -> Result<R,Error>;
}

fn expect(c: char) -> @Parser<()>;
fn expect_string<'a>(s: &'a str) -> @Parser:'a<()>;

Here in the expect_string case I have taken advantage of the fact that object types will also carry bounds similar to closure types. An advantage of this design is that using a trait allows the Parser objects to carry more methods as well.

If we were to extend the example to include an actual combinator, I imagine it would look something like this:

fn or<'a, R>(p1: @Parser:'a<R>, p2: @Parser:'a<R>) -> @Parser:'a<R> {...}

Of course, for maximum efficiency, one would avoid using object types altogether. Then you would just implement Parser directly on the char and &str types, and perhaps write the or combinator like so:

struct or<P1,P2>(P1, P2);

impl<R,P1:Parser<R>,P2:Parser<R>> Parser for or<P1,P2> {
  fn parse(&self, state: &mut ParseState) -> Result<R,Error> {
    let (ref p1, ref p2) = *self;
    state.try(); // (*)
    match p1.parse(state) {
      Ok(r) => { state.confirm(); Ok(r) }
      Err(_) => { state.backtrack(); p2.parse() }
    }
  }
}

// (*) Here you see my imperative roots. A true functional
// programmer would not use in-place mutation here but rather
// clone and return a new parser state.

Summary

Another long post mostly targeted at rust devs and myself. Sorry about that. I think the bottom line is that we should move sigils for closures and have them appear after the fn keyword. This makes me sad, because this is how things used to be, and in fact one of the main goals of the dynamically sized types (DST) proposal was to move the sigils in closure types in front. But of course soundness comes first, and I think the general wins of the DST proposal (consistent behavior for all &T, @T, ~T etc) outweigh the need to write fn~ on occasion (I don’t really see much use for fn@).

There is also one final solution I didn’t mention in my initial paragraphs. We could adopt the “principled” solution of using &mut for closures but change the way we notate &mut. I have largely avoided thinking about because I want to avoid destabilizing syntax changes. However, I have toyed around occasion with an idea for reorganizing our types to emphasize ownership and de-emphasize mutability, which goes in this direction. I may indulge myself and write it up at some point. Still, I largely consider this a non-starter.

Adopting the “move sigils in back” proposal does have another casualty, though. There has been some talk of figuring out ways to make @ and ~ less special (as in, allowing user-defined pointer types like RefCounted<T> that are on equal footing). The DST proposal is clearly a step in that direction. Moving the sigils backwards on fn types is, well, a step backward, because closures would always be allocated using a limited set of allocators (stack, ~, or @).

In an odd way, finding this interaction makes me feel good. I’ve been concerned that the DST proposal seemed too easy, which meant we weren’t thinking hard enough about it. But there is another reason as well: I have also been concerned that closure types were becoming a bit too… special, particularly with regard to copyability. Basically I’ve been concerned that although the syntax for a borrowed closure was &fn, borrowed closures didn’t really behave like & pointers—without the DST proposal, this was something that we could safely enforce as part of the type system, but it’s still confusing for users. So I think the DST proposal forces us to be more honest, and that’s a good thing all around.

Michael Kaply: More MAJOR changes coming in Firefox 21

Mike Kaply — Mon, 13 May 2013 13:56:21 +0000

In my previous post, I talked about the moving of some files for Firefox 21. Due to some major bugs related to partner builds, the distribution directory was moved back. So you no longer need to move distribution files into the browser directory. So here's a summary of what has changed:

defaults/preferences -> browser/defaults/preferences
defaults/profile -> browser/defaults/profile
extensions -> browser/extensions
searchplugins -> browser/searchplugins

I've also been asked about the file override.ini - this file should now be placed in the browser subdirectory as well.

Remember, Firefox 21 is scheduled to be released tomorrow.

Gregory Szorc: The State of Mercurial at Mozilla

Mon, 13 May 2013 13:25:00 +0000

I have an opinion on the usage of Mercurial at Mozilla: it stinks.

Here's why.

The server is configured poorly

Our Mozilla server, hg.mozilla.org, is currently running Mercurial 2.0.2. In terms of Mercurial features, stability, and performance, we are light years behind.

You know that annoying phases configuration you need to set when pushing to Try? That's because the server isn't new enough to tell the client the same thing the configuration option does. It will be fixed when the server is upgraded to 2.1+.

Furthermore, we are running the server over NFS, which introduces known badness, including slowness.

I believe we blame Mercurial for issues that would go away if we configured the Mercurial server properly.

Fortunately, it appears the upgrade to 2.5 is near and I've heard we're moving from NFS to local disk storage as part of that. This should go a long way to making the server better. The upgrade can't happen soon enough.

User education is poor

I think a lot of people are ignorant on the features and abilities of Mercurial.

I commonly hear people are dissatisfied with the behavior of their Mercurial client. They encounter performance issues, bugs, corruption, etc. Nine times out of ten this is due to running an old Mercurial release. Just last Friday someone on my team asked me about weird behavior involving file case. My first question: what version of Mercurial are you using? He was running 2.0.2. I told him to upgrade to 2.5+. It fixed his problem. If you aren't running Mercurial 2.5 or newer, upgrade immediately.

I've heard people say we should switch to Git because Git has feature X. Most of the time, Mercurial has these features. Unfortunately, people just don't realize it. When I point them at Mercurial's extensions list their eyes light up and they thank me for making their lives easier.

I think a problem is a lot of new Mozilla contributors knew Git before and only pick up the bare essentials of Mercurial that allow them to land patches. They prefer Git because it is familiar and just don't bother to pick up Mercurial. The potential of Mercurial is thus lost on them.

Perhaps we should have a brown bag and/or better documentation on getting the most out of Mercurial?

The branching model is far from ideal

For Gecko/Firefox development, we maintain separate repositories for the trunk and release branches. This introduces all kinds of annoying.

We should not have separate repositories for central, inbound, aurora, beta, release, etc. We should be using some combination of branches and bookmarks and have all the release heads in one repository, just like how the GitHub mirror is configured.

As an experiment, I created a unified Mercurial repository. Each current repository is tracked as a bookmark (there are instructions for reproducing this). Unfortunately, the web interface isn't showing bookmarks (perhaps because the version of Mercurial is too old?), so you'll have to clone the repository to play around. Just run hg bookmarks and e.g. hg up aurora after cloning. Warning: I'm not actively synchronizing this repository, so don't rely on it being up to date.

A Mercurial contributor (who is familiar with Mozilla's development model) suggested we use Mercurial branches for every Gecko release (20, 21, 22, etc). I think this and other uses of branches and bookmarks are ideas worth exploring.

We're failing to harness the extensibility

Gecko/Firefox has a complicated code lifecycle and landing process. This could be significantly streamlined if we fully harnessed and embraced the extensibility of Mercurial. While there are some Mozilla-centric extensions (details in my recent post), I don't think they are well known nor used.

I think Mozilla should embrace the functionality of extensions like these (whether they be for Mercurial, Git, or something else) and invest resources in improving the workflows for all developers. Until these tools are obviously superior and advertised, I believe many developers will unknowingly continue to toil without them. And, it's likely hurting our ability to attract and retain new contributors as well.

Conclusion

Mozilla's current usage of Mercurial is far from ideal. It's no wonder people don't like Mercurial (and why some want to switch to Git).

Fortunately, little has to do with shortcomings of Mercurial itself (at least with newer versions). If you want to know why Mercurial isn't working too well for Gecko/Firefox development, most of the problems are self-inflicted or the solutions reside within each of us. Time will tell if we as a community have the will to address these issues.

Axel Hecht: git-merge 2013

Axel Hecht — Mon, 13 May 2013 12:36:49 +0000

I spent Friday and Saturday on git-merge, an unconference on git. Thursday was developer day, for core contributors to git itself, and libgit2/jgit. I didn’t go there. Friday was “user” day, and Saturday was hackday. I figured it might be useful to go to the userday, and turned out, it was. It wasn’t all that much about users at all. It was strictly unconference, so people would take the stage and give a quick lightening talk, and later in the day we had a few breakout sessions. The most user-like people were folks migrating to git just now, and they had a good deal of folks to talk to in the breakout session.

The rest of Friday was actually library and tool hackers talking about what they’re doing. There are notes on github.com/git-merge/user-day, but I do want to hightlight a few.

imerge is probably the most interesting to gecko hackers. It allows you to merge or rebase intensive histories with conflicts incrementally, with tons of automation. It even allows to do test runs on the merges it does automatically. If you ever resolved the same conflict 10 times in one rebase, this is for you.

libgit2 was under the hood of many, with core contributors of the library itself being there, plus the maintainers of at least the C# and the python bindings. There was also a good deal of tooling based on jgit, a pure-java implementation of git.

Much debate on java vs not, actually. And Cmd-F1. Little conflict between git and hg.

I also got to enjoy the github backend talk by vmg, with Ernie, Bert, and smoke through chimneys. I had seen a recording of it before, but it was well worth it seeing it live.

I joined the breakout session about teaching git, and had the pleasure to be in a group that tought git in various parts of the globe. Yes, that might be relevant to me, so that was good exercise. I talked with Scott Schacon about localized version of the git book, and localized github docs, too.

Given that I had the chance to talk and co-hack with all those tooling guys, I decided to drop by on Saturday for the hack day. I wanted to take the opportunity to talk about my weird repository rewrite questions, and so I did.

Saturday was great. I only got 20 lines of code written, but I finally understood what git is about in the back-end. There’s loads of hackery that you can do, and funny enough, both I and Ben ended up hacking on a repository rewrite algorithm, which helped me a lot, both talking about the structure of the code, as well as vetting the approach. His code in C# is actually in a PR. Worth a look for people that want to see how to hack tooling on top of that binding. I tried the same in python, and succeeded to some extent. David Ibáñez helped me a great deal. But it also showed me the difference between the C# API and the python bindings. If only mono was easy to get up on the mac.

On the conference itself, it was set up at the Radisson Blu next to the Berliner Dom, which is a nice venue for that size. Wifi worked, food and beer were there. It’s sad that many people claimed tickets and didn’t show up. Now you know what you missed, and what you made other people miss. Bad bunny, no chocolates! Thanks again for Jen for setting things up great.

Thanks to all the folks at git-merge for making it a great event. See you in Berlin…

Brandon Savage: Compiling PHP 5.5 From Scratch

Brandon Savage — Mon, 13 May 2013 11:00:06 +0000

There’s always a lag behind new releases of PHP and releases of packages for operating systems such as Ubuntu. This lag time means that you could be kept from upgrading to the latest and greatest PHP for a year or more, unless you use an outside repository like Dotdeb. Of course, even when using one [...]

Doug Belshaw: Reading, feeding and seeding

Doug Belshaw — Mon, 13 May 2013 10:16:06 +0000

After several days of migraines and with an upcoming intense week in the form of a Mozilla All-Hands meeting, this week I’m going to be focusing on inputs rather than outputs.

This is likely to be my only blog post of the week (apart from my weeknote) and I’ll be a lot lighter in my use of Twitter / Google+ / Facebook.

If you want to make me smile while I’m ensuring I don’t get burned out, how about donating to #LettingGrow? That would be awesome.

Image CC BY Pink Sherbert Photography

Gervase Markham: 3D Printing A Replacement Cot Clip

gerv — Mon, 13 May 2013 07:00:08 +0000

Some things made with 3D printers could possibly be made more easily using other manufacturing methods – although printing them yourself is often more fun. But sometimes you come across something which would be fairly tricky to make otherwise, unless perhaps you had high-end equipment like a laser cutter.

Cots, in the UK at least, often have a “drop side” – a side which is normally at the standard height, but can be lowered to make it easier to insert or remove a child. The side is usually held up by some kind of safety clip at each end, such that when you raise the side, the clips automatically engage, and when you want to lower it, you need to lift the side a little and pull both clips out of the way from the outside. This prevents a child pulling off an escape :-)

One of the two cots we have used to belong to my wife’s brother. It has such clips, and long ago one of them broke. He made a flat replacement out of cut perspex, but it has also recently broken. This is a problem because our changing table fits on top of the cot, and if the sides aren’t the same height, the table can’t be used and we are having to change William and John with the mat on the floor! So I thought I’d try and print one.

As you can see from the right of the photo below, the existing clip is a rather specific shape, and would have been hard to produce by many other methods without a lot of trial and error. Many parts have to be exactly the right angle and length or it won’t open, or close, or pass the cot side, or whatever. I took a scan of the existing clip (which was a bit blurry, as the clip is not flat) and traced around it in LibreCAD. I then saved that trace as a DXF file, and wrote an OpenSCAD file to make a new clip, based on a linear extrusion of the outline, plus (on later attempts) two extra blocks for a stop and a handle. Here are my efforts:

The first was too small, the second marginally too big (and the blocks not aligned right), and the third, like Little Bear’s porridge, was just right :-) Well, it suffices anyway, although the hook could do with being 1mm smaller for easy passage of the side.

Here’s the clip in situ, supporting the side of the cot:

And this photo, beautifully modelled by my wife, shows how it fits when the side is down:

Eitan Isaacson: GNOME Workstation OS

Eitan — Mon, 13 May 2013 00:40:35 +0000

I wanted to give my one cent about the GNOME project, and where I think it could be successful. It would be two cents if I were actually involved in any constructive manner, but I am not. So it is one cent.

Ever since I started contributing to GNOME, the looming questions have been mobile, web, and social. Every keynote at GUADEC has tugged us in that direction, or promised to “reboot” the effort. If it is Big Board and Mugshot, Pyro Desktop, Telepathy and the collaboration it was supposed to bring, the countless OpenedHand and Nokia innovations, etc. We have all been running around like a chicken with its head cut off ever since I remember, trying to capture the essence of these new trends and remain relevant.

We failed.

Apple revolutionized not just mobile, but reinvented the mainstream computing form factor. Facebook made “social” ubiquitous, and Google is doing what it is doing to the web. In the meantime, I never gained any following on Last.fm from all those years of scrobbling music with Rhythmbox and Banshee, I never got an opportunity to use Telepathy tubes with any real live person, and apparently my Mugshot profile is gone. My eyes also got tired of squinting at XTerm on my N900.

Last year in Berlin, the lack of direction was apparent. Almost every keynote that I remember was given by one kind of designer or seo company. Somewhere along the line we confused design with leadership. At least there wasn’t as much self delusion about our bright future on mobile.

But there is a way out of this rut. And it requires acknowledging our weaknesses and exploiting our strengths.

Our weak areas are apparent: We are not mobile and we are very far from it. We will never achieve any significant social critical mass, we have had limited successes in embracing web technologies, but the web will always be a better web. Deploying “apps” is a nightmare.

Our strengths are pretty obvious too: In the last few years we successfully refreshed the desktop work flow and our entire framework. We support many productivity and authoring tools. We created a distraction free environment that lets users get work done. We run on commodity hardware. We are free. We have a windowed multitasking environment. We work really well with a screen, mouse and keyboard (not to be taken for granted, look at all the awkward Android tablet keyboard combos out there). More than one web browser supports us. There is a more than fully functional office suite that works well with us. Etc.

So instead of aspiring to be in every consumer gadget out there, I think we should aspire to be the work horse of choice for every content creator out there. This includes mobile/web developers, graphic designers, artists, bloggers, video bloggers, authors, journalists, podcast producers, and every other kind of content creator that makes the mobile web and social such a vital space for the rest of the world. We need to refocus on the desktop.

Let’s leave the mission of bringing free software to mobile and the web to others. Other groups are doing a great job there. They are in their element; let’s remain in ours. We should focus on the production end of the New Media pipeline.

Projects such as The GIMP, PiTiVi, Anjuta, Blender and Libre Office should be our bread and butter. We should strive to stay ahead of the curve on the authoring end. We should document and support Android and Unity development. The Wacom tablet support that landed recently is a good example of what we could be doing.

It feels like GNOME is being maintained by a skeleton crew, and a shrinking set of corporate stakeholders. I think it is time to be realistic about what we could excel at, and go there. We don’t have to be on every existing form factor to achieve world domination. The cloud, and all these cheap new gadgets have lowered the barrier to access. We could lower the barrier of authorship, and enable people to create new and rich content.

John O'Duinn: “Brag! The art of tooting your own horn without blowing it” by Peggy Klaus

John — Mon, 13 May 2013 00:14:54 +0000

Normally, a small book like this (193 pages) would be a quick read for me, but this book took me literally months. Not, I hasten to add, because of any problems with the book or the writing style, that was all fine. The problem was that this book uncovered a bunch of things I am personally working through. I found myself reading a few pages, highlighting some lines, then walking away thinking. Repeat a few times a week. Occasionally, I’d go back and re-read entire chapters.

For me, bragging has negative connotations and is something I avoid like the plague. Stereotypes of obnoxious, pretentious people, loudly telling all within range just how great they are. The very last thing I ever want to be. Whether that is cultural, learned from family, something I developed myself growing up, or a mixture, I don’t know. But it is part of who I am. This book is all about encouraging people to find a comfortable place in between these extremes. As Peggy is quick to note, this means different things to different people, so you need to pay attention to what is authentic for you, as that authenticity is important. People have generations of experience spotting fakes, and worst of all, deep down, you’ll know you are faking it too.

Because of the book title, it took several people pushing to get me to even start reading this book. Chapter#1 opened with a line that stopped me dead in my tracks.

“Myth#1: A job well done speaks for itself.”

I’ve always thought that if I did a good job, or handled a tricky situation well, people would notice. If I solved some complex problem, that people would understand the complexity, understand the importance of the achievement and appreciate the work. In those circumstances, having others recognize and complement the achievement was fine, but any attempt on my part to “brag” about my work would in some way “cheapen the victory”. After reading this book, I now think that is *sometimes* true but not always true. While the people working beside me in the same trenches, working side-by-side with me on the problem might understand the scale of the accomplishment, most people simply don’t know the details. Over time, people might eventually notice that a recurring problem hasn’t happened in a while, or they might simply forget about a previously-annoying problem because it hasn’t happened in a while… but they’d never stop and wonder why. Another common trend is for people to not notice one problem is fixed, but instead notice that a different problem has “appeared”. Oh, and meanwhile, people don’t know what you are working on. Over time, this becomes frustrating for everyone. After reading this book, I’ve learned that I need to make sure I inform people of the work I’m doing, and why it’s important to them. I don’t need to go into all the complexities of the project, unless they ask for more details, but it’s important to make sure others are aware of my work, and the impact it has on them and their work.

I found this a tough read, yet super worth the time. And, yes, I strongly recommend it.

“It ain’t bragging if you done it” (Dizzy Dean)

Christian Heilmann: “just use technology $x” is a terrible answer to a question

Chris Heilmann — Sun, 12 May 2013 23:55:51 +0000

A few days ago a vertical video went viral of the student Jeff Bliss telling his teacher off for being a bad teacher who just hands out materials without explaining anything.

And we all applauded him for his gall and his eagerness to learn and to point out the obvious flaws in the education system. Teachers are not paid enough, are under more stress to be seen as someone who has students with good grades rather than understanding and have to deliver course materials they don’t believe in but are forced to go through as those are the ones that are easy to mark.

Terrible, isn’t it? So why do people in our little world of web development constantly and voluntarily become this bad, bored and ineffective teacher?

What am I talking about? The thing I mentioned in large detail in my talk at BaconConf this year but here it is for the generation who wants things in 140 chars or as a cute image with large letters on it:

Whenever you answer a question of another developer with “just use $x” you breed an expert idiot. You did not teach them anything, you showed a way out of learning.

In my ongoing task to de-clutter my life I just un-subscribed from several communities on Google+, namely the HTML5 community and the JavaScript one. Not because I am not interested in these matters any more, quite the opposite: because I care much about these communities and all I found there is frustration and annoyance. Nearly every single question new developers have is answered in three ways:

Use jQuery – here is a plugin
Just Google for it
You’ll need to use framework $x / JavaScript and/or HTML5 is not good enough for that

None of these answers are satisfactory if you really want to help someone who needs to solve a problem and learn how to repeat the solution in the future. The latter in most cases is a plain lie and shows that you are blaming a technology for your lack of interest in it.

What gets answered far too quickly is the “how” – how to achieve a massively complex result (which yet has to be proven to be really necessary) without thinking about it or understanding the solution that you apply. We assume that is enough and that we are doing something good – we let a new developer have a positive experience of having something achieved very quickly and that will obviously entice him or her to learn more and go explore in more detail later on.

That is not necessarily the case. We showed people a shortcut and how to focus on the outcome and hope the step where they understand what they are doing comes later. Sadly in a lot of cases this never comes but it fills people with fake confidence that gets shattered once they have their first job interview in a real company who cares about what they build.

If you want to teach people, make them understand the “why” and let them find their own “how”. That is much harder work, but also much more rewarding when you see them grow and explore.

We do not teach people how to write by copying and pasting the style of other authors. We tell them about similes, metaphors, rhetoric devices, orthography and grammar rules. Why bother with that? We could just show them TXTSPK and explain that anyone who knows English will understand what they try to convey. The reason why we do it is that we teach the fun of playing with language and finding your own style.

“But I don’t have time for that – I just want to help someone solving their problem”

Is a very common, admirable, but misguided idea. What you do is advertise the solution that made most sense to you as you already solved the problem. You steal the learning experience away from the other person and the way we learn is our most personal asset and differs vastly from person to person.

If you don’t want to really teach and see people grow and learn on their own terms, please stop spouting truisms and “best quick solutions” in places where people come to learn. If all they want is for you to solve their issue, they should hire you to do so for them.

Don’t be the grumpy teacher you learned to first despise and later on pity. We can do better on the web.

Peter Bengtsson: What stumped me about AngularJS

Sun, 12 May 2013 20:53:35 +0000

So I've now built my first real application using AngularJS. It's a fun side-project which my wife and I use to track what we spend money on. It's not a work project but it's also not another Todo list application. In fact, the application existed before as a typical jQuery app. So, I knew exactly what I needed to build but this time trying to avoid jQuery as much as I possibly could.

The first jQuery based version is here and although I'm hesitant to share this beginner-creation here's the AngularJS version

The following lists were some stumbling block and other things that stumped me. Hopefully by making this list it might help others who are also new to AngularJS and perhaps the Gods of AngularJS can see what confuses beginners like me.

1. AJAX doesn't work like jQuery

Similar to Backbone, I think, the default thing is to send the GET, POST request with the data the body blob. jQuery, by default, sends it as application/x-www-form-urlencoded. I like that because that's how most of my back ends work (e.g. request.GET.get('variable') in Django). I ended up pasting in this (code below) to get back what I'm familiar with:

module.config(function ($httpProvider) {
  $httpProvider.defaults.transformRequest = function(data){
    if (data === undefined) {
      return data;
    }
    return $.param(data);
  };
  $httpProvider.defaults.headers.post['Content-Type'] = ''
    + 'application/x-www-form-urlencoded; charset=UTF-8';
});

2. App/Module configuration confuses me

The whole concept of needing to define the code as an app or a module confused me. I think it all starts to make sense now. Basically, you don't need to think about "modules" until you start to split distinct things into separate files. To get started, you don't need it. At least not for simple applications that just have one chunk of business logic code.

Also, it's confusing why the the name of the app is important and why I even need a name.

3. How to do basic `.show()` and `.hide()` is handled by the data

In jQuery, you control the visibility of elements by working with the element based on data. In AngularJS you control the visibility by tying it to the data and then manipulate the data. It's hard to put your finger on it but I'm so used to looking at the data and then decide on elements' visibility. This is not an uncommon pattern in a jQuery app:

<p class="bench-press-question">
  <label>How much can you bench press?</label>
  <input name="bench_press_max">
</p>

if (data.user_info.gender == 'male') {
  $('.bench-press-question input').val(response.user_info.bench_press_max);
  $('.bench-press-question').show();
}

In AngularJS that would instead look something like this:

<p ng-show="male">
  <label>How much can you bench press?</label>
  <input name="bench_press_max" ng-model="bench_press_max">
</p>

if (data.user_info.gender == 'male') {
  $scope.male = true;
  $scope.bench_press_max = data.user_info.bench_press_max;
}

I know this can probably be expressed in some smarter way but what made me uneasy is that I mix stuff into the data to do visual things.

4. How do I use controllers that "manage" the whole page?

I like the ng-controller="MyController" thing because it makes it obvious where your "working environment" is as opposed to working with the whole document but what do I do if I need to tie data to lots of several places of the document?

To remedy this for myself I created a controller that manages, basically, the whole body. If I don't, I can't manage scope data that is scattered across totally different sections of the page.

I know it's a weak excuse but the code I ended up with has one massive controller for everything on the page. That can't be right.

5. `setTimeout()` doesn't quite work as you'd expect

If you do this in AngularJS it won't update as you'd expect.

<p class="status-message" ng-show="message">{{ message }}</p>

$scope.message = 'Changes saved!';
setTimout(function() {
  $scope.message = null;
}, 5 * 1000);

What you have to do, once you know it, is this:

function MyController($scope, $timeout) {
  ...
  $scope.message = 'Changes saved!'; 
  $timeout(function() {
    $scope.message = null;
  }, 5 * 1000);
}

It's not too bad but I couldn't see this until I had Googled some Stackoverflow questions.

6. Autocompleted password fields don't update the scope

Due to this bug when someone fills in a username and password form using autocomplete the password field isn't updating its data.

Let me explain; you have a username and password form. The user types in her username and her browser automatically now also fills in the password field and she's ready to submit. This simply does not work in AngularJS yet. So, if you have this code...:

<form>
<input name="username" ng-model="username" placeholder="Username">
<input type="password" name="password" ng-model="password" placeholder="Password">
<a class="button button-block" ng-click="submit()">Submit</a>
</form>

$scope.signin_submit = function() {
  $http.post('/signin', {username: $scope.username, password: $scope.password})
    .success(function(data) {
      console.log('Signed in!');
    };
  return false;
};

It simply doesn't work! I'll leave it to the reader to explore what available jQuery-helped hacks you can use.

7. Events for selection in a `<select>` tag is weird

This is one of those cases where readers might laugh at me but I just couldn't see how else to do it.
First, let me show you how I'd do it in jQuery:

$('select[name="choice"]').change(function() {
  if ($(this).val() == 'other') {
    // the <option value="other">Other...</option> option was chosen 
  }
});

Here's how I solved it in AngularJS:

$scope.$watch('choice', function(value) {
  if (value == 'other') {
    // the <option value="other">Other...</option> option was chosen 
  }
});

What's also strange is that there's nothing in the API documentation about $watch.

8. Controllers "dependency" injection is, by default, dependent on the controller's arguments

To have access to modules like $http and $timeout for example, in a controller, you put them in as arguments like this:

function MyController($scope, $http, $timeout) { 
  ...

It means that it's going to work equally if you do:

function MyController($scope, $timeout, $http) {  // order swapped
  ...

That's fine. Sort of. Except that this breaks minification so you have to do it this way:

var MyController = ['$scope', '$http', '$timeout', function($scope, $http, $timeout) {
  ...

Ugly! The first form depends on the interpreter inspecting the names of the arguments. The second form depends on the modules as strings.

The more correct way to do it is using the $inject. Like this:

MyController.$inject = ['$scope', '$http', '$timeout'];
function MyController($scope, $http, $timeout) {
  ...

Still ugly because it depends on them being strings. But why isn't this the one and only way to do it in the documentation? These days, no application is worth its salt if it isn't minify'able.

9. Is it "angular" or "angularjs"?

Googling and referring to it "angularjs" seems to yield better results.

This isn't a technical thing but rather something that's still in my head as I'm learning my way around.

In conclusion

I'm eager to write another blog post about how fun it has been to play with AngularJS. It's a fresh new way of doing things.

AngularJS code reminds me of the olden days when the HTML no longer looks like HTML but instead some document that contains half of the business logic spread all over the place. I think I haven't fully grasped this new way of doing things.

From hopping around example code and documentation I've seen some outrageously complicated HTML which I'm used to doing in Javascript instead. I appreciate that the HTML is after all part of the visual presentation and not the data handling but it still stumps me every time I see that what used to be one piece of functionality is now spread across two places (in the javascript controller and in the HTML directive).

I'm not givin up on AngularJS but I'll need to get a lot more comfortable with it before I use it in more serious applications.

Gregory Szorc: Thoughts on Mercurial (and Git)

Sun, 12 May 2013 12:00:00 +0000

My first experience with Mercurial (Firefox development) was very unpleasant. Coming from Git, I thought Mercurial was slow and perhaps even more awkward to use than Git. I frequently encountered repository corruption that required me to reclone. I thought the concept of a patch queue was silly compared to Git branches. It was all extremely frustrating and I dare say a hinderance to my productivity. It didn't help that I was surrounded by a bunch of people who had previous experience with Git and opined about every minute difference.

Two years later and I'm on much better terms with Mercurial. I initially thought it might be Stockholm Syndrome, but after reflection I can point at specific changes and enlightenments that have reshaped my opinions.

Newer versions of Mercurial are much better

I first started using Mercurial in the 1.8 days and thought it was horrible. However, modern releases are much, much better. I've noticed a steady improvement in the quality and speed of Mercurial in the last few years.

If you aren't running 2.5 or later (Mercurial 2.6 was released earlier this month), you should take the time to upgrade today. When you upgrade, you should of course read the changelog and upgrade notes so you can make the most of the new features.

Proper configuration is key

For my workflow, the default configuration of Mercurial out of the box is... far from optimal. There are a number of basic changes that need to be made to satisfy my expectations for a version control tool.

I used to think this was a shortcoming with Mercurial: why not ship a powerful and useful environment out of the box? But, after talking to a Mercurial core contributor, this is mostly by design. Apparently a principle of the Mercurial project is that the CLI tool (hg) should be simple by default and should minimize foot guns. They view actions like rebasing and patch queues as advanced and thus don't have them enabled by default. Seasoned developers may scoff at this. But, I see where Mercurial is coming from. I only need to refer everyone to her first experience with Git as an example of what happens when you don't aim for simplicity. (I've never met a Git user who didn't think it overly complicated at first.)

Anyway, to get the most out of Mercurial, it is essential to configure it to your liking, much like you install plugins or extensions in your code editor.

Every person running Mercurial should go to http://mercurial.selenic.com/wiki/UsingExtensions and take the time to find extensions that will make your life better. You should also run hg help hgrc to view all the configuration options. There is a mountain of productivity wins waiting to be realized.

For reference, my ~/.hgrc. Worth noting are some of the built-in externsions I've enabled:

color - Colorize terminal output. Clear UX win.
histedit - Provides git rebase --interactive behavior.
pager - Feed command output into a pager (like less). Clear UX win.
progress - Draw progress bars on long-running operations. Clear UX win.
rebase - Ability to easily rebase patches on top of other heads. This is a basic feature of patch management.
transplant - Easily move patches between repositories, branches, etc.

If I were on Linux, I'd also use the inotify extension, which installs filesystem watchers so operations like hg status are instantaneous.

In addition to the built-in extensions, there are a number of 3rd party extensions that improve my Mozilla workflow:

mqext - Automatically commit to your patch queue when you qref, etc. This is a lifesaver. If that's not enough, it suggests reviewers for your patch, suggests a bug component, and let's you find bugs touching the files you are touching.
trychooser - Easily push changes to Mozilla's Try infrastructure.
qimportbz - Easily import patches from Bugzilla.
bzexport - Easily export patches to Bugzilla.

I'm amazed more developers don't use these simple productivity wins. Could it be that people simply don't realize they are available?

Mozilla has a bug tracking easier configuration of the user's Mercurial environment. My hope is one day people simply run a single command and get a Mozilla-optimized Mercurial environment that just works. Along the same vein, if your extensions are out of date, it prompts you to update them. This is one of the benefits of a unified developer tool like mach: you can put these checks in one place and everyone can reap the benefits easily.

Mercurial is extensible

The major differentiator from almost every other version control system (especially Git) is the ease and degree to which Mercurial can be extended and contorted. If you take anything away from this post it should be that Mercurial is a flexible and agile tool.

If you want to change the behavior of a built-in command, you can write an extension that monkeypatches that command. If you want to write a new command, you can of course do that easily. You can have extensions interact with one another - all natively. You can even override the wire protocol to provide new capabilities to extend how peers communicate with one another. You can leverage this to transfer additional metadata or data types. This has nearly infinite potential. If that's not enough, it's possible to create a new branching/development primitive through just an extension alone! If you want to invent Git-style branches with Mercurial, you could do that! It may require client and server support, but it's possible.

Mercurial accomplishes this by being written (mostly) in Python (as opposed to C) and by having a clear API on which extensions can be built. Writing extensions in Python is huge. You can easily drop into the debugger to learn the API and your write-test loop is much smaller.

By contrast, most other version control systems (including Git) require you to parse output of commands (this is the UNIX piping principle). Mercurial supports this too, but the native Python API is so much more powerful. Instead of parsing output, you can just read the raw values from a Python data structure. Yes please.

Since I imagine a lot of people at Mozilla will be reading this, here are some ways Mozilla could leverage the extensibility of Mercurial:

Command to create try pushes (it exists - see above).
Record who pushed what when (we have this - it's called the pushlog).
Command to land patches. If inbound1 is closed, automatically rebase on inbound2. etc. This could even be monkeypatched into hg push so pushes to inbound are automatically intercepted and magic ensues.
Record the automation success/fail status against individual revisions and integrate with commands (e.g. only pull up to the most recent stable changeset).
Command to create a review request for a patch or patch queue.
Command to assist with reviews. Perhaps a reviewer wants to make minor changes. Mercurial could download and apply the patch(es), wait for your changes, then reupload to Bugzilla (or the review tool) automatically.
Annotating commits or pushes with automation info (which jobs to run, etc).
Find Bugzilla component for patch (it exists - see above).
Expose custom protocol for configuring automation settings for a repository or a head. e.g. clients (with access) could reconfigure PGO scheduling, coalescing, etc without having to involve RelEng - useful for twigs and lesser used repositories.
So much more.

Essentially, Mercurial itself could become the CLI tool code development centers around. Whether that is a good idea is up for debate. But, it can. And that says a lot about the flexibility of Mercurial.

Future potential of Mercurial

When you consider the previous three points, you arrive at a new one: Mercurial has a ton of future potential. The fact that extensions can evolve vanilla Mercurial into something that resembles Mercurial in name only is a testament to this.

When I sat down with a Mercurial core contributor, they reinforced this. To them, Mercurial is a core library with a limited set of user-facing commands forming the stable API. Since core features (like storage) are internal APIs (not public commands - like Git), this means they aren't bound to backwards compatibility and can refactor internals as needed and evolve over time without breaking the world. That is a terrific luxury.

An example of this future potential is changeset evolution. If you don't know what that is, you should because it's awesome. One of the things they figured out is how to propagate rebasing between clones!

Comparing to Git

Two years ago I would have said I would never opt to use Mercurial over Git. I cannot say that today.

I do believe Git still has the advantage over Mercurial in a few areas:

Branch management. Mercurial branches are a non-starter for light-weigh work. Mercurial bookmarks are kinda-sorta like Git branches, but not quite. I really like aspects of Git branches. Hopefully changeset evolution will cover the remaining gaps and more.
Patch conflict management. Git seems to do a better job of resolving patch conflicts. But, I think this is mostly due to Mercurial's patch queue extension not using the same merge code as built-in commands (this is a fixable problem).
Developer mind share and GitHub. The GitHub ecosystem makes up for many of Git's shortcomings. Bitbucket isn't the same.

However, I believe Mercurial has the upper hand for:

Command line friendliness. Git's command line syntax is notoriously awful and the concepts can be difficult to master.
Extensibility. It's so easy to program custom workflows and commands with Mercurial. If you want to hack your version control system, Mercurial wins hands down. Where Mercurial embraces extensibility, I couldn't even find a page listing all the useful Git extensions!
Open source culture. Every time I've popped into the Mercurial IRC channel I've had a good experience. I get a response quickly and without snark. Git by contrast, well, let's just say I'd rather be affiliated with the Mercurial crowd.
Future potential. Git is a content addressable key-value store with a version control system bolted on top. Mercurial is designed to be a version control system. Furthermore, Mercurial's code base is much easier to hack on than Git's. While Git has largely maintained feature parity in the last few years, Mercurial has grown new features. I see Mercurial evolving faster than Git and in ways Git cannot.

It's worth calling out the major detractors for each.

I think Git's major weakness is its lack of extensibility and inability to evolve (at least currently). Git will need to grow a better extensibility model with better abstractions to compete with Mercurial on new features. Or, the Git community will need to be receptive to experimental features living in the core. All of this will require some major API breakage. Unfortunately, I see little evidence this will occur. I'm unable to find a vision document for the future of Git, a branch with major new features, or interesting threads on the mailing list. I tried to ask in their IRC channel and got crickets.

I think Mercurial's greatest weakness is lack of developer mindshare. Git and GitHub are where it's at. This is huge, especially for projects wanting collaboration.

Of all those points, I want to stress the extensibility and future potential of Mercurial. If hacking your tools to maximize potential and awesomeness is your game, Mercurial wins. End of debate. However, if you don't want to harness these advantages, then I think Git and Mercurial are mostly on equal footing. But given the rate of development in the Mercurial project and relative stagnation of Git (I can't name a major new Git feature in years), I wouldn't be surprised if Mercurial's feature set obviously overtakes Git's in the next year or two. Mind share will of course take longer and will likely depend on what hosting sites like GitHub and Bitbucket do (I wouldn't be surprised if GitHub rebranded as CodeHub or something some day). Time will tell.

Extending case study

I have removed the case study that appeared in the original article because as Mike Hommey observed in the comments, it wasn't a totally accurate comparison. I don't believe the case study significantly added much to the post, so I likely won't write a new one.

Conclusion

From where I started with Mercurial, I never thought I'd say this. But here it goes: I like Mercurial.

I started warming up when it became faster and more robust in recent versions in the last few years. When I learned about its flexibility and the fundamentals of the project and thus its future potential, I became a true fan.

It's easy to not like Mercurial if you are a new user coming from Git and are forced to use a new tool. But, once you take the time to properly configure it and appreciate it for what it is and what it can be, Mercurial is easy to like.

I think Mercurial and Git are both fine version control systems. I would happily use either one for a new project. If the social aspects of development (including encouraging new contributors) were important to me, I would likely select Git and GitHub. But, if I wanted something just for me or I was a large project looking for a system that scales and is flexible or was looking to the future, I'd go with Mercurial.

Mercurial is a rising star in the version control world. It's getting faster and better and enabling others to more easily innovate through powerful extensions. The future is bright for this tool.

Erik Vold: Old School To Jetpack Part 1 - The Retrofit

evold@mozilla.com (Erik Vold) — Sun, 12 May 2013 07:00:00 +0000

Old school add-ons were developed with a similar mindset, which is that creating new windows is ok, and using XUL this became an easy to-do common practice. The times have changed, and opening new windows is no longer considered acceptable practice, opening tabs is now the preferred user experience.

Now, Firefox has a huge cast of popular, restart addled, old school mannered add-ons, which is a problem. So how do we retrofit these geezers?

I see three parts that need to be addressed:

New XUL windows (windows created by the add-on)
XUL overlays (content added to windows created by Fx or another add-on)
Back-end logic

I plan to write more about all three topics later, because the real solutions require more words and work than the shortcuts will.

Shortcuts

Chrome.manifest

Because of Bug 559306 we can now include the old school add-on’s chrome.manifest file in jetpacks. There are some limitations however, for more information on that see this page on the chrome.manifest in restartless add-ons. For instance, overlays will not be regestered, resource uris, components, and more. One can register a chrome uri however, with a skin and locale.

This means that add-ons which implement XUL windows can continue to use them as a Jetpack.

These windows should be redesigned however, but this is no longer a bar to entry, and the resdesign of these windows can be done in a separate step from the conversion to Jetpack.

Also, remember that any windows open which are associated to an add-on should be closed when the add-on unloads, if you do not do this then your add-on will not pass a full review on addons.mozilla.org.

Browser UI

Again, there are a few useful third party modules which handle most common use cases, and if they do not cover your use case then they should provide some direction how to implement them.

I’ve got a module in mind that should help with converting overlays, it’s a messy prototype right now though, so we’ll see how that goes..

Components

There are a few useful third party modules out already which handle things like registering custom uris (this includes about: uris, resource: uris, and whatever:), content policies, and in many cases there are better alternatives. For instance I’ve seen quite a few add-ons register components just in order to hear their ‘startup’ event, this is no longer necessary, with a Jetpack you can simply call require('self').loadReason in main.js.

Pascal Finette: The Web Manifesto

pascal@finette.com (Pascal Finette) — Sun, 12 May 2013 00:44:00 +0000

We have come to build the most important thing in the world today, perhaps the most important thing ever. It connects us. It widens us. It deepens who we are. We don’t know what it is or what it will be in the future, but we do know it has made us better so far. Our dream is to enlarge it so that all people can join us and share the good in it while ameliorating the bad. This thing we are working on has no borders and – as far as we can see – no end. If everyone can join it, with equal access and no undue ownership, the world will be a much better place. That is why we are working here today.

~ Kevin Kelly / WIRED 21.05

Jared Wein: Firefox OS at Mobile Monday Detroit

msujaws — Sat, 11 May 2013 16:39:34 +0000

If you haven’t heard about Firefox OS before, or would like to learn more about it, you should come to this month’s Mobile Monday Detroit. I’ll be presenting at the meeting about Mozilla’s motivations for Firefox OS, how we’re opening up mobile hardware, and roadmap for the future.

It has been estimated that in the next 10 years, five billion people will gain access to the internet. Many of these people will be accessing the internet using mobile connection exclusively.

One of the main goals of Firefox OS is to provide a device to people that brings with it the full web as compared to the limited subset of the web that is currently accessible on more basic feature phones. This also means providing web applications better access to hardware, so application developers can provide rich environments comparable to that of other mobile phone operating systems.

As we build these hardware APIs, we have been working very hard to standardize them and push for other browsers to adopt them. You can follow along with the progress of the WebAPI project by visiting our wiki. All of the FirefoxOS roadmaps, feature lists, etc can all be viewed online. This project really defines openness

Mobile Monday Detroit is hosted at Compuware’s headquarters in downtown Detroit, at One Campus Martius. The event will start at 5:30pm and run until 8pm. Food and drinks will be provided. Please RSVP (it’s free!) if you plan on attending so the organizers will have a good idea of how much food to order.

Randy Nunez from Ford will also provide information on a variety of additional open source mobile OS platforms that are catching the attention of the developer community.

Monday, May 13, 2013
5:30 PM to 8:00 PM
Compuware Building
One Campus Martius, Detroit, MI (map)

Tagged: detroit, firefox, firefoxos, mozilla, planet-mozilla, presentation

Erik Vold: Blocking Content With Jetpacks

evold@mozilla.com (Erik Vold) — Sat, 11 May 2013 07:00:00 +0000

Many of the most popular add-ons have a common function, they block content. Adblock Plus blocks ads, NoScript blocks scripts, and Ghostry blocks tracking software which uses images, scripts and other means to do their business.

This is why I wrote a content-policy module for Mozilla’s Add-on SDK. The API is rough at the moment, and I plan to change that quite a bit. For the moment though we can take the No Google Analytics add-on as an example source here:

lib/main.js

const { ContentPolicy } = require('content-policy');
const GA_REGEX = /(.*\.)?google-analytics\.com$/i;
const { URL } = require('sdk/url');

ContentPolicy({
  shouldLoad: function({ location }) {
    // if the hist is google-analytics.com then block it
    if (URL(location).host.match(GA_REGEX))
      return false; // returning false rejects the request before it starts
    return true; // returning true does not reject the request
  }
});

I’ll improve the match later, but you can see how easy that is, and I promise I’ll make it easier.

Selena Deckelmann: TIL: Formatting, search_path and colorcolumn

selena — Fri, 10 May 2013 22:34:24 +0000

The last six months have involved a lot more writing of code than the previous couple of years.

I’ve been tweeting little things I learn on a daily basis and thought I’d look back on this week.

format()

A reocurring problem with report writing is getting numbers formatted properly for the occassion. I discovered ‘format’ in Python this week:

print "{0:.2f}%".format(float(1)/3 * 100)

That prints out a float to 2 decimal places. I looked around and Dive Into Python has similar syntax, but without the format() function. So, the equivalent would be:

print "blah %.2f" % (float(1) / 3 * 100)

So, why use one over the other? A user on StackOverflow suggested that compatibility with 2.5 might drive a person to use ‘%’ over ‘format()’, but otherwise, the poster suggested that format() is the cleaner looking and more flexible choice.

set search_path = bixie

I’m working on a new schema for a project. We’re rolling out a prototype quickly, so we’re going to house it in our existing production database for now. To keep things easy to clean up, Laura suggested that we put things into a separate schema. For managing our database models, I’ve switched to using SQLAlchemy, and also alembic for migrations. This made it super easy to specify that I wanted all the Bixie related tables in their own schema:

class BixieCrash(DeclarativeBase):                                              
    __table_args__ = {'schema': 'bixie'}                                        
    __tablename__ = 'crashes'

And that was it.

Then, to avoid having to add ‘bixie.’ to all the table paths in test queries, I put this command into the tests:

 cursor.execute(""" SET search_path TO bixie """)

I imagine there are some other ways to handle this. We’re not really using the ORM for anything other than schema loading, so I’ll probably add that to our connection initialization code for the new app. Then developers can write their queries as without any concerns about being in the correct schema.

And I’ll glow just a little bit about deploying alembic on stage!

set colorcolumn=80

I’ve been trying to write prettier Python. Today’s micro-effort was figuring out how display a vertical line to tell me when I exceed the 80 character width. The proper command to add to .vimrc is:

:set colorcolumn=80

Which looks something like:

Doug Belshaw: Weeknote 19/2013

Doug Belshaw — Fri, 10 May 2013 19:03:52 +0000

This week I’ve been:

Taking a day off. It was Bank Holiday on Monday – a national holiday in the UK. I still however spent 4pm-5pm… <drumroll>
Hosting the weekly Web Literacy standard call. We motored through our first pass of defining the skills under the competencies in the ‘Connecting’ strand.
Writing a post for Week 1 of the Mozilla #teachtheweb MOOC: How transferable are coding skills to other domains? Why is learning a little code important?
Responding to enquiries by people and organisations about integrating with the OBI.
Travelling to and from London to meet with Lord Jim Knight and STiR education about using Open Badges for teacher education in India.
Enjoying a conversation over lunch London with the ever-enthusiastic Eugenie Teasley from Spark + Mettle.
Collating questions about Open Badges and then answering them in this blog post.
Suffering from a migraine on Thursday. I couldn’t see much due to the aura so I called it a day about 10:30am. I lay down and listened to podcasts. The Moral Maze episode on The Ring of Gyges was fascinating.
Travelling to BBC North in Salford to deliver a session on Open Badges. It went pretty well, but I felt like I wasn’t getting my words out properly or explaining things as well as I usually do. It’s often an issue post-migraine. Slides here.

Next week, after five straight weeks of travelling and hotels, I’m home for the entire week. Woohoo! The week after I’m in Toronto for the Mozilla All-Hands meeting, so plenty to psych myself up for…

Will Kahn-Greene: My thoughts on Elasticsearch: Part 1: indexing

Fri, 10 May 2013 17:49:00 +0000

Summary

I just finished up an overhaul of ElasticUtils and then an overhaul of the search infrastructure for support.mozilla.org. During that period of time, I thought about extending the ElasticUtils documentation to include things I discovered while working on these projects. Then I decided that this information is temporal---it's probably good now, but might not be in a year. Maintaining it in the ElasticUtils docs seemed like more work than it was worth.

Thus I decided to write a series of blog posts.

This one covers indexing. Later ones will cover mappings, searching and other things.

It's also long, rambling and contains code. The rest is after the break.

Kim Moir: Releng 2013 keynotes: John O'Duinn (Mozilla) and Roman Scheiter (LinkedIn)

noreply@blogger.com (Kim Moir) — Fri, 10 May 2013 14:44:00 +0000

There are two exciting keynotes planned for Releng 2013

John O'Duinn, Director of Release Engineering at Mozilla will kick off the workshop with his keynote Release Engineering as a Force Multiplier. The build and release process used to be a pain point at Mozilla, but now makes the company and community more productive as a whole. John will describe how the team added support for project branches to allow concurrent development, rethought continuous integration and increased capacity by moving to a hybrid-cloud build infrastructure. These changes improved several aspects of the business, including switching to a rapid release model and reducing turnaround time on a release from weeks to hours. As a result, Mozilla improved its abilities against much bigger and better funded competitors in the marketplace while also allowing them to enter new markets and help ensure its long-term success.

Roman Scheiter, Engineering Services Director at LinkedIn, will present the afternoon keynote entitled Against All Odds – Completely Overhauling Linkedin's Release Process. This session will cover the evolution of LinkedIn’s release process from its earliest days to the point where the rapidly growing engineering team necessitated a radical shift. This shift, an executive sponsored effort to address technical debt and introduce new thinking to boost engineering efficiency allows six hundred developers to release thousands of changes per week without compromising quality. As part of this undertaking, LinkedIn learned many best practices, developed tools and custom infrastructure, and lived through the internal cultural changes needed to make this independent release process work. Roman will detail the evolution and results of this shift so you can learn directly from LinkedIn's pain.

In addition to these fantastic keynotes, we also have talks from release engineers and researchers from Netflix, Google, Microsoft, Gnome, Red Hat, IBM, several universities and more! We'll also have a panel at the end of the day to discuss the future of release engineering.

Check out the full program on the Releng 2013 site. To register for the conference, which is managed as part of the larger ICSE conference, you can follow this link and choose the one-day-workshop. See you in San Francisco on May 20!

Laura Hilliger: Planning, Running, Visualizing #teachtheweb

Laura Hilliger — Fri, 10 May 2013 13:14:42 +0000

Week 3 of #teachtheweb is right around the corner, and I thought it might be nice to do a quick shareout of the planning and running of the MOOC.

Kudos!

Before I say another word, I have to give my partner in crime a gigantic love bomb. The word “collaboration” only begins to describe the epic brain connection Michelle and I have seemingly developed. Additionally, as Michelle noted, the entire Webmaker Mentor team has been massively supportive and absolutely integral to getting Mozilla's first MOOC off the ground. High-fives all around!

Conspiring

We've been having calls with wonderful community members who signed up to help create the #teachtheweb experience. We've been calling them the Super Mentor Calls. The Super Mentor Kick-off call on April 19 had 41 attendees from 17 different countries. They are designers, developers, educators, makers, architects, librarians, youth workers, entrepreneurs, teens and more, who answered our call to be super mentors in #teachtheweb. This call was used to help the Super Mentors understand what we were asking them to do throughout the 9-week course. This blog post explains the job. We also used the opportunity to explain the concept of a cMOOC and to thank everyone for signing up before they even knew what they signed up for. Follow our Twitter List of Super Mentors The next week, April 25, we took some time to celebrate Super Mentor makes after many created introductions using Webmaker tools. We gave each other virtual high fives for being engaging, thoughtful digital citizens and shared a few moments of excitement centered around the fact that the Webmaker Community is full of intelligent human beings looking to change the world. Next, we explained our launch strategy with a “Heads up! It's going to get busy!” and proposed the formation of Study Groups. We created a lightweight google doc and asked if anyone would be interested in running a smaller study group during the experience. Many were, and the Google Doc started filling up with ideas for Study Groups based on interest, language and/or geography. We also talked about the content creation procedure and came up with a plan to braindump into Etherpads, filter and curate. We want to make sure any one who wants to contribute to the content of #teachtheweb can. If you want to help plan, see the planning page! We talked about the connection between #teachtheweb and #MakerParty (#teachtheweb is kind of like Party Prep), and the Super Mentors began to volunteer to spread the word about both initiatives in their local areas as well as online. Finally, Super Mentors volunteered to help moderate the various #teachtheweb channels during the May 2^nd Live Session and #teachtheweb Kick-off. On the #teachtheweb launch date, May 2^nd, we had a relatively quick call with Super Mentors to celebrate the G+ Community's growth and talk about the Live Session. Super Mentors signed up to participate via video, monitor channels, serve as tech support and help out on IRC. The session went, I heard, nicely. You can watch the video here. Yesterday, May 9^th, we spent a little time talking about how to get participants more involved and how to be better at sharing with each other and the larger community. We decided to try to give each other weekly report backs on how Study Groups were doing in the following Super Mentor calls.

Content Creation

Each week a post is released in the Planning section of #teachtheweb, which invites any and everyone to contribute their thoughts to a particular topic. The Super Mentors get extra reminders, but the content creation process is completely open. If you're interested in contributing your thoughts to a particular topic, check the planning page for new topic etherpads each Monday. Monday is also the day that we've been distilling these Etherpads into a digestible blogpost with extra readings, resources and tasks. The filtering and curation of everyone's braindump is massively interesting but quite difficult. The posts need to be relatively short to keep people engaged. We would rather that people spend their allotted MOOC time completing the weekly Make Projects and reflecting on their own work, instead of focusing too heavily on truncated descriptions of complex topics. We believe in Making as Learning, so we want to spend as much time as possible making, sharing, remixing, iterating and making some more. Tuesday's we post new topics to the #teachtheweb site. We also do shareouts in the Weekly Webmaker Community call. Thursday's are the days we do live sessions (only 3 throughout the 9 weeks, one on May 2^nd, May 23^rd and June 13^th) and Twitter Chats. Which leads me to the visualization piece of this post. This morning, I really just wanted to MAKE stuff. So I started fishing through the Twitter archive Jeannie Crowley created for #teachtheweb. Then I got a data bug and started looking at other numbers. I made two things:

A Visualization of Relationships between Words used in #teachtheweb Tweets

[caption id="attachment_1916" align="aligncenter" width="800"] Click the image to play with this interactive visualization thingie[/caption]

A Slightly Interactive Infographic

[iframe src="//infogr.am/teachtheweb-27148" width="550" height="2161" scrolling="no" frameborder="0" style="border:none;"]

teachtheweb | Infographics

MOOC Attrition

One of the things that has been bugging me is the rumors of MOOC attrition rates. I've read percentages from 85% to 98% in the last couple of months, and I was worried that the participants in #teachtheweb will soon be dropping out and the conversation will come to a standing halt. Up until a few days ago, I was basically waiting for the ball to drop. Then I read this post from a mathematician at Stanford and decided to just stop worrying about it. It's easy to sign up for things. You just put in one of your many email addresses and that's that. It's easy to begin the journey into a specific community. You just use the hashtag, comment on people's posts, follow some people and start trying to know people through their digital representations and artifacts. But it's not easy to remain an active member of a community. Being active takes time, perseverance, patience, understanding, collaboration, connection, mentorship, and a whole mess of other things. We all participate in multiple communities, and it's natural that one community or another will take precedence on a given day. I just hope that the those of you who have found the Webmaker Mentor Community through #teachtheweb decide to give this community precedence from time to time. We're teaching the world the Web – and we definitely need your help.

Brandon Savage: Making better object oriented design decisions

Brandon Savage — Fri, 10 May 2013 11:00:36 +0000

Last week I finished up the first section of The Object Oriented PHP Masterclass. The class was great, and I learned about as much from my students as they learned from me. One of the most common questions I got from my students was “how do I make decisions about how to design my object [...]

Christian Heilmann: #justcode

Chris Heilmann — Fri, 10 May 2013 09:50:29 +0000

As developers we are incredibly lucky. We work in a very growing and immensely well paid market, our companies shower us with benefits, companies offer us jobs rather than having to send out hundreds of CVs on the off-chance and even the mass media and politicians start talking about “coding” being a skill everybody needs.

Quite some part of this success is based on the stubbornness we showed in the past. When we got a task to build something we didn’t give up on it and said it is impossible. Instead we went back in our corner and tried and failed and tried again with sparks flying and code explosions happening until we achieved what we wanted. Think Dr. Bunsen Honedew’s laboratory instead of Statler and Waldorf.

This gave especially the web a strange “hack it together” reputation that many people keep bringing up when it comes to replacing JavaScript for example with “more organised and professional” languages. But you know what? I really think when it comes to the web, this is its main strength.

The fun of coding

As explained earlier in my Flash is not the enemy post, whimsy and spontaneous ideas is what made the web a larger media outlet than it was. It wasn’t the large sites that got non-technical people excited. It was the funny animation and short-lived game that you could mail to your friends.

Therefore I think it is important to celebrate this for yourself from time to time. Personally I find myself extremely lucky to have been at the right time (and moving around to the right places) when the web exploded into an offering of amazingly cool things and while I am sure as hell not proud of the code I had to write to get things done in the past, I am happy that I did and that I didn’t give up or wait until someone else solves my problems for me.

Having just taught a workshop on HTML5 at Industryconf I found that we are losing a bit on that. Attendees were worried that they need to learn a lot of libraries and find the right plugins to get started and once shown that they have the power to do most of what they want using the things browsers come with out of the box got quickly into enjoying themselves reaching new levels.

One thing I did with the attendees is a To-Do List App in plain HTML/JS/CSS (No sound).

This is what the Mozilla Webmaker Project is about – to get non-programmers excited about building things for the web. And it is incredibly exciting to see some of these events as a “professional”.

I think it is very important to never forget about the wonder we experienced the first time we made something show up on screen or wrote our first condition that printed out “is amazing” when you entered your name or “is boring” when it was another one.

Be fearless

A lot of times being creative means being fearless. Watching Bret Victor’s talks and seeing his Learnable programming course and Seb Lee-Delisle’s training courses they consist of one main thing – play with things and worry about them breaking later. Amazing results happen when the outcome and the input get as close together as possible – not when things happen using dozens of abstractions.

This does not have to be visual from the get-go though. The MPEG-1 decoder in pure JavaScript for example is pure byte-shifting but blew me away in its fearlessness of what could go wrong.

Go, code!

Why not have a go? Take 10 minutes, half and hour, an hour out of your life right now and use it to #justcode something, anything. Just play with an idea, put it on JSFiddle, Codepen, JSBin, Dabblet, or whatever other amazing tool we have right now and share it.

Don’t build a perfect plugin, don’t build a solution dependent on preprocessors and libraries. Go vanilla and just play with what we have in browsers today. CSS Animations and Transitions, Canvas, Audio and Video, HTML5 and Friends – we have so many cool toys to play with. Don’t explore the main use case either. Yes, Canvas is for putting things on the screen, but it is also about reading image data.

We got were we are by playing with things. Never forget this and never stop playing.

Just Browsing: Programming Brain Teaser #6: Fixed-point principle

Roman Kaspar — Thu, 09 May 2013 08:00:49 +0000

As promised last time, let’s do something more serious today…

Let a be any expression. We will call x the fixed point of expression a if x names ax. (Reminder: lower-case letters in our programming language represent a combination of upper-case letters.)

The fixed-point principle says that for each expression a there exists its fixed point x. Furthermore there is a (quite simple) “manual” for constructing the fixed point x of any expression a.

So the question of the day is: can you find that “manual” proving fixed-point principle?
(E.g. a program x that names ABCx, for a = ABC.)

Solution

To see the correct solution, click 'Show' button...	Select Show
Answer: *x = RQaRQ* Why? Because that names *aRQaRQ*, which is ax, which is exactly what we wanted .

The post Programming Brain Teaser #6: Fixed-point principle appeared first on Salsita Software.

Byron Jones: happy bmo push day!

glob — Thu, 09 May 2013 07:12:42 +0000

the following changes have been pushed to bugzilla.mozilla.org:

[869025] create a report to sanity check product’s default security group
[869220] SecureMail message when no key is set doesn’t seem to be grammatically correct.
[869261] In the bug entry form the default security group should be displayed to let user know which group will be set
[863686] “Congratulations” email received for first approval+ instead of first review+
[652334] splinter doesn’t support hg/git ‘rename’ or ‘copy’
[695662] splinter doesn’t support bzr rename
[709897] splinter doesn’t like hg diffs where a file was copied and then modified
[685645] make it clearer in splinter that a hunk’s section/function header is not a removal
[870109] Project Kickoff Form: Doesn’t create Security Review bugs

Filed under: bmo, mozilla

Tantek Çelik: Reforming the W3C Advisory Board

Thu, 09 May 2013 01:32:00 +0000

I am running for the W3C Advisory Board (AB) for one of four open seats on a reform and openness platform. If your company or organization is a member of the W3C (World Wide Web Consortium), please ask your AC Representative to vote (W3C member-only link) for myself and Chris Wilson in support of more openness and reform.

In short, the W3C Advisory Board keeps track of W3C process issues (W3C member-only link) and ... manages the evolution of the [W3C] Process Document. It also advises the W3C Team on topics like licensing.

I have participated in a couple of Advisory Board meetings by invitation in the past. I was asked to contribute experience and opinions related to more rapid and more open standards development practices. Though cautiously optimistic at the time, I have to admit that I have not seen as much progress as I'd hoped or expected. Community Groups with a more open license are nice but insufficient exception. The recently revised HTML Working Group Charter is another (more on that below).

So with that experience, why am I running? Two big reasons.

First, in the recent W3C TAG election, everyone who was elected was from the reform slate. The remaining unelected reformer was subsequently appointed. Politically, it seems the W3C membership and leadership are now more supportive of reform than in the past, top down, starting with the Technical Architecture Group.

Second, there are additional reformers running for the AB who are dedicated to process simplification and more openness, in particular, more open licensing:

Chris Wilson (Google)
... (awaiting confirmation) ...

If have you have W3C Member Access, you can read all nominees' statements. I'll update this list as I receive public confirmation from fellow reformers.

From my experience interacting with the other existing/continuing AB members, I know that some of them do express a desire for simplification and are supportive of more openness, both in process and in the specifications we produce (e.g. with more open licensing).

If we are able to elect a set of reformers to the AB, they plus the existing reform-friendly members have a good chance of effecting real change in the AB, and thus potentially the W3C as a whole.

My official nomination statement is posted at W3C but as that link is member-only (no reason for it to be in my opinion), I'm republishing it here, openly since I wrote it. Pardon the third person perspective.

Tantek Çelik is Mozilla's web standards lead and a 15 year participant in the W3C. Having contributed to working groups such as CSS and HTML, he has helped edit and produce several key Recommendations for the open web platform including: CSS 2.1, Selectors, and CSS3 Color.
Tantek's contribution to open web standards began with leading the implementation of the Tasman rendering engine in Microsoft Internet Explorer 5 for Macintosh, a watershed achievement of solid CSS1, HTML4, and PNG 1.0 support in the year 2000.
In addition to W3C activities, Tantek participated in the development of vCard4 at IETF, contributes to the WHATWG, and co-founded the microformats.org standards community.
As a longstanding champion of pushing for increasing openness in the methods and means of open standards development, he has both helped working groups (like CSS) do so, and has written extensively on the topic, e.g.: http://tantek.com/2011/168/b1/practices-good-open-web-standards-development
Tantek hopes to bring his practical experience working with (and helping evolve) a variety of standards organization models and processes to the W3C Advisory Board, to improve and modernize W3C's processes accordingly.
Tantek holds B.S. & M.S. degrees in Computer Science from Stanford University and shares his thoughts on his personal website: http://tantek.com/

There are two big changes in particular that I'd like to help make happen:

More open licensing. W3C Community Groups are allowed to publish with an open license (though a bespoke one W3C came up with). Official W3C Working Groups are not. My goal here is to have W3C adopt a standard open license (e.g. CC0 or CC-BY) for all specifications moving forward. Other modern standards organizations and communities like the WHATWG and microformats.org have already done so (with CC0) to great benefit (and no apparent downside). Just last week a new HTML Working Group charter was proposed that allows for the possibility of publishing HTML Extension Specifications with CC-BY. This is an excellent first step. W3C as a whole can benefit from more open licensing.
More open process. 15 years ago the CSS Working Group conducted the vast majority of its technical discussions behind closed doors, member-only mailing lists. Today the opposite is true, CSS is developed in the open, on open IRC channel, open wiki, and open email lists. I believe we can and should do the same for the process of running the W3C itself. Transparency in how we create standards is an end in its own right and the more we practice it, the more confidence, support, and trust we can build in a broader community of the work that we do.

If you work for a member of the W3C, please consider asking your AC Representative (W3C member-only link) to vote for myself and Chris Wilson for the W3C Advisory Board (W3C member-only link). I'll update the list of reform candidates as I receive confirmation of their public positions, and link to their blog posts accordingly. Thanks for your consideration.

Justin Lebar: MemShrink process, weeks 97-98

Thu, 09 May 2013 00:45:00 +0000

Nicholas Nethercote is on vacation, so this fortnight's MemShrink report is a guest post written by Andrew McCreight.

Images

Joe Drew made it so that on B2G images being displayed on the active page aren't locked, allowing them to be discarded when there is memory pressure. This is important to display image-heavy pages like Pinterest on B2G without OOM crashes.

Andreas Gal added a preference to control the size of the canvas image cache. The use of canvas with large images was causing OOM crashes for the B2G Gallery app. The limit is set to 10mb for B2G, and remains unlimited on desktop.

Leak fixes

Scoobidiver noticed a large increase in empty crashes on Firefox 23. Previously, these have been identified as being caused by leaks of virtual address space. Scoobidiver and Benjamin Smedberg investigated, found a possible culprit, and saw that the number of empty crashes went down to its previous level, or even lower, after some followup work for that bug was landed.

Timothy Nikkel fixed a leak caused by some recent changes to reflow-on-zoom.

Justin Lebar investigated a leak involving long-running automated testing on B2G and determined that it was the same layers leak previously found and fixed, but not backported to B2G18, by new contributor Christophe Mourand.

Randell Jesup fixed a WebRTC leak.

Miscellaneous

Brian Hackett reduced the memory usage of IonMonkey compilation, which should help on things like JS games that intensively run a lot of JS.

Nicholas Nethercote wrote a patch to change how decommitted GC arenas are shown in about:memory. Decommitted arenas don't actually use physical memory, so they need to be accounted for differently.

Boris Zbarsky reduced the memory usage of CSS in certain cases.

Geoffrey MacDougall: Rethinking Small Dollar Fundraising

Geoffrey — Wed, 08 May 2013 18:42:05 +0000

Money-making
As a non-profit, Mozilla has access to four types of revenue: grants, gifts, donations, and earned income. Our earned income streams come through Firefox, which leaves the other three – grants, gifts, and donations – to be driven by our fundraising strategy (the focus of this post).

So far, and while we can always get better, we’re pretty good at securing grants. We’ve built meaningful working relationships with large philanthropies and government agencies, which contribute approximately $5M each year to Webmaker, Open Badges, and our policy work.

We don’t have a major gifts program (as yet; more on this later in the year).

And, while we’ve spent a fair amount of time building a small dollar donations program, it’s definitely not what it could be. Last year, we generated around $750K from e-mails to more than 500K people and an end-of-year presence in the main Firefox channels.

There are many and valid reasons for this result. Fundraising as a large-scale social enterprise is challenging and I believe we’ve done well to get where we are. But given our reach and the importance of the Web there is massive room for improvement.

NOW: Help us get better at fundraising. Tell us what you think about Mozilla, our mission, and our fundraising.

Solving Small Dollar Fundraising
Building an effective small dollar program is important for several reasons:

It’s the most stable form of revenue available to the Foundation: small amounts of funding from a large and diverse group of donors;
It mitigates risk by distributing external dependence and increasing revenue diversity; and
It’s a meaningful contribution path for a large number of Mozillians who don’t have the time or capacity to code.

Grants and (eventually) gifts will always be crucial to the health and growth of Mozilla. They allow us to work with amazing partners, test new ideas, and launch new products. But building a sustainable small dollar program is our top development priority for 2013.

Enter the Study
Good fundraising – like many things – involves constantly questioning and challenging assumptions. We’ve used the first part of year to hold conversations to gain perspective and input from the Mozilla community: people who care about our mission and want to help it succeed.

We’ve asked what people think about Mozilla. Why they have chosen to become involved. Whether they think being a non-profit is central to what we do. Whether fundraising should be a part of that. And, if yes, how we can make it more effective. (Get involved and be heard.)

The Story So Far
The discussions have been amazing. We’re learning a lot about Mozilla and why our community and staff devote their time to the cause. And we’re also seeing themes emerge around how people view Mozilla, our mission, and our fundraising:

Being a non-profit matters. It’s the foundation of our brand, what differentiates our products in the market, and the source of a lot of pride. Not having external shareholders is empowering. But people resist being identified as a charity. The sense is that ‘charity’ carries connotations of need and behaviours that don’t apply to Mozilla.

Tension between being ‘mission-driven’ and ‘product-driven’. There’s a healthy, though challenging, day-to-day tension between the ‘why’ of our work, the mission, and the ‘what’, shipping products. That tension is surfacing more as we enter new markets with different operating cultures. And in terms of fundraising, the tension impacts decisions like whether to direct site traffic to fundraising or product marketing campaigns.

Fundraising has been a black box. Very little is understood about how or why we raise money. Most of the participants in the study wanted to learn more: to have a chance to engage and help shape our fundraising. So we, as the development team, need to get better at sharing our work. (This post you’re reading and this post and these posts and this dashboard are our first attempts to fix this.)

Donating is considered a more accessible contribution path than coding. Mozilla’s competitive advantage is and has always been its community. We don’t have the financial strength or employee base to go head-to-head with our competitors. When we win, it’s because of the people who contribute to our work. As our mission gains importance – as more things move to the Web – we will need to attract contributors from outside our usual channels. People who don’t code, but have other sources of expertise that can advance the mission (see the groundswell of educators gathering around Webmaker). Donating is seen as an easy yet meaningful way to become a Mozillian.

Online fundraising isn’t the only way to raise money. To date, small dollar fundraising and online fundraising have been synonymous. But as we scale Webmaker events around the world, as we get more people into more rooms, and as ReMo continues to grow and kick ass, we increasingly have opportunities to raise money in-person. We need to look away from the Lost Ark of online fundraising to new ways to engage directly with donors.

Donors want to know where their money goes. In line with trends across the non-profit sector, people want to understand the specific impact of their donations. Crowdfunding, social media, and micro-lending platforms have led donors to expect a direct relationship with the recipients of their support. We can do more to draw the line between a $30 donation and a scientist, teacher, or teenager learning how to express themselves on the Web. (We also need to get our act together around an annual donor report, most likely as a fork of the yearly State of Mozilla.)

We should be good at being a non-profit. One of the most interesting themes was the sense that if we’re going to be a non-profit, we should be good at it. That we should leverage all the advantages – volunteers, movement building, partnerships, activism, fundraising, etc. – that come along with it. Not at the expense of our strength – building meaningful products – but as a way of pursuing our mission and expressing our brand.

The study is also surfacing a healthy amount of skepticism:

“Why do we need my $30 when we have all the Google revenue?”
“I delete all those e-mails. I don’t think they’re effective.”

Confusion:

“Doesn’t everyone get paid from the Google deal?”
“Why do we even have a Foundation?”

And resistance:

“We shouldn’t be asking people for money. It’s annoying and unnecessary.”
“We need people closing bugs, not giving money.”

Perhaps the best finding so far is that many of people who work for Mozilla also donate to the mission. This means that the people who experience Mozilla on a daily basis continue to believe in the organization and its work. As a fundraiser, I know that one of the first things major donors ask is what percentage of employees also donate to the mission. So we’re in good shape.

What Next?
We want to talk to more people. But as we can’t take everyone to lunch (sadly!), we’ve put together a survey. Please take 5 minutes and let us know what you think about Mozilla, our mission, and our fundraising. This is your chance to help us rock. We hope you’ll participate.

What Will Happen From There
In early June, we will:

Post the final results and analysis of the study;
Use the results to shape a new fundraising strategy; and
Work to engage every Mozillian – including you – in what we hope will be our most effective end-of-year campaign ever.

So far we’re thrilled at the number of people who actually want to step up and help Mozilla raise money. The challenge is on us, as a team, to make sure you can and to do so with pride.

Filed under: Mozilla, Pitch Geek