Benjamin KerensaUnder the hood: GeeksPhone Keon

Here is what the internals of the GeeksPhone Keon powering Firefox OS looks like:

keon inside 300x207 Under the hood: GeeksPhone Keon

Photo by FCC

keon2 300x245 Under the hood: GeeksPhone Keon

Photo by FCC

Benjamin KerensaFirefox OS Keon Unboxing

Here are some photos of my unboxing of the GeeksPhone Keon running Firefox OS that I got today:

IMGP0605 300x168 Firefox OS Keon Unboxing

One thing I immediately liked was the packaging for the Keon is pretty darn unique.

IMGP0602 300x168 Firefox OS Keon Unboxing

     I like the images on the side of the box which represent some of the things it can do.

IMGP0607 300x168 Firefox OS Keon Unboxing

 Each phone comes with a simple booklet and a Firefox OS sticker

IMGP0617 300x168 Firefox OS Keon Unboxing

The inside of the box is neat and lets you use it to keep your phone organized and also holds a pair of earbuds, european USB charger and a regular USB cable.

IMGP0612 300x168 Firefox OS Keon Unboxing

The vinyl screen protector it shipped with was a bit messy but it didn’t matter much to me.

IMGP0615 300x168 Firefox OS Keon Unboxing

The back cover is a vibrant looking orange and feels rubberized

IMGP0622 168x300 Firefox OS Keon UnboxingThe phone booting up

IMGP0624 168x300 Firefox OS Keon UnboxingPick a language… So many available!

IMGP0630 168x300 Firefox OS Keon UnboxingSet your time and date

IMGP0628 168x300 Firefox OS Keon UnboxingConnect to wifi!

IMGP0634 168x300 Firefox OS Keon UnboxingImport contacts from SIM, Facebook and more (perhaps coming soon!)

IMGP0635 168x300 Firefox OS Keon UnboxingMozilla values improving products so I picked to send anonymous data

IMGP0638 168x300 Firefox OS Keon UnboxingTaking a tour

IMGP0642 168x300 Firefox OS Keon UnboxingMoving icons

IMGP0641 168x300 Firefox OS Keon UnboxingSwiping for the win

IMGP0643 168x300 Firefox OS Keon UnboxingSwipe down for notifications

IMGP0644 168x300 Firefox OS Keon UnboxingHome button

IMGP0646 168x300 Firefox OS Keon UnboxingLock Screen

IMGP0652 168x300 Firefox OS Keon UnboxingAll the things we love

IMGP0649 168x300 Firefox OS Keon UnboxingHome screen area

IMGP0657 168x300 Firefox OS Keon UnboxingAnd finally a quick peak at the settings

For those who have not been using Firefox OS for while that sums up much of the platform. There really is so much more including the Marketplace and speed that make this open mobile platform the wave of the future.

I think were moving towards a time where mobile platforms like Firefox OS, Ubuntu Touch and maybe even Tizen can compete with platforms like Android and iOS. I think Mozilla is doing a great job and has a super strategy for evangelizing the platform and getting partners and developers on board.

Jessilyn Davis“Firefox & You” gets a mobile-friendly redesign

Our monthly newsletter, Firefox & You, has seen quite a few changes since its inception in October of 2010.

October 2010

October2010

April 2011

April2011

April 2013

April2013

May 2013

And this month, it got another face-lift and its code became mobile friendly (click on the image to see the full email):

Desktop

May2013_desktop

Mobile

May2013_mobile2

Thank you Team!

After countless hours of design, coding, testing across 30 different email clients, browsers and mobile devices, localizing, re-testing, and troubleshooting a persnickety Android Tablet rendering issue (that made me and nigelb feel a little like this… who knew Android tablets would be so picky about any extra space in a style attribute?) — the May newsletter has finally launched to all 7 languages. (And Polish is on deck to launch next month!)

I think I can finally go into this 3-day holiday weekend knowing a great feat has been accomplished… just in time to start the June newsletter next week :)

PS – Want to know what a checkbox list looks like for putting together a new newsletter in 7 different languages? Take a look…


Chris IliasGoogle Reader alternatives for Firefox

If you use Firefox and need an alternative to Google reader, Feedly isn’t the only option.

First, you can try the live bookmarks feature that’s part of Firefox. Here are instructions on how to use it.

Second, there are plenty of extensions available for reading web feeds. I used to use Sage. Other popular ones:

Third, if you use Mozilla Thunderbird, the email and newsgroups app built on Mozilla, it contains a great rss reader.  For instructions on how to use it visit Thunderbird support – How to Subscribe to News Feeds and Blogs.

Joey ArmstrongSubject: Transition to mozbuild: ASFILES – Assembly language

Subject: Transition to mozbuild: ASFILES – Assembly language source files   Part #1 of the mozbuild conversion for bug 869135 has landed: http://bugzilla.mozilla.org/show_bug.cgi?id=869135   This patch has added ability to define lists of assembly language files within mozbuild configuration.  Files … Continue reading

Selena DeckelmannThe People of Postgres: Tom Lane

This post was originally posted on Medium, a new blogging platform made up mostly of people who aren’t necessarily subscribed to Planet. So, please forgive the obvious statements, as the target audience are people who don’t know very much about Postgres. Tom Lane, taken by Oleg Bartunov

Wednesday May 23, with no fanfare, Tom Lane’s move to Salesforce.com was made public on the Postgres developer wiki.

For 15 years, Tom has contributed code to Postgres, an advanced open source relational database that started development around the same time as MySQL but has lagged behind it in adoption amongst web developers. Tom’s move is part of a significant pattern of investment by large corporations in the future of Postgres.

For the past few years, Postgres development has accelerated. Built with developer addons in mind, things like PLV8 and an extensible replication system have held the interest of companies like NTT and captured the imagination of Heroku.

Tom has acted as a tireless sentry for this community. His role for many years, in addition to hacking on the most important core bits, was to defend quality and a “policy of least surprise” when implementing new features.

Development for this community is done primarily on a mailing list. Tom responds to so many contributor discussions that he’s been the top overall poster on those mailing lists since 2000, with over 85k messages.

Really, he’s a cultural touchstone for a community of developers that loves beautiful, correct code.

Someone asked: “What does [Tom’s move] mean for Postgres?”

You probably don’t remember this:

Salesforce.com bases its entire cloud on Oracle database,” Ellison said, “but its database platform offering is PostgreSQL. I find that interesting.

When I read that last October, I was filled with glee, quickly followed by terror. I love my small database community, my friends and my job. What if Oracle shifted its attention to our community and attacked it, directly? So far, that hasn’t happened.

Instead, Salesforce advertised they were hiring “5 new engineers…and 40 to 50 more people next year” for a “huge PostgreSQL project.

Tom’s move probably won’t change much for the day-to-day operation of Postgres itself. Hopefully, things are about to get real at Salesforce.

I’m a major contributor to Postgres. I started in 2006, learning about relational databases through work at a small bike parts manufacturer and ERP. My contributions include code, starting conferences, encouraging user group leaders and introducing Postgres to communities that otherwise would never hear from us. I’m a data architect at Mozilla.

David Rajchenbach TellerProject Async & Responsive, issue 2

Our heroes continued on their quest to move add-on- and front-end-accessible APIs off the main thread. Arrayed against them, work weeks, conferences, holidays and terrible weather. Still, they pressed on.

Tooling and new APIs

  • JavaScript module loader for workers (bug 872421). Working prototype.
  • Constant stack implementation of Promise (bug 810490). Working prototype, improving gradually.
  • Async API for Downloads (bug 825588). In progress. Numerous blockers.

Refactorings

  • Making Session Restore non-blocking (bug 838577). Working prototype, improving gradually.
  • Numerous simplifications to Session Restore (bug 874381 and blockers). Many patches are ready to land, once we have decided of a good policy to revert them in case they destroy too many add-ons.
  • Async mozStorage connection (bug 702559). Working prototype. Pending review.

Testing

  • Adding support for Task.jsm in mochitest (bug 872229). In progress.

Matěj CeplFirefox OS phone … first observations

Yes, even I’ve got my Geeksphone Peak, and yes I like it. I am sure, that I have collectible item in my hands … it will be either that historical model which changed the world to the open standards on mobile, or maybe it will be in one of those “10 biggest flops” articles (yes, I would love to have Lisa!). I am not sure about this mass success, but I feel like I may eventually like it for myself (which is actually all I care for; after all, I am a happy user of Linux). The first news is very good news: it works perfectly as a phone, which means I use it as my day to day phone (well, music player doesn't switch off on picking up the phone). Also, camera Just Works™, it is possible even lightly edit the image on the phone (a feat which the standard Android camera was not able to accomplish, BTW). Now the problematic parts.

Calendar just doesn't work for me. It works with CalDAV on Firefox OS Simulator 3.0, but not with my phone. Sad but expected (I don’t know why Calendars get so bad treatment by most software providers, not mentioning missing Tasks). Also, originally, Mail didn’t work for me, but I was able to get it working more or less with some later images from Geeksphone (still from time to time the app doesn't load the body of the mail ever). Which leads to another problem: there are no OTA updates yet. I am willing to use Debian/unstable or Fedora/Rawhide, but I really need to be sure, that all those bugs which I hit will be soon fixed and delivered to me. Getting developing-quality with no updates, is really a bad experience for the user, I would say. In the end I at least found the latest images from Geeksphone, but that still has only 1.0.1 version, although there are rumours of 1.1 already existing on the Mozilla servers. Other bad experience is that fonts are tiny, almost unreadable, and I haven’t a way (even a one which would include adb or something similarly drastic) to work around it.

The worst and the best part of the phone is the situation with Contacts. Out of the box, FF OS imports just from SIM card (who still uses contacts there?) and from Facebook (I don’t have an account, and even if I had I would never ever store my main address book there). That was pretty bad, as I really didn't look forward typing of couple hundred contacts I had on pretty crappy keyboard (with unreadably small characters). So, my programmer’s pride raised its head, and I have wrote Javascript LDIF parser and with that I was able to write a simple (and ugly) single-webapp for importing a LDIF file to the system address book. While doing that I have found that there is already Importer from GMail Contacts. I don’t use GMail, so it wasn’t useful for me as such, but eventually I hope to merge my code into this app, because it has much better user interface.

Generally I am enjoying the ride so far, and I am looking to the time where I could get my fix of nightlies everyday.

Al BillingsWeird Records from 3D Printers and Laser Cutters

Warning: This post is rather heavy with the use of Flash to show videos. Sorry!

I've seen a bit of weirdness involving records, of all things, lately. Records, specifically Vinyl Records, are a pretty obsolete technology at this point. Yes, I'm well aware of the undying love for the "superior" sound quality of vinyl from a certain subset of bibliophiles. These people adore their record collections until the end of time. The rest of us, like my wife, have long given up on vinyl and moved, first, to compact discts (if not tapes) and then to mp3 files and our iPod-like devices. This moves record technology for most people, especially the young, into the same realm as wind up gramaphones or rotary telephones and HAM radio: old technology loved by a view but looked upon oddly by the rest.

Now, in spite of this (or maybe because of this), people are doing some weird things with record technology right now as artists. I wanted to share a few instances that I have come across, two from the same person. I came upon all of these within the last two weeks so they kind of stuck in my mind.

First, we have ice records. Cue video:

A band, the Shout Out Louds, sent out ten kits to people that have a silicone negative image of a single from their recent (unreleased at the time, I believe) album. This came with instructions to pour enclosed distilled water on it, freeze it for six hours, and pop and ice record from the mold to then play. You can view the results above.

Next is Amanda Ghassaei's attempts to play with record technology as an artist. First, she used some high end printers to 3D print a record. She details this (at great length) on a post on instructables on this. She figured out a method to convert an mp3 music file into a grooved shape thaat would play on a traditional record player. This was then printed on a high end printer, though I suspect that a well tuned hobbyist printer might work well enough. She mentions that it has a sampling rate of only 11kHz and 5-6bit resolution, which makes it sound a bit like an ancient Edison era recording. The video below gives an example of the result:

Amanda has more recently gone on to try the same trick using a laser cutter, resulting in a wood record etched by the laser. She wrote another instructable describing this process. You can also watch her video about it below to hear the wood one:

She's actually done this with acrylic using the laser as well and I think that it sounds a bit better than the wood one:

All of that said, the audio on all of these is horrible. I actually think the ice record sounds the best. What I've found the most interesting about this is the intersection of an obsolete and very analog technology, the record player and the records, with interesting art and even cutting edge tools like a laser cutter or a 3D printer. Technically, the ice record could have been made 50 years ago (I don't think making a silcone negative is likely a difficult technical challenge) but this marriage of art and technology is fascinating.

People often ask me, "What do you make with a 3D printer?" The joke, of course, is the answer, "Parts for more 3D printers" (thought it is often true). That said, the ability to quickly prototype, especially to prototype from an artistic vision created within a computer, is really cool for art. Amanda wrote a script in Processing, an open source language used by artists a lot and often used with arduinos, to do the conversions for her to turn the music files into a three dimensional construct that she could then print or cut. What other art do people have locked inside of them that they can use these tools to express?

Nigel BabuCIS Anniversary and Encyclopedia of Indian Cinema

Centre for Internet and Society celebrated their 5-year anniversary with an exhibition at their Bangalore and Delhi offices and a series of talks in Bangalore. I was there on Tuesday and managed to spend some time at the exhibition and attend the talks.

The exhibition showed off some of the work that CIS has been doing and the work of several independent artists. The bits that are particularly in my memory is Tara Kelton’s work as well as Sharath’s work.

Later in the day, Lawrence Liang talked about the Encyclopedia of Indian cinema. It was a very interesting talk, especially for me since it encompasses open data, open source software, and copyright issues! A convergence of a lot of my interests :) Lawrence talked about what they’ve built and the problems they’ve faced and how internet as a medium for a film encyclopedia is very powerful, but is limited by the legal issues surrounding copyright laws.

Indian Cinema

Indian Cinema Wiki

On that note, I’ll close with this video about copyright.

 

I know Disney is great, but I’m not sure I like them as much after this video.

Doug BelshawExplaining Open Badges through analogy

One of the best ways to help people understand something they’ve not come across before is through metaphor and analogy.

A year or so ago, for example, my son had a cold and said “my nose is deaf” – and I knew exactly what he meant. It contained just the right balance of ambiguity.* When explaining Open Badges to people I’ve found “X is kind of like Y because of Z” helpful in getting them to grasp what I mean. The more useful metaphors, similes and analogies I can find, therefore, the better.

Below are some I’ve used recently to explain Open Badges. They may or may not help you or the people you’re talking to about badges. But give them a read and tell me what you think. Oh, and the animated GIFs are just for fun! ;-)

The difference between ‘a badge’ and ‘a badge system’

Mozilla is developing the OBI – the Open Badges Infrastructure. People are free to use it to create their own badges for whatever purpose they like.

Anchorman-flute

It’s a bit like a water company providing the infrastructure so that instead of having to go to a well, you can get water coming out of a tap. What you use that water for, what you mix it with, and how you share it is entirely up to you.

A different analogy might be that a badge is akin to an ‘app’ in an app store. Mozilla may produce some badges of its own, but it’s looking after the entire ‘app store’ in terms of the OBI. This metaphor breaks down for two reasons, however: there’s no one place to see all of the badges (at present) and it’s not a walled garden as many app stores are. Anyone can use badges for any purpose without reference to Mozilla. It’s an open, decentralised system and standard.

Metadata in badges

Metadata is data about data. It’s like when you tag someone in a photo on Facebook – you’re adding data about the data already in the system. In this case the data is the photograph and the metadata you’re adding is the name of an individual in the photo. The index at the back of a book is metadata as well – data about the data in the book.

Sunglasses on

One way to think about Open Badges is that they’re a bit like barcodes that can be understood by humans. Just as when you scan a barcode you get extra data such as the price of a product, so when clicking a badge you get details of what the earner had to do to get the badge, the evidence for it, etc.

The metadata is hard-coded into the Open Badge. So, just like when you make a cake, it’s made up of lots of different ingredients (the name of the badge, the identity of the badge earner, the Criteria URL, etc.). Once you’ve baked the cake or the badge, you can’t change those ingredients or get them out. That badge is unique to the individual. If you’ve baked a chocolate cake and now you want a Victoria sponge, then you’re going to have to bake another cake. Similarly, you can’t change a badge once it’s been issued.**

Badge pathways

In life, some pathways and routes definitely lead somewhere. That could be a route into employment, a journey to a holiday destination, or some other ‘place’ that you want to get to. There are almost-guaranteed ways to get to that destination, such as going to a travel agent and getting them to take care of your flight, transfers and accommodation. Likewise, completing a recognised project management qualification greatly increases the chances of being employed as a project manager.

Cat high five

There are other ways of getting to your holiday destination and becoming a project manager, however. You could book all of the different parts of the trip yourself. You could hitch-hike. You could use websites like Couchsurfing or Airbnb. Likewise, with the project manager position you could have learned how to manage projects on the job and have lots of experience of delivering successfully. Or, indeed, you may have transferable skills.

But there are some people for whom the journey is the destination. They don’t have a particular path in mind – or, perhaps, they’re blazing a new trail unsure of where it will lead. Being able to capture the knowledge, experience and skills they gain along the way would seem to be a useful thing to do. It surfaces the slightly meandering journey that I think we’ve all experienced during our careers. Badges can help validate these non-linear pathways.

Badge quality

Think of the last time you stayed in a hotel. Unless that was booked on your behalf, how did you end up staying where you did? Some of it may have been down to money, but what other factors were involved? There would have been the 5-star rating system which, until recently, would have been one of the only ways to ascertain the quality of a hotel. But is that the only thing you used? I bet, nine times out of ten, it was either TripAdvisor or some other social ratings/recommendation system.

Wow - men in suits drinking tea

The value of an Open Badge comes from at least from three different places. First, there’s the reputation of the individual or organisation that issues (or, in future, endorses) the badge. Second, there’s the (essential) Criteria URL in the badge that tells the consumer what the earner of the badge had to do to get it. And, finally, there’s the (optional) Evidence URL that shows just what the earner did with that criteria. It’s a triangulation very much akin to deciding which hotel to stay at: the star rating, a description of the facilities/amenities, and reviews from sites like TripAdvisor.

The point here is that top-down ‘quality’ systems can work, but they’re even more powerful (and can sometimes be replaced) by horizontal, peer-to-peer recommendation engines. It’s the difference between how a system should work and how it actually works.

Badge equivalency

Deciding that one thing is equivalent to another is not something that Mozilla is (at the moment, at least) concerned with.

Baby losing it

I think of badge equivalency as being a bit like mobile phone tariffs. There’s many different plans and tariffs that it’s possible to use/sign up for as a mobile phone user. Most of them offer fairly similar combinations of talktime minutes, SMS messages and 3G data. Some, however, may offer 4G data. Even if there are differences between providers, it’s still possible to weigh up what’s best for you. What you decide to be ‘equivalent’ might not be the same as what someone else believes to be so. It depends upon context.

There will potentially be many different providers of similar badges. The value of the badge will be ascertained by employers and other people providing opportunities by comparing those badges against the others available. Credentials are always used for a purpose, after all. Eventually, some kind of ‘BadgeRank’ algorithm (similar to Google’s ‘PageRank’) may help both earners and employers find the most relevant badges in their industry.

Badge backpacks

Badges are hosted in a badge backpack and then displayed across the Web. It’s similar to videos being stored on YouTube or Vimeo and being embedded on many different websites. Likewise, you can make them private or public. The difference here is that, once we’ve got federated badge backpacks, you will get to choose where your ‘videos’ (badges) are hosted as well as where they’re embedded.

Unwrap - BOOM

Some of these work better than others. I’d very much appreciate feedback as well as any analogies you’ve used successfully! :-)


* More about different types of ambiguity in this paper that I (co-)wrote.

** So, technically, we are thinking very carefully about badge revocation but we don’t want people reaching into people’s badge backpacks willy-nilly and changing them. There possibly will be a ‘nuclear’ option for revocation, however – such as when you’ve accidentally awarded a PhD-level badge to a six year-old…

Image CC BY-NC-SA howard.hall

Daniel GlazmanNew customer

I am extremely pleased to welcome the European Parliament as a major user of BlueGriffon.

David Rajchenbach TellerAdd-on breakage, continued: List of add-ons that will probably be affected

Ongoing chances to Session Restore will probably break the following add-ons:

  • MSN For Firefox
  • Bing For Firefox
  • Twitter
  • Tab Mix Plus
  • IE Tab
  • Undo Closed Tabs Button
  • SessionPage
  • Tab Scope
  • Taboo
  • Tree Style Tab
  • New Tab JumpStart
  • TabGroups Manager
  • Browse Periodically
  • Tab Utilities
  • Tile Tabs
  • IE Tab 2
  • Vertical Tabs
  • Master Password+
  • Home Dash
  • Extension Test
  • Progress Bar on Tab
  • Fox Splitter
  • Dormancy
  • Tabkit 2nd Edition
  • UnloadTab
  • Tabby2
  • Tab Restore
  • Side Tabs
  • Home Dash Lite-Dial
  • Fire IE
  • Tab Utilities Lite CustomEdition
  • Tabulous
  • Bar Tab Lite X
  • PageExpand
  • IE Tab 2 (SeaMonkey)
  • All Tabs Helper
  • multiplaceHolder

All of these add-ons tap directly into undocumented, internal data structures that will disappear in the near future (expected: Firefox 25). If you are the author or maintainer of one of these add-ons, you should monitor carefully bug 874381 and its blockers for all technical details.

If you need technical help or if you need us to provide alternative public APIs, please get in touch as soon as possible.

Edit I should have given more details about what exactly will break. We are progressively getting rid of all fields whose name starts with __SS. These add-ons are the add-ons that depend on such fields.


Frederic Wenzelimagetwist, an add-on rotating images according to their EXIF data

Ever so often, friends send me emails with pictures they took on their phones. When looking at the photo, it frequently appears turned on its side, like this:

Mona Lisa

A little research revealed that cameras take both sideways ("landscape") and upright ("portrait") photos the same way. Then, instead of arranging the pixels differently (an expensive process), many cameras, especially phones, just add a little bit of EXIF metadata defining the way the device was oriented when the picture was taken.

It is then up to the image viewing software to read this metadata and display the image correctly.

An old bug

Some image viewers respect this and show the image right-side-up. Firefox, it turns out, does not.

The bug report, almost 8 years old, lists many good reasons why you wouldn't want to auto-rotate images inside web pages based on their metadata. For example, some image editors will turn your image, but not clear the orientation metadata, so the browser would overcompensate if it applied additional rotation. Also, silently changing a major part of the web like image rendering is likely to break many existing sites that have come to rely on the previous behavior.

On single-image pages ("view image"), however, I and many people I talked to believe the image metadata should be honored. Not doing so, for fear of the off chance the image has broken metadata, is breaking the 99% case (photos from digital cameras) in favor of the 1% false positive (broken image editing software).

Time for a fix

So I went ahead and wrote a Firefox add-on that fixes this issue. Meet imagetwist. It adds two features to your browser.

First, it fixes the problem described above: If you look at an individual image, and the metadata says it should be turned, the add-on will turn it automatically.

Second, it provides a right-click menu on any image, anywhere, allowing you to rotate it as needed (to the right, left, or 180 degrees). This is useful for the occasional case where a wrongly oriented image is visible on a website. For this feature, image metadata is not considered, you can rotate images any way you want.

The imagetwist context menu

How does it work?

To turn an image, I spin up a canvas element and project the image on it, rotated. Then I export the canvas data and replace the image source with it.

Granted, I would have preferred a pure CSS solution. I explored CSS transforms, but that will only rotate the image itself, not its bounding box. So, frequently, the image will rotate underneath some other content on a page and become partially obscured.

A better CSS option would be the image-orientation property, but alas, it's not implemented quite yet. When it lands, I will happily adapt imagetwist's code accordingly.

Try it out! Contribute!

I know that you can't wait to try it out: Install imagetwist from the Mozilla Add-ons website.

As usual, this add-on is open source software. Check out the source code on github. Feedback and other contributions welcome!


I'm blogging about once a week in 2013, on various topics. This is my eleventh post of the year.

Byron Joneshappy bmo push day!

the following changes have been pushed to bugzilla.mozilla.org:

  • [873153] Add in-moztrap, tracking-fennec to Android Background Services product
  • [870202] investigate ways of improving the performance of bugmail composition

Filed under: bmo, mozilla

Michael KaplyBrand Thunder and WebSearch+

Somehow in the past few weeks, lots of sites have started repeating the same article and the same incorrect information that WebSearch+ from Brand Thunder is malware. This is patently false. WebSearch+ is simply the method Brand Thunder uses to monetize its add-ons and it is 100% configurable by the user. Every feature can simply be turned off in preferences.

Brand Thunder put up a post with more information - What is WebSearch+

Hopefully that will clear things up.

Zack WeinbergRadical simplification will not save the world

Seems like every time I go to a security conference these days there’s at least one short talk where people are proposing to start over and rebuild the computer universe from scratch and make it simple and impossible to use wrong this time and it will be so awesome. Readers, it’s not going to work. And it’s not just a case of nobody’s going to put in enough time and effort to make it work. The idea is doomed from eight o’clock, Day One.

We all know from practical experience that a software module that’s too complicated is likely to harbor internal bugs and is also likely to induce bugs in the code that uses it. But we should also know from practice that a software module that’s too simple may work perfectly itself but will also induce bugs in the code that uses it! “One size fits all” APIs are almost always too inflexible, and so accumulate a “scar tissue” of workarounds, which are liable to be buggy. Is this an accident of our human fallibility? No, it is an inevitable consequence of oversimplification.

To explain why this is so, I need to talk a little about cybernetics. In casual usage, this word is a sloppy synonym for robotics and robotic enhancements to biological life (cyborgs), but as a scientific discipline it is the study of dynamic control systems that interact with their environment, ranging in scale from a simple closed-loop feedback controller to entire societies.1 The Wikipedia article is decent, and if you want more detail, the essay “Cybernetics of Society” is a good starting point. Much of the literature on cybernetics talks about interacting systems of people—firms, governments, social clubs, families, etc—but is equally applicable to systems of, around, or within computers. One of the fundamental conclusions of cybernetics, evident for instance in Stafford Beer’s viable system model, is that a working system must be as least as complex as the systems it interacts with. If it isn’t, it will be unable to cope with all possible inputs. This is a theoretical explanation for the practical observation above, and it lets us put a lower bound on the complexity of a real-world computer system.

Let’s just look at one external phenomenon nearly every computer has to handle: time. Time seems like it ought to be an easy problem. Everyone on Earth could, in principle, agree on what time it is right now. Making a good clock requires precision engineering, but the hardware people have that covered; a modern $5 wristwatch could have earned you twenty thousand pounds in 1714. And yet the task of converting a count of seconds to a human-readable date and vice versa is so hairy that people write 500-page books about that alone, and the IANA has to maintain a database of time zones that has seen at least nine updates a year every year since 2006. And that’s just one of the things computers have to do with time. And handling time correctly can, in fact, be security-critical.

I could assemble a demonstration like this for many other phenomena whose characteristics are set by the non-computerized world: space, electromagnetic waves, human perceptual and motor abilities, written language, mathematics, etc. etc. (I leave the biggest hairball of all—the global information network—out, because it’s at least nominally in-scope for these radical simplification projects.) Computers have to cope with all of these things in at least some circumstances, and they all interact with each other in at least some circumstances, so the aggregate complexity is even higher than if you consider each one in isolation. And we’re only considering here things that a general-purpose computer has to be able to handle before we can start thinking about what we want to use it for; that’ll bring in all the complexity of the problem domain.

To be clear, I do think that starting over from scratch and taking into account everything we’ve learned about programming language, OS, and network protocol design since 1970 would produce something better than what we have now. But what we got at the end of that effort would not be notably simpler than what we have now, and although it might be harder to write insecure (or just buggy) application code on top of it, it would not be impossible. Furthermore, a design and development process that does not understand and accept this will not produce an improvement over the status quo.

1 The casual-use meaning of “cybernetics” comes from the observation (by early AI researchers) that robots and robotic prostheses were necessarily cybernetic systems, i.e. dynamic control systems that interacted with their environment.

Nagappan[Ann]: Cobra 3.5 - Windows GUI test automation tool


New features:
* ooldtp python client
* Support setting text on combo box
* Added simple command line options
* Support state.editable in hasstate
* Handle valuepattern in click API
* Support ToolBar type on click
* Write to log file if environment variable is set (set LDTP_LOG_FILE=c:\ldtp.log)
* Support control type Table, DataItem in Tree implementation
* Added scrollbar as supported type

New API:
* MouseMove
* setcellvalue
* guitimeout
* oneup
* onedown
* oneleft
* oneright
* scrollup
* scrolldown
* scrollright
* scrollleft

Bugs fixed:
* Fix to support taskbar with consistent index
* istextstateenabled API
* Fallback to object state enabled if value pattern is not available
* Fix to support InvokePattern on Open button
* Use width, height if provided while capturing screenshot
* Work around for copying text to clip board
* QT 5.0.2 specific changes
* Check errno attribute to support cygwin environment
* Fix keyboard APIs with new supported key controls (+, -, :, ;, ~, `, arrow up, down, right, left)
* Don't grab focus if type is tab item

Java client:
* Fixed selectRow arguments
* Fixed compilation issues
Python client:
* Fix optional argument issue in doesrowexist
C# client:
* Added new APIs (scrollup, scrolldown, scrollleft, scrollright, oneup, onedown, oneleft, oneright)
Ruby/Perl client: No changes

Credit:

Nagappan Alagappan, John Yingjun Li, Helen Wu, Eyas Kopty, VMware colleagues

Please spread the word and also share your feedback with us (email me).

About LDTP:

Cross Platform GUI Automation tool Linux version is LDTP, Windows version is Cobra and Mac version is PyATOM.

* Linux version is known to work on GNOME / KDE (QT >= 4.8) / Java Swing / LibreOffice / Mozilla application on all major Linux distribution.
* Windows version is known to work on application written in .NET / C++ / Java / QT on Windows XP SP3 / Windows 7 / Windows 8 development version.
* Mac version is currently under development and verified only on OS X Lion. Where ever PyATOM runs, LDTP should work on it.

Download source / binary (Windows XP / Vista / 7 / 8)
System requirement: .NET 3.5, refer README.txt after installation

Documentation references: For detailed information on LDTP framework and latest updates visit http://ldtp.freedesktop.org

LDTP API doc / Java doc
Report bugs

Frédéric BuclinBugzilla 4.4 and 4.2.6 released

We released Bugzilla 4.4 and 4.2.6 a few minutes ago. Bugzilla 4.2.6 is a bugfix release, and contains no security fixes. It adds support for MySQL 5.6 and fixes a crash with Oracle.

Bugzilla 4.4 contains many new features:

  • The searching system has been improved: it allows multiple search criteria to match one field; it has improved support for flags; it has better performance, especially with complex queries.
  • The tagging system has been redesigned: the old tagging fields in the page footer are gone in favor of a Tags field in the bug itself. Tags remain private and are only visible by you. They can also be used in queries as any other field.
  • Bugzilla can now correctly identify uploaded attachments which have a MIME type of application/octet-stream.
  • You can now save tabular and graphical reports as you already do with saved searches. Previously, you had to bookmark them in your web browser.
  • You can customize columns displayed in whinemails.
  • As usual, the WebServices have been improved with several new methods.
  • The security has been improved, such as using HMAC SHA-256 to generate tokens instead of MD5.
  • etc ….

This release also means the End Of Life (EOL) of Bugzilla 3.6, which was the last series for Bugzilla 3.x. This branch is no longer supported, and any new security bug found on this branch will remain unfixed. Installations still running Bugzilla 2.x or 3.x are urged to upgrade to Bugzilla 4.4 or 4.2.6.

So, what’s next? Well, the main focus for Bugzilla 5.0 will be usability and user experience. Bugzilla sometimes (often?) looks old-fashioned because we wanted to support old browsers and browsers with JavaScript disabled. But we decided to move to a more interactive interface where JavaScript will help accomplish some tasks with less page reloads, such as submitting changes to a bug or adding an attachment. We also plan to add a new skin to Bugzilla which should look like this. Some pages will also be entirely redesigned, such as the Group Controls page to administrate access to bugs which was considered too complex for the average admin. More information will come with the release of development snapshots.


Honza BambasC/2011 L4 (PanSTARRS)

C/2011 L4 (PanSTARRS)

C/2011 L4 PanSTARRS v blízkosti hvězdy Errai (ɣ Cep) v souhvězdí Cepheus.

16.5.2013 1:48 – 2:48  SELČ
Jižně od Prahy
Canon 60d + CLS CCD Clip
Canon 200mm/f2.8 L II USM
HEQ5, bez pointace
8x5min 800 ISO @ f3.5
Flat 8x, Bias 5x (bohužel focené dodatečně doma, zapomněl sem si flat field..)

Později přišli mraky, takže fotka je trochu rozmazaná.

DeepSkyStacker 3.3.2, stack na kometu dle tohoto návodu.  I přes to ale nejsou dráhy hvězd vidět.  Při bližším pohledu jsou jen znatelné duchy.  Doladění PhotoShop (32bit).

C2011-L4-PanSTARRS-animation

Brandon SavageCode complexity and clean code

Many people wonder how they can improve the overall readability and cleanliness of their code. It seems impossible to understand exactly how to rewrite code in such a way that makes it clean, easy to understand and simple to work with. But clean code is about a few simple principles, one of which is reducing [...]

Planet Mozilla InternsNick Fitzgerald: "Compiling To Javascript, And Debugging With Source Maps" On Mozilla Hacks

I wrote a tutorial titled Compiling to JavaScript, and Debugging with Source Maps for the Mozilla Hacks blog:

This is a tutorial on how to write a compiler which generates JavaScript as its target language, and maintains line and column meta-data in source maps for debugging. Storing line and column coordinates in a source map allows the end-user of the compiler to debug the source code that they wrote, rather than the ugly, generated JavaScript they are not familiar with.

Go read it!

Ben HearsumBuildbot Scheduler and Builder graphing

One of the the most important systems I work is the release automation for Firefox and Thunderbird. The process behind the automation long predates me, but I’ve been deeply involved in automating, refining, and optimizing it. It shouldn’t come as any surprise that one of the biggest challenges of working on such a complex system is understanding how the smaller pieces fit together to make the whole system. For the release automation we have an advantage though: the smaller pieces are generally Buildbot Builders, and the things that fit them together are generally Buildbot Schedulers. Awhile ago I was improving parallelism for l10n repacks and found it extremely difficult to reason about whether or not my changes would actually create the desired Builders and string them together correctly. I threw together some (terrible) code that spat out a digraph of the release automation’s Builders and Schedulers. By comparing the before and after graphs I was able to iterate on some parts of my code without spending hours and hours testing.

This week I finally got around to tidying up and packaging this code as a more general purpose tool. It’s not nearly complete and has many rough edges, but as a very basic tool to help you understand non-trivial Buildbot installations, I think it’s wonderful. It’s pip installable (“buildbot-scheduler-graph”) and available on Github. Once you’ve got it, try it out with “buildbot-scheduler-graph /path/to/your/master.cfg /path/to/output-dir”. Here’s what Mozilla’s scheduler graphs looks like. What do yours look like?

Alexander SurkovIAccessible2. New features and performance.

After months of long discussions the IAccessible2 group released 1.3 version. It was probably most disputable release after the initial shipping. A number of assistive technology vendors requested a number of changes for better integration with the browser. So it took a time to find a common multiplier for all ideas.

So updated IAccessible2 API require some engineering love and thus it will be implemented in Firefox by parts. It'd be helpful to know which features are most crucial and needed for you as AT developer so we can give them a priority. To let us know please leave a comment in the bug or just ping me saying that. Alternatively you can vote for or comment right into the interesting bug (see sections below).

Roles


IA2_ROLE_COMPLEMENTARY_CONTENT is supposed for HTML5 aside element and WAI ARIA complementray role (see Mozilla bug).

States


A new IA2_STATE_PINNED state will be exposed on pinned tabs in Firefox UI (see Mozilla bug).

Technically speaking IA2_STATE_CHECKABLE was introduced in place of hacks used to workaround a lack of this state. For example Firefox exposes checkable object attribute (IAccessible2 hack) and STATE_SYSTEM_MARQUEED (pure MSAA approach). See Mozilla bug.

Relations


There is a bunch of new relations introduced to simplify a navigation through the hierarchy. Now you can quickly move to

Note, IServiceProvider is an alternative to described relations (strictly speaking a subsest of described relations). See Mozilla bug.

Also now you can get a next and previous elements in tab order (IA2_RELATION_NEXT_TABBABLE and IA2_RELATION_PREVIOUS_TABBABLE). It's useful for AT that implement own in-page navigation (see Mozilla bug).

New IA2_RELATION_NODE_PARENT_OF is a dual relation to IA2_RELATION_NODE_CHILD_OF. It's useful to get all logical children of the accessible (as you know a logical parent-child relation may be different from parent-child relation defined by accessible tree). Example is aria-owns (like aria-owns tree) and flat structured hierarchies like XUL tree (used for message list in Thunderbird) or ARIA tree where the hierarchy is supplied by aria-level like:

  <ul role="tree">
    <li role="treeitem" aria-level="1">Item 1</li>
    <li role="treeitem" aria-level="2">Item 1.1</li>
  </ul>

See Mozilla bug.

 

IAccessibleDocument


IAccessibleDocument is a new interface that allows you to get an anchor target, i.e. an element that the page was scrolled to after the page was loaded. Also an object of this interface is returned by relations like IA2_RELATION_CONTAINING_DOCUMENT and IA2_RELATION_CONTAINING_WINDOW. See Mozilla bug.

 

Manage the media content


The idea was to manage the media content by accessible actions (IA2 group got parsimonious for a new interface so the existing mechanism was reused).

If a screen reader starts, say, a subtitled video then it invokes IA2_ACTION_OPEN action. The browsers plays a portion of subtitled video and stops it until a screen reader invokes IA2_ACTION_COMPLETE action. This action is used to notify the browser that screen reader consumed the portion of played video subtitles, for example, it finished reading of them. If screen reader is no longer interesting to manage the media playback then it invokes IA2_ACTION_CLOSE action.

If more than one AT tries to manage the same media (i.e. AT that invoked 'open' action) then the browser holds on the video playback until all AT invoke 'complete' action. See Mozilla bug.

Performance


After IAccessible2 implementation by Firefox and AT it got clear that IAccessible2 API can be improved in certain ways to make the intercommunication between the browser and AT faster. So number of changes was introduced for performance matter.

Object attribute by name


Now you can obtain object attributes by name rather than get them all together and then parse the string (see Mozilla bug).

Relation by type


Now you have a method to get relation targets by type. You don't need to get all relations when you need a single one (see Mozilla bug).

Accessible containing the caret


You have a fast way to get a text accessible containing a caret in the subtree and the caret offset in  the text accessible. Previously the algorithm to find an accessible containing the caret failed on some cases. The idea of that approach was that a screen reader should crawl down the hierarchy asking for caret position until it reaches a deepest text accessible having the caret. It failed if you meet non text accessible on the way. As a workaround the screen readers navigated children. While the workaround worked well but it was really slow (see Mozilla bug).

Text attributes


The change is similar to object attributes. A light alternative to the method used to get all text attributes was added.  Now you can specify a string filter and get attributes you need (see Mozilla bug).

Get all hyperlinks


Usually AT needs all hyperlinks at once. Getting them one by one is unperformant (especially for out of process AT) and requires a cache implementation on the browser side. So a new method to get all hyperlinks has been introduced (see Mozilla bug).

Byron Joneshappy bmo push day!

the following changes have been pushed to bugzilla.mozilla.org:

  • [828344] “contains all of the words” no longer looks for all words within the same comment or flag
  • [874081] Add a class to declare which skin is currently used
  • [874327] add Blocking_B2G to tcl connector
  • [870202] investigate ways of improving the performance of bugmail composition
  • [874464] unable to file bugs with the guided bug entry form using IE

 


Filed under: bmo, mozilla

L. David BaronOpen licensing at the W3C

In response to the objection I made on behalf of Mozilla to the previous proposed charter of the HTML Working Group, the W3C has proposed a new charter that allows the group to experiment with an open document license for extension specifications to HTML. If you work for a member company of W3C, I'd encourage you to encourage your AC representative to support the open licensing experiment. Here's why:

W3C publishes specifications, generally written by members of the W3C's working groups, under W3C's copyright ownership [1] and licensed under the W3C Document License, which does not allow other parties to modify the documents. This means that if, for some reason, W3C is no longer the right place to standardize something, those who want to work on that standard elsewhere have to start over rather than starting from the W3C document.

I'm one of several people who would prefer that the W3C license its specifications in a way that allows others to modify and republish the documents (a license that allows forking). I want this to be possible so that if the W3C stops being the right venue for standardizing something, the work can continue elsewhere. I don't want this to be a regular occurrence, nor do I want those who do it to do it without appropriate credit to the specification's original authors. But I want it to be possible to do when it's necessary, so I don't want legal restrictions that prevent forking, only social ones.

I also want it to be possible so that the W3C can collaborate effectively with other venues (such as the WHATWG) that do publish documents under a forking license. Currently this sort of collaboration isn't possible because the W3C can and does take documents from elsewhere that allow modification (often with little credit), but the movement can't happen in the other direction.

For example, when Ian Hickson set out to write what has become HTML5, he had to start over from scratch because previous HTML specifications were not licensed in a way that they can be modified. Ian wrote a new HTML specification at the WHATWG under a license that does allow modifications, and the W3C is using this specification as the bulk of their HTML5 specification (often with little credit to its original author).

Having licenses that allow modification and forking is also a useful governance mechanism. Without such licenses, once W3C has become the place for a particular specification, it is under relatively little pressure to continue to be a good forum. Allowing modification puts a little more competitive pressure on W3C to remain a good forum for standardization. This will reduce the chance of the history of HTML standardization from around 1999-2006 repeating itself.

Some might counter that this lessens the point of standards. I disagree. Standards are useful where there is benefit from a common agreement on how things work. If that agreement still has more value than changing it, then the party modifying the specification won't change it. When the value of sharing a common standard isn't large enough to motivate people to make and keep an agreement, that probably means it isn't worthwhile to do so.

For these reasons, I (as Mozilla's representative to the W3C's Advisory Committee) objected to the previous draft charter of the HTML Working Group. While we'd like to see all the documents of the group published under a liberal license, our objection didn't ask for this. We asked only that the charter allow the group to experiment with liberal licensing on extension specifications. We hope this experiment will give other members of the W3C more experience in working in an environment with liberal licensing and make them more comfortable with extending the experiment to other documents in the future.

The new proposed charter is under review by the W3C's Advisory Committee (the committee consisting of one representative per W3C member company) through May 29. If you are a member of the Advisory Committee or know your company's representative on the Advisory Committee, I'd encourage you to support liberal licensing.

[1] Ownership is transferred through the membership agreement, the clause on "should the Submission be used as a contribution towards a W3C Activity" in the submission request, or the invited expert agreement.

Gregory Jost“Song Reader” by Beck, an open music project An...


Song Reader website lets you upload your own take on Beck's songs


The album was released as sheet music only

“Song Reader” by Beck, an open music project

An album entirely composed but never recorded… Beck’s latest project “Song Reader” is only released as sheet music without any official interpretation or record to support it.

What’s great about it is that it opens the door to an exciting open collaboration music project. Everyone can create their own take on the songs, without being influenced by “the original” (because there is none).

Good news! The Song Reader website, lets you do just that by inviting you to upload your own interpretation.

Ben HearsumLoading Python modules from arbitrary files

tl;dr: Use imp.load_source.

I’ve been hacking on a tool on and off that needs to load Python code from badly named files (eg, “master.cfg”). To my surprise, there wasn’t an obvious way to do this. My “go to” method of doing this is with execfile. For example, this will load the contents of master.cfg into “m”, with each top level object as a key:

m = {}
execfile("master.cfg", m)

This works well enough for simple cases, but what happens when you try to load a module that loads other modules? It turns out that execfile has a nasty limitation of requiring modules that aren’t in sys.path to be in the same directory as the file that calls execfile. You can’t even chdir your way around this, you have to copy the files you need to the caller’s directory. (We actually have some production code that does this.

Someone in #python on Freenode suggested using importlib. That seemed like a fine idea, especially after recently watching Brett Cannon’s “How Import Works” talk. Unfortunately, Python 2.7′s importlib only has a single method which can only load a module by name.

Eventually I came across a Stack Overflow post that pointed me at imp.load_source. This function is similar to execfile in that it loads Python code from a named file. However, it properly handles imports without the need to copy files around. It also has the nice added bonus of returning a module rather than throwing objects into a dict. I ended up with code like this, to load the contents of “foo/bar/master.cfg”:

>>> import os, sys
>>> os.chdir("foo/bar")
>>> sys.path.insert(0, "") # Needed to ensure that the current directory is looked at when importing
>>> m = imp.load_source("buildbot.master.cfg", "master.cfg")

Problem solved!

Sean MartellOpen: An Illustration Mashup

The dino head has been retired from active duty as our official brand within Mozilla but continues to be a strong symbol of pride within the global family – especially with us folk who have been around for a while. Created for Mozilla by Shepard Fairey, the dino head stood as our brand for many years and during that time Fairey went on to create many more recognizable pieces including the “Hope” poster for the Obama campaign.

Unsurprisingly, Pascal Finette had a brilliant idea (he has many of those). What if we created a mashup of the dino head plus Hope poster as a fan art tribute to the original artist? Always up for a fun challenge, I accepted and got to work.

So, here it is! I hope you all enjoy my unofficial fan art mashup (I hope Fairey enjoys it if he sees it too!) which is now my desktop wallpaper, business card art and a Gelaskin for my Nexus 4! Enjoy!

NOTE: This is unofficial fan art from me and is considered a farewell/tribute piece to the Mozilla dino. It in no way ushers in a dino brand revival.

 

 

 

 

Dave TownsendSix years revisited

Two years ago I blogged about how it had been six years since I wrote my first patch for Firefox. Today I get to say that it’s been six years since I started getting paid to do what I love, working for Mozilla. In that time I’ve moved to a new continent, found a wife (through Mozilla no less!), progressed from coder to module owner to manager, seen good friends leave for other opportunities and others join and watched (dare I say helped?) Mozilla, and the Firefox team in particular, grow from the small group that I joined in 2007 to the large company that will soon surpass 1000 employees.

One of the things I’ve always appreciated about Mozilla is how flexible they can be about where you work. Recently my wife and I decided to move away from the bay area to be closer to family. It was a hard choice to make as it meant leaving a lot of friends behind but one thing that made it easier was knowing that I wouldn’t have to factor work into that choice. Unlike many other companies I know there is no strict requirement that everyone work from the office. True it is encouraged and it sometimes makes sense to require it for some employees, particularly when starting out, but I knew that when it came time to talk to my manager about it he wouldn’t have a problem with me switching to working remote. Of course six years ago when I started I was living in the UK and remained so for my first two years at Mozilla so he had a pretty good idea that I could handle it, and at least this time I’m only separated from the main office by a short distance and no time-zones.

The web has changed a lot in the last six years. Back then we were working on Firefox 3, the first release to contain the awesomebar and a built-in way to download extensions from AMO. Twitter and Facebook had only been generally available for about a year. The ideas for CSS3 and HTML5 were barely written, let alone implemented. If you had told me back then that you’d be able to play a 3D game in your browser with no additional plugins, or watch videos without flash I’d have probably thought they were crazy pipe-dreams. We weren’t even Jitting our JS code back then. Mozilla, along with other browser makers, are continuing to prove that HTML, CSS and JS are winning combinations that we can build on to make the future of the web open, performant and powerful. I can’t wait to see what things will be like in another six years.

K Lars LohnThe TransactionExecutor vs. HBase


Yesterday, I wrote about the use of the class TransactionExecutor. Today, I'm going to propose a new way of using it within Socorro's HBase crashstorage subsystem.

When used with Postgres, the TransactionExecutor treats all of the individual steps within a transaction collectively as if they were together a single atomic operation. This is the exactly how we want a database transaction treated. Lets say we have a transaction that consists of steps A, B, C, D. We start a transaction and succeed in steps A and B. However, C fails. The TransactionExecutor rolls back the transaction, essentially undoing steps A and B. Then it retries the whole thing from the beginning after sleeping.

That idea isn't ideal for HBase. Since HBase doesn't support relational database-like transactions, the same strategy of treating multiple steps as one atomic transaction doesn't work as well. Just like the previous example, lets say that for HBase we have four steps A, B, C and D. We start a “transaction” and complete steps A and B before we get a failure in step C. If the transaction executor is used exactly like it is in Postgres, we rollback the transaction. But HBase doesn't support transactions, so steps A and B are not actually undone. The TransactionExecutor then sleeps and after waking, retries the whole thing from the beginning.We end up redoing steps A and B that have already been done. 

What are the consequences of this? It is my understanding that rows in tables in HBase are static. We don't really change them, we just make a new version of the row. The old version of the row still sticks around (the number of old rows that HBase saves is configurable). By starting the transaction over from the beginning, we're increasing the amount of space that HBase needs to save the same information.

Honestly, this isn't much of a problem. During a failure, having multiple copies of the same information will happen only to a single row (or in our case, one row per actively writing thread during the HBase problem). That's not a lot of extra storage wasted in comparison to the huge size of our data storage.

However, if we care to, we can resolve this duplication problem by using the TransactionExecutor in a different manner. Rather than treating all the steps A, B, C, D as if they are collectively atomic, we could use the TransactionExecutor individually on each step. If we don't fail until step C, then only step C will be retried. We can avoid the duplication of steps A and B. 

For purity sake, it makes sense to use the  TransactionExecutor on operations that are truly atomic.  In our application with HBase, it really doesn't matter too much and would be likely more trouble than it is worth to change.




Gervase MarkhamCorrectly Indicating Newsworthiness

In free software, there is a fairly smooth continuum between purely internal discussions and public relations statements. This is partly because the target audience is always ill-defined: given that most or all posts are publicly accessible, the project doesn’t have full control over the impression the world gets. Someone—say, a slashdot.org editor—may draw millions of readers’ attention to a post that no one ever expected to be seen outside the project. This is a fact of life that all open source projects live with, but in practice, the risk is usually small. In general, the announcements that the project most wants publicized are the ones that will be most publicized, assuming you use the right mechanisms to indicate relative newsworthiness to the outside world.

– Karl Fogel, Producing Open Source Software

David Rajchenbach TellerAdd-on breakage warning: Session Restore is changing

As part of project Async, we have been working on refactoring Firefox’ Session Restore to ensure that it does not block the main thread. Part of the work has been cleaning up the code and the data structures involved in Session Restore both to give us some maneuverability and to improve the chances of catching refactoring errors.

Unfortunately, a large number of add-ons seem to rely upon these undocumented data structures. Some of their features might therefore stop working. If you are the author of one such add-on, you should monitor carefully bug 874381 and its blockers. If you realize that we are about to break your add-on, please inform us asap, so that we can work out a solution.


Patrick WaltonSafe Manual Memory Management

If there’s one feature of Rust that is probably the most unique among languages in industry, it’s safe manual memory management.

It’s easiest to explain safe manual memory management by explaining how it differs from the memory models of other languages. There are a few different models in common use in industry languages:

  • Unsafe manual memory management—These languages provide very fine-grained control over memory allocation; heap memory can be explicitly allocated and deallocated. The most important examples here are C and C++. The well-known downside of this approach is that memory safety violations can only be detected at runtime with a memory safety checker such as Valgrind or Address Sanitizer. Memory safety violations that go unchecked can lead to crashes at best and exploitable security vulnerabilities at worst.

  • Full garbage collection—The majority of modern languages expose a memory model that falls into this category—the space is very diverse, ranging from Java to Go to JavaScript to Ruby to Haskell. In general, such languages place all allocations into the heap instead of the stack, although escape analysis and value types may be used to reduce the number of heap allocations. Periodically, a garbage collector scans all pointers on the stack and in the heap, judges unreachable objects dead, and reclaims them. This approach has the advantage of memory safety at compile time—the language arranges for there to be no dangling pointers, wild pointers, and so forth. The downsides, however, are:

    1. The garbage collector may run at an inconvenient time. This can be mitigated by explicit control over when the GC runs, although if the garbage collector must collect multiple threads’ heaps at the same time, this may be difficult to synchronize. This can also be mitigated by using manual memory pooling and free lists, although pooling has undesirable safety properties—much like unsafe manual memory management, there is no static guarantee that objects allocated from a pool are returned properly or that an object is not reachable when returned to the pool. Incremental and concurrent garbage collectors help here, but they are not free, as they typically require write and/or read barriers, reducing throughput.

    2. When it runs, the garbage collector must mark all pointers to discover which ones are live, reducing throughput of the application. Essentially, the GC must discover at runtime what a C++ (say) programmer knows at compile time. Not much can typically be done about this cost in fully garbage-collected languages, short of falling back to unsafe manual memory management. Pools don’t help much here, because the GC must still trace the pointers into the pool. Even pointers into the stack generally must be traced.

  • Garbage collection with value types and references—This category includes languages like C#. (I believe D falls into this category as well, although I may be mistaken.) These languages are essentially garbage-collected, but they include value types which are guaranteed to be stack-allocated if in local variables. Additionally, and most importantly, they include reference parameters (and sometimes reference locals), which allow stack-allocated values to be temporarily aliased when calling another function. Effective use of value types can reduce marking and sweeping time. In general, this system is an effective addition to a garbage-collected system, allowing a good measure of additional control without much cost in complexity and no cost in memory safety. It is not, however, typically sufficient to write programs without using the garbage collector at all; the system is too simple to statically encode anything other than the most basic memory management patterns.

Where does Rust fit in? Actually, it fits into a category all to itself among industry languages (although one shared by various research languages, like Cyclone). Rust offers safe manual memory management (although some have objected to the term “manual” here). It extends the system described above as “garbage collection with value types and references” in two important ways:

  1. You can allocate memory that will not be traced by the garbage collector, and free it manually if you choose. This is the feature of Rust known as “unique pointers”. Rust will automatically free memory that is uniquely owned when its owning pointer goes out of scope. It’s also easy to write a function that acts exactly as free does, so you can precisely choose when your objects die. Unique pointers are not traced by the GC (unless they point to a type that transitively contains a garbage-collected pointer), so they are an effective way to cut down marking times.

  2. You can return references and place references into data structures. Like other references, these references are not traced by the garbage collector. As long as the references follow a stack discipline, meaning that they point to memory that was allocated by one of the callers of the current function, the compiler allows them to be placed anywhere. This adds a great deal of expressiveness over the reference parameter approach, and it enables a large number of programs to be written without using the garbage collector at all.

In terms of safety and performance, safe manual memory management is having your cake and eating it too. You get memory safety like a garbage-collected language, but control like that of unsafe manual memory management. But this system has its downsides as well—most importantly, complexity of implementation and interface. Learning to use references and unique pointers poses a significant learning curve. But, once the system is learned, it’s remarkably flexible, with an attractive combination of performance and safety.

Selena DeckelmannDistributed databases: a series of posts including 2-phase commit in Postgres

There’s a fantastic set of blog posts about distributed databases and network partitioning, starting with this post explaining the perils of trying to “communicate with someone who doesn’t know you’re alive.”

The next post is about Postgres and 2-phase commit. And there are four additional posts in the series.

The whole series worth reading for anyone interested in data stores, consistency and Postgres! :)

Patrick ClokeInstantbird 1.4 Released!

After a bunch of l10n build problems, we've finally released Instantbird 1.4, which includes updates to libpurple 2.10.7 and Mozilla 20.  In particular this includes:
  • Updated Twitter code that uses v1.1 of their API (v1.0 will be disabled on June 11th, 2013).
  • Better character counter for Twitter (it now takes into account if URLs are embedded).
  • Updated log viewer which organizes logs by date (and nests them by week, month, etc.)
  • Better support for IRC bouncers.
  • Support for overriding self-signed/invalid/out-of-date certificates for IRC.
If you've been using some other instant messaging client (e.g. Pidgin or Adium); I'd highly suggest giving Instantbird a try, especially if you also go on IRC. Instantbird has great IRC support! (And...if you do have issues, feel free to ping me in #instantbird on irc.mozilla.org and let me know what your issue is.)

You can download it here, or view the full release notes.

Austin KingVidyo improvement ideas

I’ve been remote with Mozilla for almost 5 years. This is very difficult, from an emotional and a shared vision perspective.

Vidyo, among other tools, have been very important in maintaining a “lifeline” to my day to day teammates.

I have one big pet peeve, which I’d like to document, so that Zandr and others can upstream this to the Vidyo team.

The Problem

Many calls have 8-12 people. Vidyo does a Brady Bunch type visualization, but at some point it starts hiding participants.

Technically, I can imagine there are several good reasons for hiding participants: Full video from all participants is a bandwidth hog, managing latency is more difficult, etc.

But humans are “out of sight, out of mind”. I’ve been in so many calls where people start speaking as though Jane is not on the call, as soon as her video window gets hidden.

“Uhmmm, I’m on the call. Vidyo is just being dumb” — Jane

“Sorry, you weren’t on my screen” — Tarzan

A workaround I use, but which is not a common practice, is to display the list of all participants on a call. I have this overlay part of my Vidyo window and it’s fugly.

Aside: How to show everyone

Click Gear icon. Click Participants.

A couple ideas

Vidyo should always display some kind of representation of each person on the call.

Low bar would be a textual list of participant names down in a new column on the left.

Vidyo with all names

A better solution would be to add an avatar grid for participants who had been de-prioritized and would be hidden in the current version of the software. These avatars would be smaller, so you could fit 100% of the participants on the screen.

Vidyo with static avatars

The small squares on the left are static. These are very cheap from a technical point of view. They give us the visual human connection, which we need.

The large squares on the right are video, just as we see today.

The avatar image would be taken a few seconds into the call via the webcam.

There would be visual feedback to a user showing their current avatar frame.

I’d love to be able to “Refresh Avatar” which would capture my current webcam view and make it a static avatar.

K Lars LohnThe history of the TransactionExecutor



Or, "yet another awkwardly named class"

One of the most important tenets of Socorro is to be resilient when external resources fail. The Mozilla Socorro deployment depends on Postgres and HBase to work. However, these are two external resources that can fail.

What happens when we try to write to one of these and we find that the resource in unavailable? Earlier versions of Socorro treated the HBase and Postgres failure cases separately.

For Postgres, since it is a transactional storage system, Socorro employed the native transactional behaviors. Interacting with Postgres involves a series of steps (insert, update, delete, select) followed by commit or rollback. If one of the intervening steps were to fail, we didn't want the program to quit, nor did we want errors to be ignored. Socorro implemented a “backing off retry” behavior. On failure of a step, the code would classify failure into one of two types: retriable and fatal. In either case, a rollback would be issued. In the retry case, the code would sleep for a predetermined amount of time and then retry the transaction from the beginning. In the fatal case, there is no choice except to allow the program to shutdown.

For HBase, true transactions are not supported. However, the behavior Socorro wanted was just the same as the Postgres case: classify the failure and then, if retriable, repeat the steps until we have success. HBase doesn't have the concept of commit and rollback, but the intervening steps of a transaction may be repeated without negative consequence.

Even though the behavior was similar, the two cases were coded independently and shared no code. In the grand Configman refactoring of Socorro, the two cases were merged into one class to maximize reuse. The Postgres case was used as the canonical example. Dummy null op commit and rollback were added to the HBase connection classes to facilitate the use of the class.

How do the TransactionExectorclasses work? There are three of them with slightly different behaviors:
  • TransactionExecutor
  • TransactionExecutorWithLimitedBackoff
  • TransactionExecutorWithInfiniteBackoff
The code can be found at: https://github.com/mozilla/socorro/blob/master/socorro/database/transaction_executor.py

These classes implement methods that accepts a function, a connection context to some resource and arbitrary function parameters. When instantiated and invoked, these classes will call the function passing it the connection and the additional parameters. The raising of an exception within the function indicates that a failure of the transaction: a rollback is automatically issued on the connection context. If the function succeeds and exits normally, then a 'commit' is issued on the connection context.

The first class in the list above is the degenerate single-shot case. It doesn't implement any retry behavior. If the function fails by raising an exception, then a rollback is issued on the connection and program moves on. Success results in a commit and the program moves on.

The latter two classes implement a retry behavior. If the function raises an exception, the Transaction class checks to see if the exception is of a type that is eligible for retry. If it is eligible, then a delay amount is selected and the thread sleeps. When it wakes, it tries to invoke the function again with the same parameters. The time delays are specified by a list of integers representing successive numbers of seconds to wait before trying again. For the class TransactionExecutorWithLimitedBackoff, when the list of time delays is exhausted the transaction is abandoned and the program moves on. The TransactionExecutorWithInfiniteBackoff will never give up, running the last time in the delay list over and over until the transaction finally succeeds or somebody kills the program.

How does the TransactionExecutor determine if an exception is eligible for retry? The connection context object is required to have a couple instance variables and methods to assist in the determination.

First, operational_exceptions defines a collection of exceptions that are eligible for the retry behavior. If one of the exceptions from this collection is raised, the retry behavior is triggered.

conditional_exceptions is a list of ambiguous exceptions that may or may not be eligible for retry. We encountered this with Postgres using psycopg2 on the ProgrammingError exception. Normally, this type of exception would not be retriable because it indicates a fundamental problem with a query such as a syntax error. Syntax errors are not retrible. However, sometimes we get network errors disguised as ProgrammingErrors; these are retriable.

If an exception found in the conditional_exceptions collection is raised, we have to further examine the error to determine if it should result in a failure or retry. The instance method is_operational_exception implemented by the connection class is used to determine in the current exception is retriable or not. In the case of Postgres, we look to the text of the exception to see if it contains the string “EOF”. We know that's a network error, not really a programming error so we can do a retry.

Is this class named poorly? Now that we've got many more external resources using this retry behavior and only Postgres is truly transactional, it seems that the name may not be right. Perhaps ExternalResourceActionRetrier?







Tarek ZiadéA step-by-step introduction to Circus

Note

Circus is a process & socket manager. See https://circus.readthedocs.org

https://farm9.staticflickr.com/8420/8751753401_0760d37279.jpg

Photo by kennethreitz

During Django Con, I was asked how to use Circus to run & monitor a Python web application. The documentation has no single page step-by-step tutorial yet, so here goes... this blog post will be integrated into the documentation for the next release.

Installation

Circus is tested under Mac OS X and Linux, on the latest Python 2.6 and 2.7. To run a full Circus, you will also need libzmq, libevent & virtualenv.

Under Debuntu:

$ sudo apt-get install libzmq-dev libevent python-virtualenv

Create a virtualenv and install circus, circus-web and chaussette in it

$ virtualenv /tmp/circus
$ cd /tmp/circus
$ bin/pip install circus
$ bin/pip install circus-web
$ bin/pip install chaussette

Once this is done, you'll find a plethora of commands in the local bin dir.

Usage

Chaussette comes with a default Hello world app, try to run it:

$ bin/chaussette

You should be able to visit http://localhost:8080 and see hello world.

Stop Chaussette and add a circus.ini file in the directory containing:

[circus]
stats_endpoint = tcp://127.0.0.1:5557
httpd = 1

[watcher:webapp]
cmd = bin/chaussette --fd $(circus.sockets.web)
numprocesses = 3
use_sockets = True

[socket:web]
host = 127.0.0.1
port = 9999

This config file tells Circus to bind a socket on port 9999 and run 3 chaussettes workers against it. It also activates the Circus web dashboard and the statistics module.

Save it & run it using circusd:

$ bin/circusd --daemon circus.ini

Now visit http://127.0.0.1:9999, you should see the hello world app.

You can also visit http://localhost:8080/ and enjoy the Circus web dashboard.

Interaction

Let's use the circusctl shell while the system is running:

$ bin/circusctl
circusctl 0.7.1
circusd-stats: active
circushttpd: active
webapp: active
(circusctl)

You get into an interactive shell. Type help to get all commands:

(circusctl) help

Documented commands (type help <topic>):
========================================
add     get            list         numprocesses  quit     rm      start   stop
decr    globaloptions  listen       numwatchers   reload   set     stats
dstats  incr           listsockets  options       restart  signal  status

Undocumented commands:
======================
EOF  help

Let's try basic things. Let's list the web workers processes and add a new one:

(circusctl) list webapp
13712,13713,13714
(circusctl) incr webapp
4
(circusctl) list webapp
13712,13713,13714,13973

Congrats, you've interacted with your Circus! Get off the shell with Ctrl+D and now run circus-top:

$ bin/circus-top

This is a top-like command to watch all your processes' memory and CPU usage in real time.

Hit Ctrl+C and now let's quit Circus completely via circus-ctl:

$ bin/circusctl quit
ok

Next steps

You can plug your own WSGI application instead of Chaussette's hello world simply by pointing the application callable.

Chaussette also comes with many backends like Gevent or Meinheld.

Read https://chaussette.readthedocs.org/ for all options.

Luis VillaAt the Wikimedia Foundation (for, um, three months now)

Since it was founded 12 years ago this week, Wikipedia has become an indispensable part of the world’s information infrastructure. It’s a kind of public utility: You turn on the faucet and water comes out; you do an Internet search and Wikipedia answers your question. People don’t think much about who creates it, but you should. We do it for you, with love.

Wikimedia Foundation Executive Director Sue Gardner, from http://blog.wikimedia.org/2013/01/14/wikipedia-the-peoples-encyclopedia/

As Sue says, the people who create Wikipedia are terrific. I’m lucky enough to say that I’ve just wrapped up my first three months as their lawyer – as Deputy General Counsel at the Wikimedia Foundation. Consider this the personal announcement I should have made three months ago :)

Wikimania 2012 Group Photograph, by Helpameout, under CC-BY-SA 3.0, available from https://commons.wikimedia.org/wiki/File:Wikimania_2012_Group_Photograph-0001.jpg
Wikimania 2012 Group Photograph, by Helpameout, under CC-BY-SA 3.0.

Greenberg Traurig was terrific for me: Heather has a wealth of knowledge and experience about how to do deals (both open source and otherwise), and through her, I did a lot of interesting work for interesting clients. Giving up that diversity and experience was the hardest part of leaving private practice.

Based on the evidence of the first three months, though, I made a great choice – I’ve replaced diversity of clients with a vast diversity of work; replaced one experienced, thoughtful boss with one of equal skill but different background (so I’m learning new things); and replaced the resources (and distance) of a vast firm with a small but tight and energized team. All of these have been wins. And of course working on behalf of this movement is a great privilege, and (so far) a pleasure. (With no offense to GT, pleasure is rarely part of the package at a large firm.)

The new scope of the work is perhaps the biggest change. Where I previously focused primarily on technology licensing, I’m now an “internet lawyer” in the broadest sense of the word: I, my (great) team, and our various strong outside counsel work on topics from employment contracts, to privacy policies, to headline-grabbing speech issues, to patent/trademark/copyright questions – it is all over the place. This is both challenging, and great fun – I couldn’t ask for a better place to be at this point in my life. (And of course, being always on the side of the community is great too – though I did more of that at Greenberg than many people would assume.)

I don’t expect that this move will have a negative impact on my other work in the broader open source community. If anything, not focusing on licensing all day at work has given me more energy to work on OSI-related things when I get home, and I have more flexibility to travel and speak with and for various communities too. (I’m having great fun being on the mailing lists of literally every known open source license revision community, for example. :)

If you’d like to join us (as we work to get the next 1/2 billion users a month), there are a lot of opportunities open right  now, including one working for me on my team, and some doing interesting work at the overlap between community, tech, and product management. Come on over – you won’t regret it :)

Gervase MarkhamA Big ‘Thank You’ To Microsoft…

…for their help in significantly mitigating the problems we were having with running out of memory when linking Firefox with Profile-Guided Optimization using Microsoft Visual C++. (If we’d have had to turn off PGO due to this problem, that would have made Firefox’s performance on Windows significantly worse.) Ted has the write-up.

Mark SurmanMozilla and Badges: where next?

Open Badges started as a modest experiment: build open source badge issuing software for ourselves and others. As momentum around this experiment has grown, it feels like the opportunity is bigger: we could build openness and user empowerment into how learning — and professional identity — work all across the web. With Open Badges 1.0 out there in the world, now is the right time to ask: where next for Mozilla and badges?

Badges Evolution

When Mozilla and MacArthur Foundation first started work on Open Badges about 18 months ago, the plan was to build a badge interchange standard (like SMTP for skills) and a collection of open source software for issuing and sharing badges (Badge Backpack, Open Badger, etc.). We’ve built all these things. And we’ve put up a reference implementation that Mozilla and others are using. This was really the limit of our original plan: build some basic open tech for badges and put it out there in the world.

The thing is: there has been way more excitement and pick up of badges than we expected. Even though Open Badges only launched officially in March, there are already over 800 unique providers who have issued almost 100,000 badges. We are also starting to see the development of city-wide systems where learners can pick up hundreds of different badges from across dozens of learning orgs and combine them all into a single profile. Chicago is the first city to do this (June 1), but Philadelphia and San Francisco are not far behind. And, this is just the tip of the iceberg: orgs like the Clinton Global Initiative and the National Science Foundation are focusing on badges in a way that is likely to drive even more educators to pick up the Open Badges standard, making their badges interoperable with others.

Of course, the fact that educators and policy makers are interested in badges doesn’t represent a victory in itself. It just shows momentum and buzz. The real opportunity — and the real impact — comes when learners and employers get excited about badges. Mozilla never planned to build offerings for these audiences. Increasingly, it feels like we should.

Badges Audiences

In the Internet era, people learn things online and out of school all the time. Whether you want to make a web page, knit a sweater or get better at calculus, the internet makes it easy to learn on your own or with a group of friends outside of a school setting. However, there is no good way to get credentials or recognition for this kind of learning. And, even if there was, there is no trusted, verifiable way to plug that recognition into Facebook, About.me and other places that make up your online identity. People have no good way to show ‘what they know’ online.

Similarly, employers are increasingly turning to the internet to find talent. They use sites like LinkedIn that let you search online resumes. Or, increasingly, to sites like Gild and TalentBin that use data mining to find potential hires. The problem: these services do not offer granular or variable skills profiles. And, with some of them, there are significant issues around privacy: people are being offered up as potential hires without even knowing that these sites are collecting data about them.

Mozilla could offer a distributed, open source and privacy-friendly solution to problems like these. We could help learners show their skills in all their online profiles and also help employers search for talent reliably. However, to do so, we’d have to build a Firefox-quality offering for learners and employers on top of Open Badges. While this hasn’t been our focus up til now, I’m thinking more and more that this is something we should consider.

Badge product

In some ways, there is a parallel to Gecko and Firefox. Gecko provides the underlying platform for shaping standards around our vision of the web. But we need a popular consumer offering like Firefox if we want this vision to actually become relevant in the market. Right now, with Open Badges, we’re mostly just playing at the underlying standards layer. If we really want to shape how learning and professional identity work on the web, we probably need to build our own offerings directly for the people who most want and need badges.

Now is the time to be looking at where the opportunity is in this space. Momentum and demand is amongst educators is growing. More and more start ups are appearing in the badges, portfolio and skills spaces. And likelihood that badges will be important for learners and employers is growing. We need to be asking ourselves: how can Mozilla — and its values — shape this space?

With this in mind, Erin Knight is leading an effort over the next few months to look at different badges product options. She’ll be providing updates on her blog. And I’ll be summarizing here as well. If you have ideas on where Mozilla should go on all of this, we’d love to have you involved as we think this through. Comments here on this post are a good place to start.


Filed under: badges, drumbeat, education, learning, mozilla, webmakers

Hal WineUsing hg & git for the same codebase

Using hg & git for the same codebase

Speaker Note

Following are the slides I presented at the RELENG 2013 workshop on May 20th, 2013. Paragraphs like this were not part of the presented slides - rather speaker notes.

If you prefer, here is a PDF version.

Hal Wine hwine@mozilla.com
Release Engineering
Mozilla Corporation

Issues and solutions encountered in maintaining a single code base under active development in both hg & git formats.

Background

  • Mozilla Corporation operates an extensive build farm that is mostly used to build binary products installed by the end user. Mozilla has been using Mercurial repositories for this since converting from CVS in 2007. We currently use a 6 week “Rapid Release” cycle for most products.

    Speaker Note

    We currently have upwards of 4,000 hosts involved in the continuous integration and testing of Mozilla products. These hosts do approximately 140 hours of work on each commit.

  • Firefox Operating System is a new product that ships source to be incoporated by various partners in the mobile phone industry. These partners, experienced with the Android build process, require source be delivered via git repositories. This is close to a “Continuous Release” process.

    Speaker Note

    A large part of the FxOS product is code used in the browser products. That is in Mercurial and needs to be converted to git. Most new code modules for FxOS are developed on github, and need to be converted to Mercurial for use in our CI & build systems.

Summary

  • What we initially set out to do:
    • Make it purely a developer choice which dvcs to use.

      Speaker Note

      Ideal was to allow developers to make dvcs as personal a choice as editor.

    • Support multiple social coding sites.

      Speaker Note

      These social coding sites, such as github and bitbucket, make it much easier for new community members to contribute.

  • That was much tougher than anticipated.
    In theory, git & hg are very close...
    ... In practice, “the devil is in the details”.
  • Where we are:
    • Changed direction to support FFOS release to partners.
    • Quickly mirror Repository of Record (RoR) between git & hg.
    • CI/build system remains Mercurial centric.

Challenge Areas

  • Changesets have different hashes in Mercurial and git.
    • We added tooling to support both in static documents such as manifest files.
    • All tools continue to use hg hash as primary value for indexing and linking.
  • Propagation delays of changesets to the “other” system.

    Speaker Note

    For most use cases, the approximately 20 minute average we’re achieving is acceptable.

    • Compounded by hash differences between two systems.

      Speaker Note

      A common use case here is a developer wanting to start a self serve build. If the commit was to git, the self serve build won’t be successful until that commit is converted to hg.

      We are continuing work on this. It is closely tied to determining which commit broke the build, when multiple repositories are involved.

  • Build details
    • Movable tags are not popular in git based workflows, but have been a common technique at Mozilla to mark “latest”.

Challenge Areas (Con’t)

  • Mixed philosophies are often linked with mixed repositories.
    • Android never wants history to appear to change. Downstream servers allow only fast forward changesets and deny deletions.

    • Mozilla uses “RoR is authoritative”.

      Speaker Note

      Either approach is self consistent. It is when the two need to interact that challenges arrise.

  • Conversion failures
    • Occasional hg-git conversion failures, due to implementation details of hg & git.

      Speaker Note

      • Dates in export patches (e.g. hg uses seconds, git uses minutes, in time)
      • Email validation (git stricter than hg)
    • Since commit already accepted by hg, hg-git must be modified

      Speaker Note

      This requires inhouse resources to respond urgently to patch the conversion machinery. Without conversion, there are no builds.

Alternate Approaches

  • To support your own “use the DVCS you want” infrastructure requires:
    • production quality hg server
    • production quality git server
    • in house ability to address conversion issues (as already mentioned)
  • I’m aware of two commercial alternatives. Both of these use a centralized RoR which supports git and/or hg interfaces for developer interaction.

    Speaker Note

    And at least one explicitly does not have a git back end.

  • You can leave it to developers to scratch their own itch independently. Given diversity of workflows, this may be more cost effective than obtaining consensus.

Future Research

Areas of particular interest for further study include:

  • What is the set of enforceable assertions which would ensure the tooling can maintain lossless conversion between DVCS?

  • What minimum conditions must be maintained in conversions to preclude downstream conflicts?

  • What workflows can be supported to minimize issues?

  • Are there best practice incident management protocols for addressing problem commits.

    Speaker Note

    The common example is a commit contains sensitive material it should not. There are cases were limiting the scope of distribution can have significant business value.

Fabien CazenaveVim Syntax Highlighting for Mozilla C++ Files

I’ve tweaked the cpp.vim file that comes with Vim 7.3 to highlight most Mozilla-specific keywords when working on the editor core. A lot of Mozilla-specific types and that can be added manually but the task gets bigger when it comes to nsI* interfaces or NS_* macros…

Most nsI* interfaces can be grabbed with find/grep/sed:

find src/mozilla -regex ".*\.\(idl\|h\)" -exec grep "^\(class\|interface\)\s*nsI" '{}' \; | sed 's/\(:\|;\|,\|{\).*$//' | sed 's/^.*nsI/nsI/' | sed 's/\s*$//' | sort -u

same thing for NS_ERROR* / NS_IMPL* macros and constants:

find src/mozilla -regex ".*\.\(idl\|h\)" -exec grep "^#define\s*NS_ERROR" '{}' \; | sed 's/^#define\s*//' | sed 's/\s.*$//' | sed 's/(.*$//' | sort -u
find src/mozilla -regex ".*\.\(idl\|h\)" -exec grep "^#define\s*NS_IMPL" '{}' \; | sed 's/^#define\s*//' | sed 's/\s.*$//' | sed 's/(.*$//' | sort -u

Here’s the resulting cpp.vim file including the ~900 Mozilla-specific lines (ouch!). Copy it to your ~/.vim/syntax/ directory and voilà, your C++ files should be much more colorful.

Now it’d be really great if:

  • we had omni-completion for the nsI* interfaces instead of just the keywords;
  • we had a similar file (keywords + omni-completion) for JavaScript — mostly for the DOM API
  • this file could be generated automatically — say, with DXR;
  • this file could be included in the Mozilla tree (e.g. a .vimrc file in the top source dir).

To all Vim fanboys among the Mozilla community: I’d love to get your input about that. Maybe we could start a “vim-moz-syntax” project on github or something?

EDIT: (2013-05-19)

  • this work is now available on github: https://github.com/mozfr/mozilla.vim
  • there’s been an article about this in Russian: http://softdroid.net/Vim-Syntax-Highlighting

Gregory SzorcUsing Docker to Build Firefox

I have the privilege of having my desk located around a bunch of really intelligent people from the Mozilla Services team. They've been talking a lot about all the new technologies around server provisioning. One that interested me is Docker.

Docker is a pretty nifty piece of software. It's essentially a glorified wrapper around Linux Containers. But, calling it that is doing it an injustice.

Docker interests me because it allows simple environment isolation and repeatability. I can create a run-time environment once, package it up, then run it again on any other machine. Furthermore, everything that runs in that environment is isolated from the underlying host (much like a virtual machine). And best of all, everything is fast and simple.

For my initial experimentation with Docker, I decided to create an environment for building Firefox.

Building Firefox with Docker

To build Firefox with Docker, you'll first need to install Docker. That's pretty simple.

Then, it's just a matter of creating a new container with our build environment:

curl https://gist.github.com/indygreg/5608534/raw/30704c59364ce7a8c69a02ee7f1cfb23d1ffcb2c/Dockerfile | docker build

The output will look something like:

FROM ubuntu:12.10
MAINTAINER Gregory Szorc "gps@mozilla.com"
RUN apt-get update
===> d2f4faba3834
RUN dpkg-divert --local --rename --add /sbin/initctl && ln -s /bin/true /sbin/initctl
===> aff37cc837d8
RUN apt-get install -y autoconf2.13 build-essential unzip yasm zip
===> d0fc534feeee
RUN apt-get install -y libasound2-dev libcurl4-openssl-dev libdbus-1-dev libdbus-glib-1-dev libgtk2.0-dev libiw-dev libnotify-dev libxt-dev mesa-common-dev uuid-dev
===> 7c14cf7af304
RUN apt-get install -y binutils-gold
===> 772002841449
RUN apt-get install -y bash-completion curl emacs git man-db python-dev python-pip vim
===> 213b117b0ff2
RUN pip install mercurial
===> d3987051be44
RUN useradd -m firefox
===> ce05a44dc17e
Build finished. image id: ce05a44dc17e
ce05a44dc17e

As you can see, it is essentially bootstrapping an environment to build Firefox.

When this has completed, you can activate a shell in the container by taking the image id printed at the end and running it:

docker run -i -t ce05a44dc17e /bin/bash
# You should now be inside the container as root.
su - firefox
hg clone https://hg.mozilla.org/mozilla-central
cd mozilla-central
./mach build

If you want to package up this container for distribution, you just find its ID then export it to a tar archive:

docker ps -a
# Find ID of container you wish to export.
docker export 2f6e0edf64e8 > image.tar
# Distribute that file somewhere.
docker import - < image.tar

Simple, isn't it?

Future use at Mozilla

I think it would be rad if Release Engineering used Docker for managing their Linux builder configurations. Want to develop against the exact system configuration that Mozilla uses in its automation - you could do that. No need to worry about custom apt repositories, downloading custom toolchains, keeping everything isolated from the rest of your system, etc: Docker does that all automatically. Mozilla simply needs to publish Docker images on the Internet and anybody can come along and reproduce the official environment with minimal effort. Once we do that, there are few excuses for someone breaking Linux builds because of an environment discrepancy.

Release Engineering could also use Docker to manage isolation of environments between builds. For example, it could spin up a new container for each build or test job. It could even save images from the results of these jobs. Have a weird build failure like a segmentation fault in the compiler? Publish the Docker image and have someone take a look! No need to take the builder offline while someone SSH's into it. No need to worry about the probing changing state because you can always revert to the state at the time of the failure! And, builds would likely start faster. As it stands, our automation spends minutes managing packages before builds begin. This lag would largely be eliminated with Docker. If nothing else, executing automation jobs inside a container would allow us to extract accurate resource usage info (CPU, memory, I/O) since the Linux kernel effectively gives containers their own namespace independent of the global system's.

I might also explore publishing Docker images that construct an ideal development environment (since getting recommended tools in the hands of everybody is a hard problem).

Maybe I'll even consider hooking up build system glue to automatically run builds inside containers.

Lots of potential here.

Conclusion

I encourage Linux users to play around with Docker. It enables some new and exciting workflows and is a really powerful tool despite its simplicity. So far, the only major faults I have with it are that the docs say it should not be used in production (yet) and it only works on Linux.

Seif LotfyGlobaleaks 0.2 Alpha

Globaleaks 0.2 Alpha is out.

Globaleaks is an open source project aimed at creating a worldwide, anonymous, censorship-resistant, distributed whistle-blowing platform. It enables organizations interested in running whistle-blowing initiatives to setup their own safe zone, where whistle-blowers and recipients can exchange data.

2 Years ago I helped out with the development of Globaleaks 0.1. And although I am not active anymore, I really support the initiative behind it. Now with the HERMES Center for Transparency and Digital Human Rights backing it up, it has grown a lot and shaped up to be a very organized and thought through project.

TL;DR:

  • Full rewrite
  • More flexible and extensible
  • Linux ready-made system and network hardened installation
  • Written in python using twisted
  • New Frontend

Try it out:

Try out the demo. It is pretty straight forward.

Help out:

As young project, Globaleaks can use some help fixing bugs. Just head to the wiki and read through it. It is pretty straight forward, and explains the modules, security concepts and set up instructions.

Globaleaks already has Debian and Ubuntu ready packages. An easy way to help out is to set up a  PPA for us on Launchpad. :D

Get in touch:

You can contact the Globaleaks team at info () globaleaks org or on IRC on #globaleaks at irc.oftc.net

Here are some screenshots of the new frontend :D

Congratulation you are using Tor

Congratulations you are using Tor

Receiver selection page

Receiver selection page

The submission receipt

The submission receipt

Configuring a receiver

Configuring a receiver

Configuring a context

Configuring a context

flattr this!

Amir AharoniAlways define the language and the direction of your HTML documents, part 02: Backwards English

In part 01 of these series, I showed why is it important to always define the language and the direction of all HTML content and not rely on the defaults: The content may get embedded in a document with different direction and be displayed incorrectly.

This issue is laughably easy to avoid: If you are writing the content, you are supposed to know in what language it is written, so if it’s English, just write <html lang=”en” dir=”ltr”> even though these seem to be the defaults. Nineteen or so characters that ensure your content is readable and not displayed backwards. Please do it always and tell all your friends to do it.

The problem is that you don’t only have to explicitly set the language and the direction, but, as silly as it sounds, you have to set them correctly, too. A more subtle, but nevertheless quite frequent and disruptive bug is displaying presumably, but not actually, translated content in a different direction. This happens quite frequently when a website supports the browser language detection feature, known as Accept-Language:

  1. The web server sees that the browser requests content in Hebrew.
  2. The web server sends a response with <html lang=”he” dir=”rtl”>, but because the website is not actually translated, the text is shown in the fallback language, which is usually English.
  3. The user sees the content just like this numbered list, which I intentionally set to dir=”rtl”: with the numbers and the punctuation on the wrong side, and possibly invisible, because English is not a right-to-left language.

Of course, it can go even worse. Arrows can point the wrong way and buttons and images can overlap and hide each other, rendering the page not just hard to read, but totally unusable.

This bug is also an example of the Software Localization Paradox: It manifests itself when Accept-Language is not English, but most developers install English operating systems and don’t bother to change the preferred language settings in the browser, so they never see how this bug manifests itself. The site developers don’t bother to test for it either.

The solution, of course, is to set a different language and direction only if the site is actually translated, and not to pretend that it’s translated if it’s not.

Here are two examples of such brokenness. Both sites are important and useful, but hard to use for people whose Accept-Language is Hebrew, Persian or Arabic.

Here’s how the Mozilla Developer Network website looks in fake Hebrew:

Mozilla Developer Network website, in English, but right-to-left

Mozilla Developer Network website, in English, but right-to-left

Notice how the full stops are on the left end and how the text overlaps the images in the tiles on the right-hand side. This is how it is supposed to look, more or less:

Mozilla Developer Network home page in English, left-to-right

Mozilla Developer Network home page in English, left-to-right

I manually changed dir=”rtl” to dir=”ltr” using the element inspector from Firefox’s developer tools and I also had to tweak a CSS class to move the “mozilla” tab at the top.

The above troubles are reported as bug 816443 – lang and dir attributes must be used only if the page is actually translated.

After showing an example of a web development bug from a site for, ahem, web developers, here is an even funnier example: The home page of Unicode’s CLDR. That’s right: Unicode’s own website shows text with incorrect direction:

The Unicode CLDR website, in English but right-to-left

The Unicode CLDR website, in English but right-to-left

The only words translated here are “Contents” (תוכן) and “Search this site” (חיפוש באתר זה), which is not so useful. The rest is shown in English, and the direction is broken: Notice the strange alignment of the content and the schedule table. A few months ago that table was so broken that its content wasn’t visible at all, but that was probably patched.

Here’s how it is supposed to look:

The CLDR home page in English, appropriately left-to-right

The CLDR home page in English, appropriately left-to-right

I tried reporting the CLDR home page direction bug, but it was closed as “out-of-scope”: The CLDR developers say that the Google Sites infrastructure is to blame. This is frustrating, because as far as I know Google Sites doesn’t have a proper bug reporting system and all I can do is write a question about that direction problem in the Google Sites forum and hope that somebody notices it or poke my Googler friends.

One thing that I will not do is switch my Accept-Language to English. Whenever I can, I don’t just want to see the website correctly, but to try to help my neighbor: see the possible problems that can affect other users who use different language. Somebody has to break the Software Localization Paradox.


Filed under: Firefox, Free Software, localization

Will Kahn-GreeneProposal: LDAP password resets as a unit of measure

Backstory

Every 3 months, we at Mozilla have to reset our LDAP passwords. The system helpfully sends the first reminder 2 weeks before your password expires, then the second reminder 1 week before your password expires and the last reminder 2 days before your password expires.

Sometimes time passes by faster than you know and you end up with a Locked out of LDAP account.

The 3 month LDAP password reset is such a large part of our lives that I propose it become a standard unit of measure for elapsed time.

Usage

Used in casual conversation:

Pat: Hi!

Jordan: Hi!

Pat: I haven't seen you before. How long have you been at Mozilla?

Jordan: I've been here for 6 LDAP password resets.

Pat: Oh, weird. I've been here for 7. Good to meet you! Would you like a banana?

Jordan: Would I ever!

Used in casual conversation on IRC:

<patbot> anyone use less?
<corycory> i only use sass. it's the best.
* riledupriley has quit (Quit: riledupriley)
<patbot> :(
<hugbot> (patbot)
* r1cky has joined #casualconversationexample
<r1cky> morning!
<nigelb> r1cky: hai!
<nigelb> Ah, it's nearly mfbt.
<mtjordan> sure. been using it for 3 ldap password resets.
<mtjordan> patbot: why do you ask?

Used in Bugzilla comments:

Jordan [:jordan]  1 day ago       Comment 0 [reply] [-]

Readonly mode causes the site to ISE.
Pat [:pat]  1 day ago             Comment 1 [reply] [-]

I looked into it. Turns out we haven't used readonly
mode in at least 4 LDAP password resets.

I think we just need to add a fake authentication
module. Easy peasy.

Used when joining a new group:

From: Pat
To: some-group@mozilla.org
Subject: Welcome Jordan to some-group!

Hi all!

I'd like to welcome Jordan to some-group! Jordan brings
expertise that is invaluable. I'm excited! Yay!

Jordan: Tell us about yourself!

Pat
From: Jordan
To: some-group@mozilla.org
Subject: Re: Welcome Jordan to some-group!

Hi!

I'm excited to join some-group! Hopefully I bring something
useful to the table.

I've been at Mozilla for 7 LDAP password resets, I like
top-posting and I make a mean cold brew coffee.

Looking forward to my first meeting!

Jordan


On Blah blah blah at blah blah blah, Pat wrote:
> Hi all!
>
> I'd like to welcome Jordan to some-group! Jordan brings
> expertise that is invaluable. I'm excited! Yay!
>
> Jordan: Tell us about yourself!
>
> Pat

Used in an email to everyone@ about departing:

Dear everyone!

It is with sadness that I tell you I'm leaving as of next
Friday. As you know, I've been with Mozilla for 32 LDAP
password resets and frankly, I'm totally out of usable
Sherlock Holmes story titles, so I'm off to new challenges.

I will miss you all.

[Comments]

Tantek Çelik #UX: "Learn more" Links in Warning Boxes Should Go To A Page With These Three Things

Sometimes web pages display brief warning boxes at the top with "learn more" links. The learn more link in a specific warning box should go to a page specifically about that warning with, in rough order:

  1. screenshot of warning box
  2. quoted full text of the warning (for searchability / search engine discovery)
  3. detailed text answering:
    • how could have the issue occurred?
    • what should the user do to resolve the issue?
    • how can the user avoid the issue in the future?

E.g. the "Learn more ›" link in the yellow warning box in this screenshot:

cropped screenshot of a tweet permalink page with a yellow warning box at the top

links to: https://support.twitter.com/articles/82050-i-m-having-trouble-confirming-my-email which:

  • Neither has screenshot nor text of warning
  • Covers several topics unrelated to the warning
  • Does not answer the above questions

And could be improved by linking to a specific page about this particular warning, containing the above points 1-3, and answering all three questions in point 3.

Related: Scary Twitter warning: "... removed the email address from your account...

J. Paul ReedEulogy for a Founding Father, revisited

In response to my post earlier this week on Tinderbox’s end-of-life, reader Carsten Mattner asked:

Reading [your post], I couldn’t figure out what replaced Tinderbox for the Mozilla builds. What feeds tbpl? Does Mozilla not use Tinderbox to build continuously?

When I left Mozilla in 2007, there was a Release Engineering project in progress to actively replace Tinderbox (Client) with buildbot. So in short, no, Mozilla does not use Tinderbox Client to drive its continuous integration builds, and hasn’t for some time.

Do they still use buildbot today?

I didn’t know the answer to that question, so I tracked down Coop on IRC, who graciously gave me a few minutes of his time to answer exactly that.

He said:

  • Mozilla currently uses “95% buildbot, with 5% Jenkins for random small projects”
  • There are multiple buildbot masters that drive the buildbot clients
  • Unlike the out-of-the-box buildbot master setup, the masters query a job scheduling database instead of monitoring source control for changes themselves; they then report their results to a database, which tbpl (and other services) use to generate their reports/dashboards; the buildbot master waterfall pages aren’t accessible to the external world (which makes sense, because they include unsecured administrative functionality1)
  • There are about 60 masters right now, but Coop said “number keeps growing though, so we need to rethink the whole solution”

So there’s your answer, Carsten!

_______________
1 A long standing criticism of mine, among others

Selena DeckelmannMigrations with Alembic: a lightspeed tour

I’ve got a Beer & Tell to give about alembic. Alembic is a migration tool that works with SQLAlchemy. I’m using it for database migrations with PostgreSQL.

So, here’s what I want to say today:

The most difficult thing to deal with so far are the many User Defined Functions that we use in Socorro. This isn’t something that any migration tools I tested deal well with.

Happy to answer questions! And I’ll see about making a longer talk about this transition soon.

Matt ThompsonUsing Bugzilla for Webmaker

cross-posted from the Webmaker blog

We use Bugzilla to work open and get stuff done

Webmaker, like many Mozilla projects, uses an issue tracker called Bugzilla for filing tickets and getting stuff done. These two new pages provide tips and tricks for filing bugs, and for getting the most out of Bugzilla:

  1. Bugzilla for Webmakerthe best place to start. How to file a Webmaker bug, plus simple tweaks for making Bugzilla easier to use.
  2. Bugzilla for Webmaker: PRO TIPS for digging deeper. How to make it easier for users to file tickets, tagging, searching and tracking bugs, Frequently Asked Questions and more.

Anyone can create, comment or contribute to a ticket

We work open. Webmaker is an open source, non-profit project powered by a global community of friendly humans like you. Anyone can create a ticket, comment on a ticket, and contribute. Just because it’s called a “bug” doesn’t necessarily mean there’s something wrong. It could just be a to-do, or a suggestion. All your tickets are welcome — don’t worry if you’re doing it right. We’re a friendly community, and we want your ideas!

Lawrence MandelMay Open Web Open Mic Toronto: Call for Presenters

owomapr2013

What: Mozilla Open Web Open Mic
When: May 22, 6-9pm ET.
Where: Mozilla Toronto, 366 Adelaide St. W, 5th Floor

Mozilla Open Web Open Mic (OWOM) is back for May. Put on by the Mozilla Toronto community, OWOM features a science share exploration and lightening talks (5 min). April’s event had a great turnout (50-60 people) and included a variety of content from WEBVTT, to MakerFaire, to my talk Building for Mobile Web Compatibility.

The call is currently open for science share participants and speakers. Sign up at
https://mozillianstoronto.etherpad.mozilla.org/owom-may-2013

Interested in attending? Sign up now at
http://owommay2013.eventbrite.ca/#


Tagged: mozilla, mozilla community, toronto

David AscherYou are more than your job title

In grad school, I remember a conversation across the campus green with an visiting psychologist from Harvard.  I don’t remember much about the conversation except that he introduced me to Isaiah Berlin’s notion of the Hedgehog and the Fox, and correctly pegged me as a Fox.  I think I was a bit offended at the simplification, but time has proven him right.  I’m certainly no hedgehog.

I got into a silly argument on twitter last night, about whether my looking to hire someone who I labeled (as job descriptions make us do) a “Coding Designer” was not just foolish (I’d seen the Unicorn references in my tweetstream already) but apparently a bad idea, because, so the ultra-simplified argument goes, you somehow can’t be both.  And so I’ll use the energy to rant a bit about what seem to be prevailing attitudes around titleism and narrow definitions of “professionalism”.

We all need to define ourselves to others. It helps us be understood, and hopefully valued.  Labels can be useful for that. We also, even more, like to label others.  It helps us simplify our approach to them.  If I can find a label for you, then I can rely on a prioris about how people with that label tend to think and behave, and I don’t need to actually get to know you too much.  The more people we interact with, the more important these shortcuts are.  Some roles are particularly subject to that (Recruiters, VCs, politicians, etc. — people who routinely talk to dozens if not hundreds of people a day).  And the best at these roles are those who use a different labeling system than their peers.  Recruiters who see the latent ambition or genius in a shy candidate; VCs who see the determination behind a stutter, or, conversely, the lack of self-confidence behind the bravado, etc.

Labels are useful and practical in the short term.  And I don’t know how one could run a large HR department without them.  But we should be careful to not take them too seriously, as in the long term, they can hurt. They hurt because people, especially interesting, worth-getting-to-know people, are much more subtle, complicated, confusing and hard to categorize creatures.  Whether you take the label too seriously when thinking about others (e.g., refuse to see the valid opinion about a design expressed by a non-Designer) or about yourself (and limit your impact on the world because “oh, that’s not something that a mere ____ like me could say/do”), you’re not getting the most out of anyone involved.

As I write this, I realize that I feel quite strongly about this topic.  Part of it is probably because I grew up in an educational system, which at least then believed way too much in labeling people and determining their fate based on that label.  Much waste ensued.  Part of it is probably because I can’t for the life of me figure out what my label should be, and if I can’t, then that must be bad. I’ve had a range of professional labels, from scientist to engineer, architect, team lead, vice president, CTO, CEO, blah blah blah.  I’ve been called a designer, strategist, entrepreneur, boss, blah blah blah. None of those words will, I hope, be in my epitaph.  And so I get cranky on twitter at night, because if there are people who strive to be both excellent at design and at coding, then by golly we should encourage them.

Titles are a poor approximation of a professional ideal, and a profession is a poor approximation of a human’s breadth, contributions, and talents.  Embrace your inner fox, and if you happen to have both design and coding skills, can see a problem, conjure up a solution, prototype it, welcome challenges to your idea from peers, data, and users, apply.

Michelle ThorneWebmaker Train the Trainer

Back in March, we kicked off the first in hopefully a series of train-the-trainer (TTT) events for webmaking.

The idea is to run events that train people who go on to train others how to teach the web. We focused on practicing an open and participatory ethos, adapting lesson plans, and facilitating events.

This is a post to share what we did and encourage people in designing their own train-the-trainer events.

How to run a Webmaker Train the Trainer

Our prototype, the Reps Training Days, ran for four days in Athens, Greece with 40 Reps from around the world. The agenda was based on Laura Hilliger’s research and insights on successful TTT program and on Allen Gunn’s participatory event methodology. It was made possible by the amazing Mozilla Greek community.

Our participants were Mozilla Reps, a fantastic ambassador program with some of the most active and thoughtful Mozillians. Reps have been early adopters and innovators with Webmaker. They organized nearly 50 events during last year’s Summer Code Party and are leading the way in developing tools, tutorials, and localization for Webmaker. It seemed like a natural fit to run our first TTT with them.

1. Participate in a Webmaker event

The first day of Training Days was spent observing and participating in a Hive Pop-Up, organized by Hive Athens. This was an opportunity for the participants to experience a webmaker event firsthand, to see the tools and activities in action, to learn about the logistics, and to understand the vibe.

We then circled up to discuss what we saw. Participants shared their reflections on what worked well at the pop-up and what they would change if they did their own.

2. Build the training agenda

Then we opened up the training days properly. While we had topics in mind we wanted to hack on together, it was more important that everyone in the room thought about what they want to learn or discuss. So we had an agenda brainstorm.

To do this: we split into groups for 3 people. On post-it notes, we wrote down topics. 1 topic per post-it and the encouragement to write it as concretely as possible.

Then everyone pasted the notes on the wall. We read them all and then clustered them by themes. This collaborative board formed both critical event documentation as well as agenda fodder for the coming days.

3. Teach someone something

To warm up to the idea of teaching, we then got into pairs. The task: teach someone something in 5 minutes.

One person would go and then switch. Even if you knew what was being taught, you were encouraged to play a good learner, asking good questions and prompting the teacher.

After this exercise, we circled up and discussed what we observed from this experience. For many, it was a great way to think about how to explain something clearly, using metaphors and knowledge building blocks. It helped bring people into a teaching mindset.

4. Make a learner profile

Now that we’ve been thinking about teachers and learners, we made small groups and hacked together a learner’s profile.

This goal of this activity was to think about who our learners are. We used Webmaker tools to make these profiles, which was also a fun, maker-y way to be introduced to these tools. Participants were encouraged to think about real people they want to teach.

5. Hack an event invitation

After we’ve made our learner profiles, we thought about the kind of event we wanted to run. Most of the participants have already organized Webmaker events in the past, so there was already some familiarity with the format.

Nevertheless, it was helpful to hack together an event invitation. The idea was to think about your target learner and to make an invitation that would speak to them. Again, we used Webmaker tools to quickly pull these invitations together on the web.

6. Deep dive into lesson plans

With a learner profile, an event invitation and some familiarity with Webmaker tools, we then introduced the hackable kits. These are remixable lesson plans that help mentors, trainers, etc. to teach the web. The idea is that they are adaptable to different contexts and that people can share new ways of teaching in a shared format.

Participants poked around in the kits and asked questions. We also did some fun icebreakers so they could see the activities in action and get some energy going.

7. Playtest lesson plans

Now came the fun part. We had to plan for a real live event the next day. So participants got into groups of five with one group facilitator.

They had to design a four-hour agenda for local youth. Using three recommended activities from the kits, they adapted the lesson plans. Then they walked through a script for the next day, including having people role-play as learners. It was a lot of fun to see and a great way to prepare for the big day.

8. Put training to practice at a live event

So with some nervousness, we got ready for the live event. About a hundred youth were coming. We split into different rooms, each group of five trainers getting about 20 learners.

While there were the inevitable challenges (the internet is down! one kid won’t listen!), the Reps did a terrific job. They rolled with their scripts, adapting them as they saw what was working. They also taught well in smaller pairs with their learners, sometimes adding new challenges or tools to fit their needs.

It was a beautiful and fun thing to see. All the training the days before paid off: the youth had a lot of fun and so did we.

9. Reflect on event, lessons learned and where from here

We ended the event with a closing circle. We talked about what we saw that day, what worked well, what didn’t. We each shared one thing we appreciated about the experience, and what we’re excited about doing next.

With that, we headed out into the city to enjoy the day and the rest of our time together.

10. Go forth and teach!

Each participant left the Training Days with a local plan. It was a short list of possible collaborators in their hometown, a date for a small team huddle to bring those people together, and then a date for a larger Webmaker event to organize with their new collaborators.

We also started interest groups in topics like localization and offline tools. And now, a few months later, the participants from Training Days are now “Webmaker Super Mentors”, mentoring people in an online course to learn how to teach the web.

In the coming months, we hope to keep remixing and improving these agendas, as well as work with people who are interested in TTT in their own cities or communities.

Let us know if you’d like to get involved! #teachtheweb

Jan OdvarkoNew Firebug feature: Use in Command Line

Firebug 1.12 alpha 6 introduces one new feature called simply: Use in Command Line

This feature allows referring various page objects (HTML elements, JS objects, network requests, cookies, etc.) from within Firebug Command Line.
The user can also use object's properties in JS expressions.

See issue 6422 for more details.

This post explains the feature in detail and also asks for feedback.

How it works

As depicted on the previous screenshot the feature is represented by a new context menu item called Use in Command Line. This item is available for various objects (in various panels).

The action is simple: an expression $p is inserted into the Command Line if you execute the command. It's a variable that refers to the clicked object. You can also use the variable to access any property of the object.

What happens on the screenshot:

  • The DOM panel shows an object person implemented on the page.
  • The user clicks on the person's value (the green right hand part) and executes Use in Command Line.
  • Command Line popup is opened within the DOM panel (so, the user doesn't have to leave the DOM panel).
  • $p expression is inserted into the Command Line (and the command line automatically gets focus).
  • The user presses the Enter key to evaluate the expression.

Note that the person object implemented on the page looks as follows:

var person = {
    name: "Bob",
    age: 38
}

If you want to access properties of an object just use dot notation. You can do following in our example.

$p.name

See, even the auto-completion works as expected!

Supported Objects

The feature is implemented for various objects across all Firebug panels. Here is a list of those objects.

  • Any JS object (even functions etc.)
  • HTML elements
  • CSS rules and properties
  • Cookies
  • Network requests

Call for Feedback

All objects accessible through the $p variable are coming from the page content except of network requests and cookies.

We are actually logging nsIHttpChannel for network requests and nsICookie for cookies. So, the object comes from chrome (privileged scope) and isn't accessible in the page content. This way, the user has more info, but it could be confusing since the user could expect that the data are actually accessible from the page content (and Firebug is for page content debugging).

Possible options:

  • Do not implement the feature for network requests and cookies
  • Implement the feature only for XHRs (and show the real XMLHttpRequest instance) and cookie (coming from document.cookie)
  • Keep it as it is
  • Anything else?

What do you think?


Also, would you like to refer more objects at the same time? Something like having more variables (a history) available: $p0, $p1, $p2, ...

How would you expect it to be working?


Thanks for any feedback!

ArkyMozilla Localization Makes a Positive Social Impact

Mozilla brings power of the web into ordinary people's hands. Every day I spend countless hours working with volunteer communities around the world to translate Firefox web browser. Reading Sudheesh Singanamalla's blog post about his encounter with a farmer in rural India was such a touching experience.

A Localization journey - A Farmer's tale - A Delightful Experience

It was on my way back in a cramped out bus, travelling researching about language changes and variations within the state of Andhra Pradesh, that I sat next to a man, quite old.

Sudheesh and the old man

Me : What do you use in the internet? How do you talk to your son?
Old man: I go to Rajat's Net Cafe nearby by house and then talk from there on Google (meant Google+)
Me : Do you know how to read English and understand which button to click and so on?
Old man : Oh, i don't know English, but i use it in Telugu. The shop guy Rajat has seen me since he was small, so after my son went to Delhi, he separately bought a Telugu keyboard so that i can be using the keyboard.
Me: Okay, but then how do you read the information on the computer screen? Isn't that in English?
Old man : (Laughs) Don't you know, there is this software something called Firefox, it is in Telugu.
Me : Really? Can you tell me how the software looks?
Old man : You should know more, you're an engineering student but if you ask i'll tell you, its a small thing like this earth picture but a small cat , orange in colour is holding it.
Me: (smiling crazily) You know how to use it in Telugu?
Old man : Yeah, its not hard, I know how to read Telugu and also know how to use mouse, so clicking gets me the job done.
 

Anton KovalyovGoogle Mobile Summit

Yesterday, Google organized an invite-only summit for mobile web and tools developers. As an engineer working on Firefox Developer Tools, I was invited to be on a panel about mobile testing and tooling. This is my attempt to write down my answers in hope they’ll be useful for someone out there. I suggest reading them as separate, individual mini essays.

Read it on Medium

Lucas RochaIntroducing The Layout

As engineers, I believe the way we approach a problem is as important as the code we write. This is especially relevant in the context of UI engineering where design is such a vital element.

Unfortunately, it seems quite hard to find good content about everything that happens around us and inside our heads when we are building user interfaces. This is what The Layout is about.

My intent is to create a space for high quality content discussing the principles, mindset, and practices that I believe shape the craft of UI engineering. It is meant to be a shared space with many voices—so, expect some awesome guest authors.

I’ve just posted the very first article, Mind the Gap. My plan is to publish a new article every other week-ish. For now, subscribe to the RSS feed or simply follow The Layout on Twitter or Google+ to get future updates.

I really hope you enjoy it!

Christian HeilmannGiving “image swivel” the vanilla web diet treatment

With my chapter for the upcoming Smashing Book 4 done and talking about “The Vanilla Web Diet” and some workshops in the making I thought it is a good idea to show in an example how to make a seemingly good solution better by rethinking it with the principles of the vanilla web diet in mind.

The solution I am targeting is the image swivel effect that was posted a few days ago on CSS tricks. I sent my solution to the author and he was very interested in getting more information as to the why. The effect as it stands works and does the job to inspire people to play with it. The comments showed that where lots of developers had a go at creating their own solutions fixing some of the issues I am about to cover here. My favourite probably is Eduardo García Sanz’s Pure CSS solution which to me is a good plan to achieve an effect like that if you don’t need touch support or keyboard access.


Before I get accused of “hating” – the code is OK as a demo to get creative juices flowing but it has many issues that would show up when used in production. It is also mouse-dependent and doesn’t work on touch devices. Therefore I’ll start by stating what I find troublesome about the published solution and then explain how to approach the problem to make it as simple and maintainable as possible. I also add support for mouse, keyboard and touch access and try to achieve the effect whilst not blocking people out.

If you prefer to see this as a screencast, here is one I recorded explaining the ideas, the issues I found with the original code and how my solution works. This is live and unscripted.

How much markup do we need?

Let’s start by looking at the solution that is shown in the CSS tricks article.
The HTML is the following:

<div id="faces">
  <div id="face-area">
    <div id="image-1" style="display: none;">
      <img src="/images/look-left-3.jpg">
    </div>
    <div id="image-2" style="display: none;">
      <img src="/images/look-left-2.jpg">
    </div>
    <div id="image-3" style="display: none;">
      <img src="/images/look-left-1.jpg">
    </div>
    <div id="image-4" style="display: none;">
      <img src="/images/look-center.jpg">
    </div>
    <div id="image-5" style="display: none;">
      <img src="/images/look-right-1.jpg">
    </div>
    <div id="image-6" style="display: none;">
      <img src="/images/look-right-2.jpg">
    </div>
    <div id="image-7" style="display: none;">
      <img src="/images/look-right-3.jpg">
    </div>
    <div id="the_faces_overlay">
      <div class="the_faces" data-number="1"></div>
      <div class="the_faces" data-number="2"> </div>
      <div class="the_faces" data-number="3"></div>
      <div class="the_faces" data-number="4"></div>
      <div class="the_faces" data-number="5"></div>
      <div class="the_faces" data-number="6"></div>
      <div class="the_faces" data-number="7"></div>
    </div> 
  </div><!-- END #face-area -->
</div> <!-- END #faces -->

Warning sign #1: Repeated HTML structures without logical connections

This is a very common mistake I see when starting an effect like that. We create something that contains the content and then something that gets acted upon to show the effect. Thus we lose the benefit of already interacting with the parent element and using event handling to find out where we are. I guess historically this is based on “CSS only solutions” that needed that kind of separation as you could not calculate or detect mouse position in CSS.

As we create two separate sets of markup for an effect we need to find a way to connect the element that was interacted with to the one we want to show. This means adding IDs to all the elements, classes to all the elements we interact with and a data attribute to tell which of the images to show. This is bad for maintainability. If we add an image, we need to also add an element to interact with. Great code is catered to making maintenance as easy as possible. Here we have a lot of dependencies to deal with when adding or removing an image.

Warning sign #2: Lack of semantic markup


The other gripe I have with this HTML is that it means nothing at all. As the order of the images is important, the right HTML construct to use here is an ordered list.

Warning sign #3: Lack of alternative content


Another big HTML mistake is adding images without an alt attribute. This means that screenreader users would get the file path of the images read out to them. Either provide a sensible alternative text or add a alt=”” to hide the image from screenreaders.

Warning sign #4: Dependency on the number of elements


One big warning sign to me here is that our effect is dependent on the number of images in the widget and that we need to have the data attributes and the IDs maintained together although they are on different elements.

The more elements we add means the more IDs we need to maintain. This is not what coding is about. Computers are good at calculating things for us.

Warning sign #5: Empty elements and inline styles

Whenever I see inline styles I know something went wrong. There is no point in them and if ever they should only be generated by code, not by humans. The same with empty HTML elements: you probably did some extra work that is not needed. HTML is there to contain content or provide interaction. If you have a lot of empty DIVs without an obvious templating use case, something went wrong.

Giving no power to CSS

The CSS of the solution is not much, and it doesn’t do much either. This is a shame seeing how much easier it is for a visual maintainer to change CSS rather than changing JavaScript.

body {
  background: #333 
}
#faces {
  height: 333px;
  width: 500px;
  margin: 0 auto;
  border: 8px solid white;
}
#face-area {
  height: 500px;
  width: 333px;
  position: relative;
}
#the_faces_overlay {
  position: absolute;
  width: 500px;
  top: 0;
  left: 0;
}
#faces .the_faces {
  height: 333px;
  width: 14.2857143%;
  float: left;
  margin: 0;
  padding: 0;
}

Warning sign #6: CSS dependent on the amount of elements

The glaring issue here is the “width: 14.2857143%;” which is calculated by dividing 100% into seven parts. This means that if you delete an image from the HTML, you also need to change the CSS width here. You should never be dependent on the amount of elements in your CSS, as those are prone to change. In this case especially there is no logical way to find out why this is the width. CSS calc() can at least make that obvious but in general it is a bad idea to create look and feel that is tied to a certain amount of elements.

Goldfish jQuery

The jQuery code to make the effect work is very short:

// Reveal the "center" image
var centerImage = $("#image-4").show();
// Bind hovers to each column
$(".the_faces").each(function() {
  $(this).on("mouseover", function() {
    $("#image-" + $(this).attr("data-number")).show();
  }).on("mouseout",function() {
    $("#image-" + $(this).attr("data-number")).hide();
  });
});
// Reset center image
$("#face-area").on("mouseleave", function() {
  centerImage.show();
}).on("mouseenter", function() {
  centerImage.hide();
});

It is, however, very demanding to the browser. Slowing down a browser can be done in many ways – the most damaging ones are heavy computation, accessing the DOM and lots and lots of event handling. The latter two is what we do here.

Warning sign #7: lack of data caching

We loop over all the elements with the class “.the_faces” and add a mouseover and mouseout handler to each of them. Every time these get fired, we read an attribute, create a string with it and access an element that has the ID of the string and show or hide it. Showing and hiding using the jQuery methods is another access to the DOM as it manipulates the display style property. We show and hide the “center image” upfront and also on another event hander on the overall parent element.

If we were to add touch and keyboard handlers we’d triple the amount of assigned event handlers as we apply them on each image.

I call this Goldfish code – we keep asking the browser for things we should already know. The widget interface we have here is static HTML - there is no loading of content, no changes in it. Therefore there is no point in continuously reading out what the data-number attribute is and ask the browser to find the element with a certain ID. Caching results is a very simple thing to do and the performance benefits are amazing.

Rethinking the solution the vanilla web diet way

I approached the solution by looking at what we need to do here:

  • We have a widget of a certain size with images in it
  • We have an unknown amount of images – it should be dead easy to remove or add one or replace them all
  • Moving the mouse over the widget should loop through the images, also touching the widget should do so and it would be nice to be able to flip forward and backward with the keyboard
  • Should things not work out it would be nice to have an display that still makes sense

The HTML - we don’t need IDs or data attributes

<a href="/productpage" id="rollover">
<ol>
  <li><img src="pics/IMG_0518.jpg" alt""></li>
  <li><img src="pics/IMG_0517.jpg" alt=""></li>
  <li><img src="pics/IMG_0516.jpg" alt=""></li>
  <li><img class="current" src="pics/IMG_0515.jpg" alt=""></li>
  <li><img src="pics/IMG_0519.jpg" alt=""></li>
  <li><img src="pics/IMG_0520.jpg" alt=""></li>
  <li><img src="pics/IMG_0521.jpg" alt=""></li>
</ol>
</a>

We should link this to something – after all a beautiful effect like that should react in some sale or deep-dive, right? Good thing is that in HTML5 links can contain other elements. So all we add is a link. This automatically gives us keyboard access to the widget – something otherwise we’d have to create by using roaming tabIndex.

As the order of the images is important, an OL is the right element to use. Each image has an empty alt attribute to ensure there is no hassle with screenreaders. Instead of defining which image to show as the first one in our JavaScript we keep this maintained in HTML, too, by adding a class of “current” to the image.

Showing and hiding with CSS

body {
  font-family: arial, sans-serif;
}
#rollover.js {
  display: block;
  margin: 2em;
  z-index: 3;
  position: relative;
  height: 270px;
  width: 200px;
  cursor: none;
}
#rollover.js img {
  width: 100%;
  position: absolute;
  top: 0;
  left: 0;
  visibility: hidden;
}
#rollover.js img.current {
  visibility: visible;
}

As our functionality is dependent on JavaScript, our styling should be, too. This we can achieve by adding a “js” class to the element when JavaScript is available and only apply the styles when needed. This CSS gives the widget a fixed size and positions all the images stacked inside it.

Instead of doing a hide() and show() in JavaScript, all we need to do is to apply a class of “current” to the element we need to show. All the hiding and showing is thus done in CSS which means that in the future we’d want to do other visual things with the “shown” and “hidden” images, all we need to change is the CSS.

(function(){
 
  if (document.querySelector) {
 
    var rollover = document.querySelector('#rollover');
    rollover.className = 'js';
 
    var images = rollover.querySelectorAll('img');
    var all = images.length;
    var width = rollover.offsetWidth;
    var ox = rollover.offsetLeft;
    var boundarywidth = width / all;
    var current = 0;
    var x = 0;
    var index = 0;
    var touched = false;
 
    var setcurrent = function(index) {
      if (images[index]) {
        images[current].className = '';
        images[index].className = 'current';
        current = index;
      }
    };
 
    var findindex = function(x) {
      index = parseInt((x - ox) / boundarywidth, 10);
      if (index !== current) {
        setcurrent(index);
      }
    };
 
    rollover.addEventListener('mousemove', function(ev) {
      if (!touched) {
        findindex(ev.clientX);
      }
    }, false);
 
    rollover.addEventListener('touchstart', function(ev) {
      touched = true;
    }, false);
 
    rollover.addEventListener('touchend', function(ev) {
      touched = false;
    }, false);
 
    rollover.addEventListener('touchmove', function(ev) {
      if (touched) {
        findindex(ev.changedTouches[0].clientX);
        ev.preventDefault();
      }
    }, false);
 
    rollover.addEventListener('keydown', function(ev) {
      var key = ev.char || ev.key || ev.which;
      if (key === 37) { index = index - 1;}
      if (key === 39) { index = index + 1;}
      if (index < 0) {index = 0;}
      if (index > all - 1) {index = all - 1;}
      setcurrent(index);
    }, false);
 
    if (rollover.querySelector('.current')) {
      for (var i = 0; i < all; i++) {
        if (images[i].className === 'current') {
          current = i;
          break;
        }
      }
    } else {
      setcurrent(current);
    }
  }
})();

Quite longer than the jQuery, but bear with me as this does a lot more and in a much less demanding fashion. We wrap our code in a closure to makes sure we don’t leave any nasty globals behind. That should always be the first step.

Then we test if the browser supports document.querySelector. This is the standard answer to jQuery’s $() and is supported by lots and lots of great browsers. It is not supported by old and outdated browsers, which is why it is a good idea to test for it. This means that old Internet Explorer versions will not get the effect but instead they get the images as a numbered list (as we made the CSS dependent on a class applied with JavaScript). This is good, as it means we don’t need to test on these old browsers, which is hard to do and frankly a waste of our time.

We get a reference to our rollover widget using document.querySelector() and add the “js” class to it. This hides all the images and sets up the look and feel of the widget – all maintained in CSS. No need to loop through a lot of elements or use inline styles to hide them.

Next we get references to all the DOM elements we need and calculate what we need to find out what to show when the mouse cursor or the finger is on a certain part of the widget.

First we get all the images and store them in images. This will not change while the page is open, and querySelectorAll() gives us references to all of them. We store the amount of images in all to compare against later on.

Next we do the thing dynamically that the original solution did by hand – calculate the width of the different strips of the interaction that shows and hides images. We do this by reading out the offsetWidth of the widget as it is defined in CSS, so we don’t know and don’t want to hard-wire any widths in our JavaScript. We find out how far left the widget is in the browser by reading the offsetLeft property and store that in ox. Then we calculate the width of the interaction boundaries by dividing the width of the widget by the amount of images and store that in boundarywidth.

This makes the widget flexible to change any time the CSS changes or the amount of images does. If you remove or add an image, the width of the boundaries is calculated newly. No need to change the CSS to reflect that width. We made this now maintainable simply by adding or removing list items.

We define the current shown image index as 0, preset an x variable as 0, define the current index of the detected movement as 0 and set touched to false.

The index will be the index of the image to be shown at a certain point in the interaction. The index of the currently shown one is be stored in current, x will be detected position of the mouse or finger on the screen and touched defines if the screen is currently being touched or not.

The setcurrent() function hides the last shown image and shows a new one by shifting the “current” class from one to the other. It then stores the new image index in current. This is a very simple way to show a new state in a collection of things that can only show one at a time. No need to ask the browser which one is visible when we can store this in a variable like current.

The findindex() function converts the detected horizontal position of the mouse cursor or the finger of the user into an index of our image array. All you need to do is to subtract the left position of the widget itself and divide the value by the boundarywidth. Convert it to an integer and compare it to the current index and if it differs, call setcurrent().

All that is left is to assign the event handlers to make the magic happen. The first is a mousemove listener on the widget that calls findindex() when no touch happened. The current horizontal mouse position is stored in clientX of the mousemove event.

Touch interaction needs to be initiated (at least in Chrome in my testing here) so we set the Boolean of touched to true when a touchstart happened and to false when touchend was detected.

When a user moves a finger over the widget the browser fires the touchmove event and the current horizontal position is stored in the clientX property of the changedTouches array. We only detect the first finger in this case.

Keyboard detection doesn’t give us a position on the screen, so all we do is manipulate the image index directly. We listen for a keydown event and check the code of the key that was pressed. If it is the left arrow we subtract one from the current index and in the case of the right arrow we add one to it. We ensure that the index stays in the allowed limits and call setcurrent().

The last thing to do is to show the current image. If there is one with the right class in the HTML we need to find out its index and we do that by looping over them until we find the right one. If there isn’t any we just apply the current class to the first image (as defined at the start of the script).

Many solutions to the same idea

I hope this gives you an idea of how to approach an effect like that when you want to put it in production. There are of course other ways of doing it, but I wanted to ensure a few things that get very often forgotten:

  • The whole effect is now generated from the HTML, so all you need to do to create a new swivel is add other images
  • The whole look and feel is defined in CSS and you can resize the widget without having to worry about the size of the different boundaries
  • It can be used with a keyboard, the mouse or on touch devices
  • The DOM interaction is kept to an absolute minimum which means the performance on low spec devices is much, much better
  • There is no jQuery dependency

Robert KaiserPreserving Software - Feedback Requested!

As Digital Preservation is part of the agenda of the US Library of Congress, they're doing a workshop on Software Preservation next week, and Mozilla was invited as an expert group. Otto de Voogd and myself are in the delegation going there (I'll be roughly in the Washington, DC, area from Saturday until June 2) for Mozilla - and the text below is a guest post by Otto with questions that we would like some feedback on so we can represent the Mozilla community as well as possible:




On the 20th and 21st of May the Library of Congress holds a workshop on the topic of preserving software.
Otto de Voogd and Robert Kaiser will be representing Mozilla, putting forward our viewpoint as custodians of a codebase with a significant heritage and importance.

Many questions and thoughts arise. Here's an overview of ours; we look forward to feedback.


- Should archivists keep source codes or executables or both?

Executables and source code are both valuable. Executables are valuable because the source code is sometimes not available, or perhaps the build tools are not, and setting up a build environment for older code can be a difficult and complex thing.

Source is valuable to determine how a program works. It also makes it possible to reuse code and algorithms, especially, but not only, in the case of open source software.


- Preserving documentation.

Preserving documentation that goes with software, seems logical.
Would this need to go as far as preserving discussion threads and entries in bug trackers?


- Preserving environments/platforms.

It seems obvious that without preserving an environment in which the software can run, it is going to be impossible to experience the software.
Preserving such an environment should therefor be part of the software preservation effort.

To avoid the physical constraints imposed by preserving old hardware (which would be a preservation effort in its own right), a solution would be to build virtual machines and emulators.
As hardware capacity constantly grows, running virtual versions of older hardware should generally be feasible.

To fully recreate an environment we'd also need to preserve the operating systems and other software tools that the preserved software needs to run.
Those being software themselves would logical already be included in any software preservation effort.

Preserving documentation concerning environments, would also be required.
To build virtual machines and emulators it would be helpful for hardware makers to make technical specifications available. One could envision this to become a legal requirement at least for older hardware.

Can we imagine a world where web based emulators would allow an online digital library to serve users worldwide? Users who would be able to run old software in emulators running in their browsers...


- Is everything worth preserving, if not how does one go about selecting what is worth preserving?

Does one need to preserve every version of software, just the last version or all major releases? What about preserving software that has not spread widely. Would there be some threshold, or some other criteria?


- How does one index software and search the library?

There will be a need to gather meta data about software and the preservation of documentation as we already mentioned. This meta data and documentation could serve to populate an index enabling for instance the search for particular features.


- Can software preservation help in making code reusable?

If there are good ways to actually find relevant and useful code, this could lead to more reuse not only of actual code, but also of algorithms and concepts.
It may also become a valuable source for students who wish to learn about actual implementations of software solutions.

At the very least a minimum of meta data, such publication dates, copyright owners and licenses should be available to determine how certain code can be reused.
In particular for open source software we believe that software libraries should strive make it available without restrictions.


- Preserving data formats.

The software preservation effort should also include an effort to preserve data formats. Including technical descriptions of those formats and the tools to read, write and edit those formats.


- Can software preservation help in the discovery of prior art?

We believe it can, and as such preserving old code could be a great tool in preventing the repatenting of existing software concepts.

Of course we believe that software patents shouldn't exist in the first place, as software is already covered by copyrights, but at the very least prior art is a good avenue to prevent some of the worst abuse of software patents.


- How do copyrights affect software libraries?

A lot of software is licensed to be used on a particular piece of hardware or only available via subscription. How does this affect software libraries? Should there be exceptions like there are for traditional libraries?

In the life cycle of software, the commercially exploitable time is limited, likely anything older than 10 years no longer has any commercial value.
Maybe copyrights on software should be significantly reduced to something like 10 years, which is more than enough to cover the commercially exploitable timeframe of the software life cycle.

Such a limit would greatly enhance the work of software libraries, increasing availability and ease of access as well as removing a lot of the red tape involving requests for permission to keep copies.


- What about software as a service?

And what about software as a service, where neither the source code nor the executables are ever published? How can something like Gmail be preserved, when neither the service's code nor the environment is available to the public?


- Preserving "illegal" or cracked copies?

What if a copy of a piece of software comes from an illegal source? A cracked version with modifications maybe? They have value in themselves as they are a cultural expression.

What if such an illegal copy is the only copy still available? Would it make sense to preserve that too?

Rob Campbell#io13

IO13

I sat in on three sessions at Google IO 2013 yesterday.

Memory Lane with Chrome Devtools and GMail was the first.

The presenters showed off their Heap Tracking Profiler and Memory Tracking tool in the Timeline and explained how to use them to track down a leaky DOM node. It was a practical application of how to use a developer tool to solve a particular problem.

One interesting takeaway that surprised the presenters during their research: Always allocating more memory (caching) as a way to improve performance in a large application like Gmail is not a panacea for slow performance. Having a large heap space actually slows down the garbage collector and your performance suffers. It’s a fine balance.

What surprised me was how they analyzed their problem by tracking a user with a known high memory problem for three days. The Google team constantly monitors their apps’ performance via the window.performance API and can single out hotspots in the population.

The next talk I sat in on was about Chrome Apps. The presenter, Erik Kay showed off some of the “Immersive” experiences of Chrome Apps and the different ways they could interact with the hardware on the Chrome Book. The talk included a demo of a small thermal printer being hooked up and controlled over USB which garnered some applause.

The Chrome Web Store lets you buy apps for Chrome.

The only real mention of Android was that they were using PhoneGap and Cordova to provide their compatibility layer. Same for IOS. There will be compatibility issues with deploying on iOS but it seems surprising that they would pursue this completely separate technology for Android. Surely they could ship a full version of the Chrome Runtime and deal with hardware incompatibilities directly.

The questions from the room were interesting. One man (not a Mozillian) asked about WebRTC compatibility across the different platforms, pointedly repeating the question of whether or not he’d be able to use WebRTC in an app on iOS. Only when their WebView supports it.

Another man asked something about interoperability between B2G and ChromeRT. Erik said that there is “no forcing function yet to drive standardization”.

I think my biggest takeaway from this talk was that people wearing Google Glass look like dorks.

My second biggest takeaway was that I was very surprised that there was zero mention of the Google Play Store for Chrome Apps.

Last talk I attended was a Fireside Chat with the Blink Team. While I was expecting an actual fire and was disappointed there wasn’t one, the team bravely took questions from an audience confused about feature-detection, unprefixed CSS and market fragmentation.

Dan Buchner asked the panel something about standardization and I felt a little badly for the Blink team who had a whole chunk of slides talking about how they’re going to be good citizens. (If you want to participate, you should join blink-dev@chromium.org.)

I was interested in ChromeStatus.com/features which shows a spreadsheet of features in-progress. Time will tell how their Intent to Implement and Intent to Ship broadcasting will work from an Open Source point-of-view, but they are currently claiming that a third of their intents to implement are coming from outside of Google.

I wanted to meet Paul Irish after the talk but Steven Shankland showed up and pushed me out of the way. When he was done I did get to meet him, but I think Buchner had made him angry or something. Maybe he was just tired. I dunno.

Joey ArmstrongTransitioning to moz.build

If you have been wondering about status for the transition to mozbuild as a replacement for makefile infrastructure, a source is becoming available. People involved with the conversion have been meeting bi-weekly on the topic – details and content can … Continue reading

Michael KaplyDisabling Safe Mode (Again)

So lots of people are having trouble disabling safe mode using my earlier instructions and I discovered it is because of problems overlaying the dialog. So here are some new instructions that should work for everyone.

First, create a disablesafemode directory in distribution/bundles where the Firefox executable is located (you'll probable have to create the distribution and bundles directories as well). Then create a file called chrome.manifest that looks like this:

content disablesafemode content/
override chrome://browser/content/safeMode.xul chrome://disablesafemode/content/safeMode.xul

Then create a subdirectory called content. In that directory, create a file called safeMode.xul that looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet href="chrome://global/skin/"?>
<dialog xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul"
        id="safeModeDialog"
        buttons=","
        ondialogcancel="closeFirefox();">
  <script type="application/x-javascript;version=1.7">
  <![CDATA[
  function closeFirefox() {
    Components.utils.import("resource://gre/modules/Services.jsm");
    Services.startup.quit(Services.startup.eForceQuit);
  }
  ]]>
  </script>
  <description>
  Safe Mode has been disabled.
  </description>
</dialog>

This will cause a dialog to be shown instead of the safe mode dialog that when the user closes, will simply close Firefox. If you'd prefer to show no dialog at all, change ondialogcancel to onload.

I've tested this a lot, and it is working for me. Please let me know if you have problems.

NagappanAnnounce: LDTP 3.5 - Linux GUI test automation tool

Highlights:
New API:
* inserttext, objtimeout, guitimeout, getcellsize, getcellvalue,
getobjectnameatcoords, getcombovalue, getaccesskey in Python client
* doubleClick, doubleClickRow, onWindowCreate, getCellSize, getComboValue,
appUnderTest, getAccessKey in Java client
* getcellsize, getcellvalue in Ruby client
* GetCellSize, GetComboValue, AppUnderTest, GetAccessKey, MouseRightClick,
DoubleClick, DoubleClickRow, RightClick in C# client

New control type:
* POPUP MENU for Ubuntu environment

Bugs fixed:
Ruby client:
* Fixed optional arguments to imagecapture
* Check window_name parameter, if empty then use @window_name passed in
constructor

Python client:
* Fixed optional argument APIs to work on both Windows and Linux
* imagecapture x, y offset, height and width parameters are disregarded if
window parameter is provided - Bug#685548
* Return unicode string all the time on gettextvalue
* Fix partial match argument in selectrow, compatible with Windows
* Patch by ebass to support Python 2.6
* Added Errno 101 as we see in ebass Ubuntu 10.04 environment

Core LDTP2
* Include label type on gettextvalue
* Don't include separators in the list

Perl client:
* Added perl client

Credit:
* Sawyer X for the Perl interface
* ebass (IRC nick name)
* Marek Rosa
* Thanks to all others who have reported bugs through forum / email /
in-person / IRC

About LDTP:
Cross Platform GUI Automation tool Linux version is LDTP, Windows version
is Cobra and Mac version is PyATOM.

* Linux version is known to work on GNOME / KDE (QT >= 4.8) / Java Swing /
LibreOffice / Mozilla application on all major Linux distribution.
* Windows version is known to work on application written in .NET / C++ /
Java / QT on Windows XP SP3 / Windows 7 / Windows 8 development version.
* Mac GUI testing is known to work on OS X Snow Leopard/Lion/Mountain Lion.
Where ever PyATOM runs, LDTP should work on it.

Download source / binary (RPM/DEB)

Documentation references: API / JavaDoc

For detailed information on LDTP framework and latest updates visit
http://ldtp.freedesktop.org

Report bugs

To subscribe to LDTP mailing lists

IRC Channel - #ldtp on irc.freenode.net

How can you help: Spread the news and send back your feedback to us

Brendan EichC is for Cookie

Mozilla is engaged in a broad, deep conversation about Internet privacy. We believe in putting users in control of their online experience, and we want a healthy, thriving web ecosystem — we do not see a contradiction. However, sometimes a crucial experiment is required to prove it.

To this end, we are testing a patch from Jonathan Mayer. Jonathan’s patch matches how Safari has worked for years, and does the following:

  • Allows cookies from sites you have already visited.
  • Blocks cookies from sites you have not visited yet.

The idea is that if you have not visited a site (including the one to which you are navigating currently) and it wants to put a cookie on your computer, the site is likely not one you have heard of or have any relationship with. But this is only likely, not always true. Two problems arise:

False positives. For example, say you visit a site named foo.com, which embeds cookie-setting content from a site named foocdn.com. With the patch, Firefox sets cookies from foo.com because you visited it, yet blocks cookies from foocdn.com because you never visited foocdn.com directly, even though there is actually just one company behind both sites.

False negatives. Meanwhile, in the other direction, just because you visit a site once does not mean you are ok with it tracking you all over the Internet on unrelated sites, forever more. Suppose you click on an ad by accident, for example. Or a site you trust directly starts setting third-party cookies you do not want.

Our challenge is to find a way to address these sorts of cases. We are looking for more granularity than deciding automatically and exclusively based upon whether you visit a site or not, although that is often a good place to start the decision process.

We plan to ship an evolution of the patch “on” by default, but we want to make refinements first. To make sure we get this right we need more data. Our next engineering task is to add privacy-preserving code to measure how the patch affects real websites. We will also ask some of our Aurora and Beta users to opt-in to a study with deeper data collection.

There are many conflicting claims about how this patch will affect the Internet. Why debate in theory what we can measure in practice? We are going to find out more and adjust course as needed. This is the essence of the release test cycle.

On Tuesday we did two things:

  1. The patch has progressed to the Beta release channel for Firefox 22, but it is not “on” by default there. This allows more people to test the patch via Firefox’s “preferences” (AKA “options”) user interface, and avoids an abrupt change for site owners while we work on handling the hard cases.
  2. The patch remains in the Aurora channel for Firefox, where it is “on” by default. This gives the patch better ongoing test coverage and facilitates A/B testing.

We have heard important feedback from concerned site owners. We are always committed to user privacy, and remain committed to shipping a version of the patch that is “on” by default. We are mindful that this is an important change; we always knew it would take a little longer than most patches as we put it through its paces.

For those who read this as Mozilla softening our stance on protecting privacy and putting users first, in a word: no. False positives break sites that users intentionally visit. (Fortunately, we haven’t seen too many such problems, but greater testing scale is needed.) False negatives enable tracking where it is not wanted. The patch as-is needs more work.

We look forward to continued dialog with colleagues, contributors, fans, and detractors. We will update all of you within six weeks so you can understand our thinking and how we will proceed. Comments welcome.

/be

P.S. Cookies (name history) were originally intended to be ephemeral (Windows 3.1 had so little usable memory with its 64K memory segments that Netscape’s founders had no choice). At first, they held only session state that could be recovered from the server by logging in again.

(Remind me to tell the story some day of Montulli’s aborted “twinkies” idea from the Netscape 2 era. UPDATE: Lou has published a new blog post about cookies.)

How far we have come in the amazing, living system that is the Web! No one planned for what actually happened, but with more work on the cookie policy in Firefox and (I hope) other browsers, I believe that we can evolve to a better space.

Erik VoldOld School To Jetpack Part 2 - XUL Bashing

In Old School to Jetpack Part 1 I briefly mentioned that xul overlays are not supported with Jetpacks, so today I want to talk about how to convert old school XUL overlays.

A typical overlay would look something like lke this addon-browser-overlay.xul:

<?xml version="1.0"?>
<overlay xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul">
  <toolbar id="addon-bar">
    <textbox id="test-status-bar-message" insertbefore="addonbar-closebutton" value="Hi!"></textbox>
  </toolbar>
</overlay>

This overlay, would then be associated to browser.xul in the chrome.manifest file with an entry like this:

overlay    chrome://browser/content/browser.xul chrome://{add-on name here}/content/addon-browser-overlay.xul

Finally, the overlay would produce this in Firefox’s addon-bar:

Screen Shot

With Jetpacks however the latter step cannot be done, because these overlay entries are ignored for restartless add-ons. One could manually force the overlay with a call to document.loadOverlay, but there is no way to undo this, that is why overlay’s are not supported for restartless add-ons at the moment.

So what is needed here is a module that will add the desired XUL elements to the desired XUL documents.

This is why I wrote a XUL package for Jetpack, it is very raw at the moment, and very simple, so you’ve been warned!

XUL Package for Jetpack

About

The source code can be found on github here.

Pros

  • Quick easy solution for coverting XUL overlays for use with Jetpacks
  • Handles unloads

Cons

  • Only works on Firefox (or other applications made with XUL)
  • At the moment this package only works for adding xul to browser.xul (although it won’t be hard to extend)

Alternatives

Before I go in to this module I want to take a moment to point out that there are many third party packages which handle most of the use cases that overlays for browser.xul support; they don’t cover everything though, like the example above, so this is why I wrote this general purpose package.

Usage

To implement the example above using this module, one would simply need to do the following:

const { XUL, getXULById } = require('xul/browser');
getXULById('addon-bar').insertBefore(XUL('textbox', {
  id: 'test-status-bar-message',
  value: 'Hi!'
}), 'addonbar-closebutton');

And this code is restartless!

Byron Joneshappy bmo push day!

the following changes have been pushed to bugzilla.mozilla.org:

  • [868044] Bugzilla should be automagically aware of which product channel a bug-filer is using.
  • [815531] splinter fails to display attachment 681995 correctly
  • [869077] Component isn’t preselected from the query_string when cloning a bug
  • [850135] hide the textarea custom fields by default with an (edit) link
  • [866447] Make form.doc set up whiteboard for scrumbu.gs
  • [871436] release tracking flag refresh (24)
  • [870907] Project Kickoff Form: javascript responsible for checking for required values not working
  • [828344] “contains all of the words” no longer looks for all words within the same comment or flag
  • [872022] don’t link a review flag to splinter unless the attachment is a patch
  • [841559] Project Kickoff Form: Simplify finance question regrading budget
  • [850920] Project Kickoff Form: Add new question to Legal subsection area
  • [841449] Project Kickoff Form: New Question within Finance Sub Questions
  • [850934] Project Kickoff Form: Make Release Date a Required Field
  • [850932] Project Kickoff Form: Rework Privacy Policy/Project sub questions
  • [826214] New file with one line isn’t shown
  • [797840] Replying to a comment on Splinter always replies to the first comment
  • [821889] Make it so that Splinter shouts loudly when a patch introduces Windows line endings

Filed under: bmo, mozilla

Alexander SurkovAccessible Mozilla: Tech overview of Firefox 22

Firefox 22 reached beta status (it will be released June 24). It's time to list accessibility improvements we made for this version.

ARIA


ARIA role="note" doesn't allow name from subtree (bug) anymore. The bug caused JAWS, for example, to announce role="note" content twice.

HTML


* HTML radio group position doesn't count hidden radio elements (bug). So if the page contains hidden input@type="radio" then a screen reader doesn't take them into account announcing the number of radios.

* HTML input@type="file" changed its hierachy. Now it contains a push button and a label. Be careful if you have dependences on this hierarchy (see bug).

* HTML5 header and footer has changed their mapping according to HTML spec:
footer element that is not a descendant of an article or section element. contentinfo role;

header element that is not a descendant of an article or section element.  banner role.

XUL


XUL label@value element now implements text interface (partially). You can obtain a text between offsets but you can't get it by words for example (refer to bug). XUL label is used wide in Firefox user interface (for example, in Options dialog). Let us know if you have problems with new implementation.

ATK


RELATION_NODE_PARENT_OF has been implemented. It's exposed for aria-owns markup and XUL trees (used in Thunderbird and Firefox bookmarks).

Text interface


As I wrote before we started text interface reimplementation. Firefox 22 got improved getTextAt and getTextBefore offsets at word boundaries. Note, in case of getTextAt we had to mimic WebKit behavior rather than follow the ATK spec to keep Orca working.

Attention. It might be important


* Document load event may be fired a bit later than we used to do that, it will be fired right after all events contained in the queue at the time when document got loaded (see bug).

* IServiceProvider interface is implemented as a tear off (bug).

Dave TownsendFirefox now ships with the add-on SDK

It’s been a long ride but we can finally say it. This week Firefox 21 shipped and it includes the add-on SDK modules.

We took all the Jetpack APIs and we shipped them in Firefox!What does this mean? Well for users it means two important things:

  1. Smaller add-ons. Since they no longer need to ship the APIs themselves add-ons only have to include the unique code that makes them special. That’s something like a 65% file-size saving for the most popular SDK based add-ons, probably more for simpler add-ons.
  2. Add-ons will stay compatible with Firefox for longer. We can evolve the modules in Firefox that add-ons use so that most of the time when changes happen to Firefox the modules seamlessly shift to keep working. There are still some cases where that might be impossible (when a core feature is dropped from Firefox for example) but hopefully those should be rare.

To take advantage of these benefits add-ons have to be repacked with a recent version of the SDK. We’re working on a plan to do that automatically for existing add-ons where possible but developers who want to get the benefits right now can just repack their add-ons themselves using SDK 1.14 and using cfx xpi --strip-sdk, or using the next release of the SDK, 1.15 which will do that by default.

Margaret LeibovicNew Coding Stewards Mailing List

As part of the Grow Mozilla effort, the coding stewards have been working to grow our coding community. Our main focus has been to increase the number of contributions to the core mozilla-central codebase, which includes making it easier for newcomers to get started, as well as keeping existing contributors engaged. Recent work has been focused on things like mentored bugs and recognizing contributors.

We hold open weekly meetings every Wednesday to discuss progress on these goals, but we want to give more developers the opportunity to get involved with this effort, so we created the coding-stewards@mozilla.org mailing list for broader discussion. If you’re interested in growing our coding community, but you might not be interested in another weekly meeting, please join the list!

Michael KaplySetting Default Application Handlers

One of the questions I get asked a lot is how to set default application handlers in Firefox (the Applications page in preferences). Most companies have resorted to creating a default mimeTypes.rdf file and loading this into their default profile. I finally took the time to understand how these handlers work and can give you some code to use.

   var handlerSvc = Components.classes["@mozilla.org/uriloader/handler-service;1"].getService(Components.interfaces.nsIHandlerService)
  // Change "image/tiff" the mime type you want to set the preference for
  var realMIMEInfo = mimeService.getFromTypeAndExtension("image/tiff", "");

  var file = Components.classes["@mozilla.org/file/local;1"].createInstance(Components.interfaces.nsIFile);
  // This should be the path to the .app file on Mac or the EXE on Windows
  file.initWithPath("/Applications/Preview.app");
  var localHandlerApp = Components.classes["@mozilla.org/uriloader/local-handler-app;1"].createInstance(Components.interfaces.nsILocalHandlerApp);
  localHandlerApp.executable = file;
  // The name that will be shown in preferences.
  // Not used on Mac
  localHandlerApp.name = "Preview";
 
  realMIMEInfo.preferredApplicationHandler = localHandlerApp;

  // This says to always use the helper app 
  realMIMEInfo.preferredAction = 2; // useHelperApp
  // This says not to ask
  realMIMEInfo.alwaysAskBeforeHandling = false;

  handlerSvc.store(realMIMEInfo);

You can add this code to your autoconfig file.

Robert O'CallahanTravel

Last week I was in California. It was my first time in the Mozilla SF office --- lovely view of the Bay from the roof. I always enjoy the free snacks and I'm always glad we don't have them in Auckland. I spent quality time with some of the people I know and love at Mozilla, and that's always exciting.

On Wednesday and Thursday I was at Half Moon Bay doing LEAD training. It was fun, but thinking about "soft skills" for two days straight is quite draining for me; my social skills are learned, not innate.

This cohort is different from previous cohorts --- most members are relatively new to Mozilla; of our cohort, Vlad and I have been at Mozilla the longest, by far. This gives me the honor and duty of representing the Mozilla old guard. I feel the power of the narrative that has me in the "crusty old engineer, harping about the old days and resisting change" role ... and I do my best to reject it :-).

One of the results of LEAD so far is that I perceive my relationships with other Mozilla staff to be warmer and stronger than they perceive them, on average. I suspect this may be related to the difficulty of maintaining deep relationships with remote employees I see a few times a year at best. I'm still trying to figure that out.

Plane movies:

  • Gangster Squad: Genre flick. OK.
  • Zero Dark Thirty: Pretty good. Not exactly entertaining, but interesting.
  • Live And Let Die: Some kind of cross between a Bond movie, a blaxploitation flick, and the Dukes Of Hazzard. Odd.
  • The Town: Genre flick. Slightly better than average.
  • I, Anna: Sort of noir-ish psychological thriller. OK.
  • Les Miserable: The movie of the musical. Pretty good. I need to read the book sometime.

Interestingly, Air New Zealand lets you see what movies they're showing on their routes. This Web interface is a pretty faithful mockup of the actual in-seat interface (which is pretty bad ... it would be great to be able to see more than one movie title at a time).

On Friday I leave for Taiwan for a week at the Mozilla office, a "Web rendering" work week. This should be even more fun than last week.

Robert O'CallahanThe Direct Route

Over time I've become increasingly impressed with the broad applicability of Matthew 18:15-17:

If your brother or sister sins, go and point out their fault, just between the two of you. If they listen to you, you have won them over. But if they will not listen, take one or two others along, so that ‘every matter may be established by the testimony of two or three witnesses.’ If they still refuse to listen, tell it to the church; and if they refuse to listen even to the church, treat them as you would a pagan or a tax collector.

The first step is often difficult but crucial. The path of least resistance can be to go behind your antagonist's back --- to your friends, or their friends, or their manager. I've seen all kinds of negative consequences from following that path --- hurt, distrust, unnecessary escalation, confusion and fear. I feel my integrity depends on people knowing that whatever I say about them to others, they will not be surprised by because they've already heard it from me.

This applies in the other direction too, when people complain about third parties to me. If the third party is unaware of the issue, I don't want to know --- go away and talk to them first.

There are rare exceptions, usually involving time-critical emergencies or complex secrecy requirements.