Hacks.Mozilla.Org — Fuzzing rust-minidump for Embarrassment and Crashes – Part 2

This is part 2 of a series of articles on rust-minidump. For part 1, see here.

So to recap, we rewrote breakpad’s minidump processor in Rust, wrote a ton of tests, and deployed to production without any issues. We killed it, perfect job.

And we still got massively dunked on by the fuzzer. Just absolutely destroyed.

I was starting to pivot off of rust-minidump work because I needed a bit of palette cleanser before tackling round 2 (handling native debuginfo, filling in features for other groups who were interested in rust-minidump, adding extra analyses that we’d always wanted but were too much work to do in Breakpad, etc etc etc).

I was still getting some PRs from people filling in the corners they needed, but nothing that needed too much attention, and then @5225225 smashed through the windows and released a bunch of exploding fuzzy rabbits into my office.

I had no idea who they were or why they were there. When I asked they just lowered one of their seven pairs of sunglasses and said “Because I can. Now hold this bunny”. I did as I was told and held the bunny. It was a good bun. Dare I say, it was a true bnnuy: it was libfuzzer. (Huh? You thought it was gonna be AFL? Weird.)

As it turns out, several folks had built out some really nice infrastructure for quickly setting up a decent fuzzer for some Rust code: cargo-fuzz. They even wrote a little book that walks you through the process.

Apparently those folks had done such a good job that 5225225 had decided it would be a really great hobby to just pick up a random rust project and implement fuzzing for it. And then to fuzz it. And file issues. And PRs that fix those issues. And then implement even more fuzzing for it.

Please help my office is drowning in rabbits and I haven’t seen my wife in weeks.

As far as I can tell, the process seems to genuinely be pretty easy! I think their first fuzzer for rust-minidump was basically just:

  • checked out the project
  • run cargo fuzz init (which autogenerates a bunch of config files)
  • write a file with this:
#![no_main]

use libfuzzer_sys::fuzz_target;
use minidump::*;

fuzz_target!(|data: &[u8]| {
    // Parse a minidump like a normal user of the library
    if let Ok(dump) = minidump::Minidump::read(data) {
        // Ask the library to get+parse several streams like a normal user.

        let _ = dump.get_stream::<MinidumpAssertion>();
        let _ = dump.get_stream::<MinidumpBreakpadInfo>();
        let _ = dump.get_stream::<MinidumpCrashpadInfo>();
        let _ = dump.get_stream::<MinidumpException>();
        let _ = dump.get_stream::<MinidumpLinuxCpuInfo>();
        let _ = dump.get_stream::<MinidumpLinuxEnviron>();
        let _ = dump.get_stream::<MinidumpLinuxLsbRelease>();
        let _ = dump.get_stream::<MinidumpLinuxMaps>();
        let _ = dump.get_stream::<MinidumpLinuxProcStatus>();
        let _ = dump.get_stream::<MinidumpMacCrashInfo>();
        let _ = dump.get_stream::<MinidumpMemoryInfoList>();
        let _ = dump.get_stream::<MinidumpMemoryList>();
        let _ = dump.get_stream::<MinidumpMiscInfo>();
        let _ = dump.get_stream::<MinidumpModuleList>();
        let _ = dump.get_stream::<MinidumpSystemInfo>();
        let _ = dump.get_stream::<MinidumpThreadNames>();
        let _ = dump.get_stream::<MinidumpThreadList>();
        let _ = dump.get_stream::<MinidumpUnloadedModuleList>();
    }
});

And that’s… it? And all you have to do is type cargo fuzz run and it downloads, builds, and spins up an instance of libfuzzer and finds bugs in your project overnight?

Surely that won’t find anything interesting. Oh it did? It was largely all bugs in code I wrote? Nice.

cargo fuzz is clearly awesome but let’s not downplay the amount of bafflingly incredible work that 5225225 did here! Fuzzers, sanitizers, and other code analysis tools have a very bad reputation for drive-by contributions.

I think we’ve all heard stories of someone running a shiny new tool on some big project they know nothing about, mass filing a bunch of issues that just say “this tool says your code has a problem, fix it” and then disappearing into the mist and claiming victory.

This is not a pleasant experience for someone trying to maintain a project. You’re dumping a lot on my plate if I don’t know the tool, have trouble running the tool, don’t know exactly how you ran it, etc.

It’s also very easy to come up with a huge pile of issues with very little sense of how significant they are.

Some things are only vaguely dubious, while others are horribly terrifying exploits. We only have so much time to work on stuff, you’ve gotta help us out!

And in this regard 5225225’s contributions were just, bloody beautiful.

Like, shockingly fantastic.

They wrote really clear and detailed issues. When I skimmed those issues and misunderstood them, they quickly clarified and got me on the same page. And then they submitted a fix for the issue before I even considered working on the fix. And quickly responded to review comments. I didn’t even bother asking them to squashing their commits because damnit they earned those 3 commits in the tree to fix one overflow.

Then they submitted a PR to merge the fuzzer. They helped me understand how to use it and debug issues. Then they started asking questions about the project and started writing more fuzzers for other parts of it. And now there’s like 5 fuzzers and a bunch of fixed issues!

I don’t care how good cargo fuzz is, that’s a lot of friggin’ really good work! Like I am going to cry!! This was so helpful??? 😭

That said, I will take a little credit for this going so smoothly: both Rust itself and rust-minidump are written in a way that’s very friendly to fuzzing. Specifically, rust-minidump is riddled with assertions for “hmm this seems messed up and shouldn’t happen but maybe?” and Rust turns integer overflows into panics (crashes) in debug builds (and index-out-of-bounds is always a panic).

Having lots of assertions everywhere makes it a lot easier to detect situations where things go wrong. And when you do detect that situation, the crash will often point pretty close to where things went wrong.

As someone who has worked on detecting bugs in Firefox with sanitizer and fuzzing folks, let me tell you what really sucks to try to do anything with: “Hey so on my machine this enormous complicated machine-generated input caused Firefox to crash somewhere this one time. No, I can’t reproduce it. You won’t be able to reproduce it either. Anyway, try to fix it?”

That’s not me throwing shade on anyone here. I am all of the people in that conversation. The struggle of productively fuzzing Firefox is all too real, and I do not have a good track record of fixing those kinds of bugs. 

By comparison I am absolutely thriving under “Yeah you can deterministically trip this assertion with this tiny input you can just check in as a unit test”.

And what did we screw up? Some legit stuff! It’s Rust code, so I am fairly confident none of the issues were security concerns, but they were definitely quality of implementation issues, and could have been used to at very least denial-of-service the minidump processor.

Now let’s dig into the issues they found!

#428: Corrupt stacks caused infinite loops until OOM on ARM64

Issue

As noted in the background, stackwalking is a giant heuristic mess and you can find yourself going backwards or stuck in an infinite loop. To keep this under control, stackwalkers generally require forward progress.

Specifically, they require the stack pointer to move down the stack. If the stack pointer ever goes backwards or stays the same, we just call it quits and end the stackwalk there.

However, you can’t be so strict on ARM because leaf functions may not change the stack size at all. Normally this would be impossible because every function call at least has to push the return address to the stack, but ARM has the link register which is basically an extra buffer for the return address.

The existence of the link register in conjunction with an ABI that makes the callee responsible for saving and restoring it means leaf functions can have 0-sized stack frames!

To handle this, an ARM stackwalker must allow for there to be no forward progress for the first frame of a stackwalk, and then become more strict. Unfortunately I hand-waved that second part and ended up allowing infinite loops with no forward progress:

// If the new stack pointer is at a lower address than the old,
// then that's clearly incorrect. Treat this as end-of-stack to
// enforce progress and avoid infinite loops.
//
// NOTE: this check allows for equality because arm leaf functions
// may not actually touch the stack (thanks to the link register
// allowing you to "push" the return address to a register).
if frame.context.get_stack_pointer() < self.get_register_always("sp") as u64 {
    trace!("unwind: stack pointer went backwards, assuming unwind complete");
    return None;
}

So if the ARM64 stackwalker ever gets stuck in an infinite loop on one frame, it will just build up an infinite backtrace until it’s killed by an OOM. This is very nasty because it’s a potentially very slow denial-of-service that eats up all the memory on the machine!

This issue was actually originally discovered and fixed in #300 without a fuzzer, but when I fixed it for ARM (32-bit) I completely forgot to do the same for ARM64. Thankfully the fuzzer was evil enough to discover this infinite looping situation on its own, and the fix was just “copy-paste the logic from the 32-bit impl”.

Because this issue was actually encountered in the wild, we know this was a serious concern! Good job, fuzzer!

(This issue specifically affected minidump-processor and minidump-stackwalk)

#407: MinidumpLinuxMaps address-based queries didn’t work at all

Issue

MinidumpLinuxMaps is an interface for querying the dumped contents of Linux’s /proc/self/maps file. This provides metadata on the permissions and allocation state for mapped ranges of memory in the crashing process.

There are two usecases for this: just getting a full dump of all the process state, and specifically querying the memory properties for a specific address (“hey is this address executable?”). The dump usecase is handled by just shoving everything in a Vec. The address usecase requires us to create a RangeMap over the entries.

Unfortunately, a comparison was flipped in the code that created the keys to the RangeMap, which resulted in every correct memory range being discarded AND invalid memory ranges being accepted. The fuzzer was able to catch this because the invalid ranges tripped an assertion when they got fed into the RangeMap (hurray for redundant checks!).

// OOPS
if self.base_address < self.final_address { 
 return None; 
}

Although tests were written for MinidumpLinuxMaps, they didn’t include any invalid ranges, and just used the dump interface, so the fact that the RangeMap was empty went unnoticed!

This probably would have been quickly found as soon as anyone tried to actually use this API in practice, but it’s nice that we caught it beforehand! Hooray for fuzzers!

(This issue specifically affected the minidump crate which technically could affect minidump-processor and minidump-stackwalk. Although they didn’t yet actually do address queries, they may have crashed when fed invalid ranges.)

#381: OOM from reserving memory based on untrusted list length.

Issue

Minidumps have lots of lists which we end up collecting up in a Vec or some other collection. It’s quite natural and more efficient to start this process with something like Vec::with_capacity(list_length). Usually this is fine, but if the minidump is corrupt (or malicious), then this length could be impossibly large and cause us to immediately OOM.

We were broadly aware that this was a problem, and had discussed the issue in #326, but then everyone left for the holidays. #381 was a nice kick in the pants to actually fix it, and gave us a free simple test case to check in.

Although the naive solution would be to fix this by just removing the reserves, we opted for a solution that guarded against obviously-incorrect array lengths. This allowed us to keep the performance win of reserving memory while also making rust-minidump fast-fail instead of vaguely trying to do something and hallucinating a mess.

Specifically, @Swatinem introduced a function for checking that the amount of memory left in the section we’re parsing is large enough to even hold the claimed amount of items (based on their known serialized size). This should mean the minidump crate can only be induced to reserve O(n) memory, where n is the size of the minidump itself.

For some scale:

  • A minidump for Firefox’s main process with about 100 threads is about 3MB.
  • A minidump for a stackoverflow from infinite recursion (8MB stack, 9000 calls) is about 8MB.
  • A breakpad symbol file for Firefox’s main module can be about 200MB.

If you’re symbolicating, Minidumps probably won’t be your memory bottleneck. 😹

(This issue specifically affected the minidump crate and therefore also minidump-processor and minidump-stackwalk.)

The Many Integer Overflows and My Greatest Defeat

The rest of the issues found were relatively benign integer overflows. I claim they’re benign because rust-minidump should already be working under the assumption that all the values it reads out of the minidump could be corrupt garbage. This means its code is riddled with “is this nonsense” checks and those usually very quickly catch an overflow (or at worst print a nonsense value for some pointer).

We still fixed them all, because that’s shaky as heck logic and we want to be robust. But yeah none of these were even denial-of-service issues, as far as I know.

To demonstrate this, let’s discuss the most evil and embarrassing overflow which was definitely my fault and I am still mad about it but in a like “how the heck” kind of way!?

The overflow is back in our old friend the stackwalker. Specifically in the code that attempts to unwind using frame pointers. Even more specifically, when offsetting the supposed frame-pointer to get the location of the supposed return address:

let caller_ip = stack_memory.get_memory_at_address(last_bp + POINTER_WIDTH)?;
let caller_bp = stack_memory.get_memory_at_address(last_bp)?;
let caller_sp = last_bp + POINTER_WIDTH * 2;

If the frame pointer (last_bp) was ~u64::MAX, the offset on the first line would overflow and we would instead try to load ~null. All of our loads are explicitly fallible (we assume everything is corrupt garbage!), and nothing is ever mapped to the null page in normal applications, so this load would reliably fail as if we had guarded the overflow. Hooray!

…but the overflow would panic in debug builds because that’s how debug builds work in Rust!

This was actually found, reported, and fixed without a fuzzer in #251. All it took was a simple guard:

(All the casts are because this specific code is used in the x86 impl and the x64 impl.)

if last_bp as u64 >= u64::MAX - POINTER_WIDTH as u64 * 2 {
    // Although this code generally works fine if the pointer math overflows,
    // debug builds will still panic, and this guard protects against it without
    // drowning the rest of the code in checked_add.
    return None;
}

let caller_ip = stack_memory.get_memory_at_address(last_bp as u64 + POINTER_WIDTH as u64)?;
let caller_bp = stack_memory.get_memory_at_address(last_bp as u64)?;
let caller_sp = last_bp + POINTER_WIDTH * 2;

And then it was found, reported, and fixed again with a fuzzer in #422.

Wait what?

Unlike the infinite loop bug, I did remember to add guards to all the unwinders for this problem… but I did the overflow check in 64-bit even for the 32-bit platforms.

slaps forehead

This made the bug report especially confusing at first because the overflow was like 3 lines away from a guard for that exact overflow. As it turns out, the mistake wasn’t actually as obvious as it sounds! To understand what went wrong, let’s talk a bit more about pointer width in minidumps.

A single instance of rust-minidump has to be able to handle crash reports from any platform, even ones it isn’t natively running on. This means it needs to be able to handle both 32-bit and 64-bit platforms in one binary. To avoid the misery of copy-pasting everything or making everything generic over pointer size, rust-minidump prefers to work with 64-bit values wherever possible, even for 32-bit plaftorms.

This isn’t just us being lazy: the minidump format itself does this! Regardless of the platform, a minidump will refer to ranges of memory with a MINIDUMP_MEMORY_DESCRIPTOR whose base address is a 64-bit value, even on 32-bit platforms!

typedef struct _MINIDUMP_MEMORY_DESCRIPTOR {
  ULONG64                      StartOfMemoryRange;
  MINIDUMP_LOCATION_DESCRIPTOR Memory;
} MINIDUMP_MEMORY_DESCRIPTOR, *PMINIDUMP_MEMORY_DESCRIPTOR;

So quite naturally rust-minidump’s interface for querying saved regions of memory just operates on 64-bit (u64) addresses unconditionally, and 32-bit-specific code casts its u32 address to a u64 before querying memory.

That means the code with the overflow guard was manipulating those values as u64s on x86! The problem is that after all the memory loads we would then go back to “native” sizes and compute caller_sp = last_bp + POINTER_WIDTH * 2. This would overflow a u32 and crash in debug builds. 😿

But here’s the really messed up part: getting to that point meant we were successfully loading memory up to that address. The first line where we compute caller_ip reads it! So this overflow means… we were… loading memory… from an address that was beyond u32::MAX…!?

Yes!!!!!!!!

The fuzzer had found an absolutely brilliantly evil input.

It abused the fact that MINIDUMP_MEMORY_DESCRIPTOR technically lets 32-bit minidumps define memory ranges beyond u32::MAX even though they could never actually access that memory! It could then have the u64-based memory accesses succeed but still have the “native” 32-bit operation overflow!

This is so messed up that I didn’t even comprehend that it had done this until I wrote my own test and realized that it wasn’t actually failing because I foolishly had limited the range of valid memory to the mere 4GB a normal x86 process is restricted to.

And I mean that quite literally: this is exactly the issue that creates Parallel Universes in Super Mario 64.

But hey my code was probably just bad. I know google loves sanitizers and fuzzers, so I bet google breakpad found this overflow ages ago and fixed it:

uint32_t last_esp = last_frame->context.esp;
uint32_t last_ebp = last_frame->context.ebp;
uint32_t caller_eip, caller_esp, caller_ebp;

if (memory_->GetMemoryAtAddress(last_ebp + 4, &caller_eip) &&
    memory_->GetMemoryAtAddress(last_ebp, &caller_ebp)) {
    caller_esp = last_ebp + 8;
    trust = StackFrame::FRAME_TRUST_FP;
} else {
    ...

Ah. Hmm. They don’t guard for any kind of overflow for those uint32_t’s (or the uint64_t’s in the x64 impl).

Well ok GetMemoryAtAddress does actual bounds checks so the load from ~null will generally fail like it does in rust-minidump. But what about the Parallel Universe overflow that lets GetMemoryAtAddress succeed?

Ah well surely breakpad is more principled with integer width than I was–

virtual bool GetMemoryAtAddress(uint64_t address, uint8_t*  value) const = 0;
virtual bool GetMemoryAtAddress(uint64_t address, uint16_t* value) const = 0;
virtual bool GetMemoryAtAddress(uint64_t address, uint32_t* value) const = 0;
virtual bool GetMemoryAtAddress(uint64_t address, uint64_t* value) const = 0;

Whelp congrats to 5225225 for finding an overflow that’s portable between two implementations in two completely different languages by exploiting the very nature of the file format itself!

In case you’re wondering what the implications of this overflow are: it’s still basically benign. Both rust-minidump and google-breakpad will successfully complete the frame pointer analysis and yield a frame with a ~null stack pointer.

Then the outer layer of the stackwalker which runs all the different passes in sequence will see something succeeded but that the frame pointer went backwards. At this point it will discard the stack frame and terminate the stackwalk normally and just calmly output whatever the backtrace was up to that point. Totally normal and reasonable operation.

I expect this is why no one would notice this in breakpad even if you run fuzzers and sanitizers on it: nothing in the code actually does anything wrong. Unsigned integers are defined to wrap, the program behaves reasonably, everything is kinda fine. We only noticed this in rust-minidump because all integer overflows panic in Rust debug builds.

However this “benign” behaviour is slightly different from properly guarding the overflow. Both implementations will normally try to move on to stack scanning when the frame pointer analysis fails, but in this case they give up immediately. It’s important that the frame pointer analysis properly identifies failures so that this cascading can occur. Failing to do so is definitely a bug!

However in this case the stack is partially in a parallel universe, so getting any kind of useful backtrace out of it is… dubious to say the least.

So I totally stand by “this is totally benign and not actually a problem” but also “this is sketchy and we should have the bounds check so we can be confident in this code’s robustness and correctness”.

Minidumps are all corner cases — they literally get generated when a program encounters an unexpected corner case! It’s so tempting to constantly shrug off situations as “well no reasonable program would ever do this, so we can ignore it”… but YOU CAN’T.

You would not have a minidump at your doorstep if the program had behaved reasonably! The fact that you are trying to inspect a minidump means something messed up happened, and you need to just deal with it!

That’s why we put so much energy into testing this thing, it’s a nightmare!

I am extremely paranoid about this stuff, but that paranoia is based on the horrors I have seen. There are always more corner cases.

There are ALWAYS more corner cases. ALWAYS.

 

The post Fuzzing rust-minidump for Embarrassment and Crashes – Part 2 appeared first on Mozilla Hacks - the Web developer blog.

Jody Heavener — Introducing 1Password for Visual Studio Code

In writing software, we’re used to embedding secrets and other configurable values right in the codebase. They might be Stripe keys to power your online shop, webhooks for a custom Slack bot, a Docker username and password for a CI config, AWS credentials, or an API token and host to set up 1Password Connect.

Secrets are used everywhere in our code. Sometimes, though, we forget when we’ve been using real secrets in our work. Maybe there’s a leftover token you dropped in to build that one feature, or maybe you didn’t delete the .env file you set up to test drive the app. Now you’ve got to rotate your secrets because you accidentally committed and pushed sensitive values for the whole world to see. Yikes.

We’ve all been there. That’s why I’m delighted that I get to announce the launch of the all-new 1Password for VS Code extension.

Go ahead, commit your secrets references

With 1Password Secrets Automation, the 1Password Developer Products team introduced the concept of secret references. It starts by storing a sensitive value, such as an API credential or client ID, in 1Password. That item and the field you’d like to get the value from can then be retrieved through a special op:// URL scheme that 1Password’s tooling knows how to parse. It’s made up of three parts: vault, item, and field. This is known as a “secret reference”.

1Password Secret Reference example consisting of vault, item, and field

Now, instead of using a real value in your configs, environment variable files, or anywhere else in the codebase, just drop in the secret reference in VS Code. When you do, you can rest easy knowing that the real value will never accidentally make its way into your codebase.

The best part? Through our suite of tools and integrations, you can work with references in both local and deployed environments.

To help make sure you’re not accidentally leaving secrets in your code, you can move them over to 1Password with just a couple clicks. The extension uses a series of secret detection techniques to look for values that might be sensitive. With these matches, it makes inline suggestions to store them in 1Password, automatically replacing them with secret references.

Example of a secret that would be replaced by the 1Password extension for VS Code

Secret reference integration doesn’t stop there. You can hover a reference to inspect the item and field details, click it to open the item in the desktop app, and even preview the real values of an entire file full of references.

Beyond secret detection suggestions, 1Password for VS Code makes it easy to retrieve items for use in your code, as well as store any bits of code you’d like in 1Password. If you’ve got multiple values you want stored in the same item – perhaps a username, password, and email – it supports that as well. Just select each value and run the “Save in 1Password” command.

VSCode Screenshot

Built using tools available to everyone

I’ll let you in on a little secret: we didn’t plan to build this extension. It wasn’t requested by our developer community, and wasn’t part of any roadmap. Instead this extension began as a side project for myself. I wanted to scratch my own itch and integrate 1Password more closely into my development workflow, and to generally learn more about developing with 1Password.

So you can imagine my excitement when, after a quick demo at an internal call, I was invited to polish it up and get it slated for release.

To my delight, after demoing the extension and then going on vacation, Dave posted a video of the presentation from his CLI launch blog post and it was met with some pretty wild enthusiasm from the developer community. There was even some love for it at our 1Password 8 for Mac Reddit AMA:

Reddit comment from user arnebr asking if the VS Code plugin will be live soon

Although not a goal from the outset, an interesting aspect of this project is that it’s built using only tools available to the public – there’s nothing internal or proprietary powering the features of the extension. We’ve even open-sourced the whole project on our GitHub, so if you want to help iterate on it or report an issue, that’s a great place to start.

VS Code extensions run in a Node environment, and we wanted to interact with the new CLI. So we built and open-sourced an entirely new package for doing exactly this: op-js. It wraps the CLI with a simple-to-use JavaScript interface and ships with TypeScript declarations, making 60+ commands, including those that support biometrics unlock, available to your Node-based application.

Ultimately my hope is that this extension demonstrates some of the neat ways you can extend the power of 1Password by building your own integrations, whether it be for yourself or others. And you should have fun doing it. We’re in early days here, with plenty more developer offerings coming down the line.

I’d love to hear what you think, and we’ll be iterating on the extension as feedback rolls in. Learn more about 1Password for VS Code and our other developer tools by checking out our developer portal. While you’re there, consider joining our Developer Slack workspace, where you’ll find myself and others on the Developer Products team who are eager to hear how you’re incorporating 1Password into your development workflow. And if you’re building something cool, be sure to tag it #BuildWith1Password!

Finally, we owe a tremendous debt of gratitude to Mike Selander, Chris Dunn-Birch, Floris van der Grinten, the incredibly helpful folks over in the VS Code Extension community, and so many more for providing endless help and guidance while working on this project. Thank you!

This Week In Rust — This Week in Rust 448

Hello and welcome to another issue of This Week in Rust! Rust is a programming language empowering everyone to build reliable and efficient software. This is a weekly summary of its progress and community. Want something mentioned? Tweet us at @ThisWeekInRust or send us a pull request. Want to get involved? We love contributions.

This Week in Rust is openly developed on GitHub. If you find any errors in this week's issue, please submit a PR.

Updates from Rust Community

Official
Project/Tooling Updates
Observations/Thoughts
Rust Walkthroughs

Crate of the Week

This week's crate is error-stack, a currently nightly-only error handling library that optimizes for ease of use while allowing to add arbitrary context data to errors.

Thanks to Alfred Mountfield for the self-suggestion.

Please submit your suggestions and votes for next week!

Call for Participation

Always wanted to contribute to open-source projects but didn't know where to start? Every week we highlight some tasks from the Rust community for you to pick and get started!

Some of these tasks may also have mentors available, visit the task page for more information.

If you are a Rust project owner and are looking for contributors, please submit tasks here.

Updates from the Rust Project

374 pull requests were merged in the last week

Rust Compiler Performance Triage

A pretty busy week for performance, with quite a few PRs landing with significant improvements to specific benchmarks. Overall the week was positive, with many benchmarks improving at least a little. No significant changes in memory usage this week.

Triage done by @simulacrum. Revision range: edab34a..abace0a

5 Regressions, 6 Improvements, 3 Mixed; 1 of them in rollups 54 artifact comparisons made in total

See full report for details.

Call for Testing

An important step for RFC implementation is for people to experiment with the implementation and give feedback, especially before stabilization. The following RFCs would benefit from user testing before moving forward:

  • No RFCs issued a call for testing this week.

If you are a feature implementer and would like your RFC to appear on the above list, add the new call-for-testing label to your RFC along with a comment providing testing instructions and/or guidance on which aspect(s) of the feature need testing.

Approved RFCs

Changes to Rust follow the Rust RFC (request for comments) process. These are the RFCs that were approved for implementation this week:

Final Comment Period

Every week the team announces the 'final comment period' for RFCs and key PRs which are reaching a decision. Express your opinions now.

RFCs
Tracking Issues & PRs
New and Updated RFCs
  • No New or Updated RFCs were created this week.

Upcoming Events

Rusty Events between 2022-06-22 - 2022-07-20 🦀

Virtual
Asia
Europe
North America
Oceania

If you are running a Rust event please add it to the calendar to get it mentioned here. Please remember to add a link to the event too. Email the Rust Community Team for access.

Rust Jobs

Mutate

Matician

Lithic

Spire Global

Tweet us at @ThisWeekInRust to get your job offers listed here!

Quote of the Week

Rwlock vs Mutex? Please, tell me like I'm 5

Mutex: "Mom says it's my turn on the synchronization primitive."
vs.
Write lock: "Hey! You all are not allowed to look until I'm done writing!" Read lock: "Hey! You are not allowed to edit what you wrote until we're done reading it!"

Thanks for an actual 5 year old reply, made me laugh

– /u/LyonSyonII and /u/everything-narrative on /r/rust

Thanks to Josh Triplett for the suggestion!

Please submit quotes and vote for next week!

This Week in Rust is edited by: nellshamrell, llogiq, cdmistman, ericseppanen, extrawurst, andrewpollack, U007D, kolharsam, joelmarcey, mariannegoldin.

Email list hosting is sponsored by The Rust Foundation

Discuss on r/rust

The Rust Programming Language Blog — Call for testing: Cargo sparse-registry

The Cargo nightly sparse-registry feature is ready for testing. The feature causes Cargo to access the crates.io index over HTTP, rather than git. It can provide a significant performance improvement, especially if the local copy of the git index is out-of-date or not yet cloned.

Overview

To try it out, add the -Z sparse-registry flag on a recent nightly build of Cargo. For example, to update dependencies:

rustup update nightly
cargo +nightly -Z sparse-registry update

The feature can also be enabled by setting the environment variable CARGO_UNSTABLE_SPARSE_REGISTRY=true. Setting this variable will have no effect on stable Cargo, making it easy to opt-in for CI jobs.

The minimum Cargo version is cargo 2022-06-17, which is bundled with rustc 2022-06-20.

You can leave feedback on the internals thread.

If you see any issues please report them on the Cargo repo. The output of Cargo with the environment variable CARGO_LOG=cargo::sources::registry::http_remote=trace set will be helpful in debugging.

Details

Accessing the index over HTTP allows crates.io to continue growing without hampering performance. The current git index continues to grow as new crates are published, and clients must download the entire index. The HTTP index only requires downloading metadata for crates in your dependency tree.

The performance improvement for clients should be especially noticeable in CI environments, particularly if no local cache of the index exists.

On the server side, the HTTP protocol is much simpler to cache on a CDN, which improves scalability and reduces server load. Due to this caching, crate updates may take an extra minute to appear in the index.

The Cargo team plans to eventually make this the default way to access crates.io (though the git index will remain for compatibility with older versions of Cargo and external tools). Cargo.lock files will continue to reference the existing crates.io index on GitHub to avoid churn.

The -Z sparse-registry flag also enables alternative registries to be accessed over HTTP. For more details, see the tracking issue.

Thank you

This project has been in the works for over 2.5 years with collaboration from the crates.io, infra, and Cargo teams.

@kornelski wrote the sparse-index RFC and initial performance proof of concept. @jonhoo created the initial implementation in Cargo and gathered performance data. @arlosi completed the implementation in Cargo and implemented the changes to crates.io to serve the index. @eh2406 provided numerous reviews and feedback to get all the changes landed. Many others from the community helped by providing suggestions, feedback, and testing.

Thank you to everyone involved!

The Mozilla Blog — Joey Amato, Publisher of Pride Journeys, Shares What Brings Him Joy Online

Here at Mozilla, we are the first to admit the internet isn’t perfect, but we are also quick to point out that the internet is pretty darn magical. The internet opens up doors and opportunities, allows for people to connect with others, and lets everyone find where they belong — their corners of the internet. We all have an internet story worth sharing. In My Corner of the Internet, we talk with people about the online spaces they can’t get enough of, what we should save in Pocket to read later and what sites and forums shaped them.

With Pride celebrations taking place throughout June, we’re featuring LGBTQ+ leaders this month as part of our My Corner of the Internet series. In this next installment, Joey Amato, publisher of Pride Journeys, shares his love of travel, Peloton and his nostalgia of MySpace.

What is your favorite corner of the internet?

I enjoy reading Billboard.com to keep up with music industry news as well as CNBC.com for business and finance news. I’m a stock market junkie so I love reading about financial news, especially in today’s crazy market.

What is an internet deep dive that you can’t wait to jump back into? 

I like keeping up with music trends and reading up new artists and concert tours. I used to be in the music industry when I lived in New York and Nashville, so to stay connected, I check out Rolling Stone, Music Row and Pollstar almost on a daily basis.

What is the one tab on your browser you always regret closing? 

Amazon – I spend way too much money there! Say what you want about Amazon and Jeff Bezos, but he changed the world. Our lives are completely different because of this company and the convenience they provide.

Who is an LGBTQ+ person with a very online presence that is a role model for you? 

I really enjoy Matty Maggiacomo who is a trainer on Peloton. He’s easy on the eyes and also pushes me in every workout. I recently began subscribing to Peloton earlier this year. I don’t own a Peloton bike, but I love their other workouts.

What can you not stop talking about on the internet right now?

The new ABBA show in London. I can’t wait to attend in person. The reviews thus far have been quite incredible.

What was the first online community you engaged with?

Probably MySpace….yes, I’m old! It was the first time I began connecting with people who I haven’t seen in a while or had an interest in. I liked being able to customize my own space. 

What articles and videos are in your Pocket waiting to be read/watched right now?

Since I have been traveling a lot recently, I am behind in my financial research and launching my new site for the LGBTQ community… details coming soon! I also need to catch up on my travel writing.

If you could create your own corner of the internet what would it look like? 

It would definitely be an LGBTQ related corner. I have been in the space for most of my professional career, so I feel like this is an area I can really contribute to.

Joey Amato is the publisher of Pride Journeys, a website dedicated to LGBTQ travel. He also publishes an annual LGBTQ travel directory which features destinations from around the world looking to reach LGBTQ travelers. Most recently, Pride Journeys published the Ultimate Pride Guide, a calendar of pride festivals for those looking to travel with pride.

Save and discover the best articles, stories and videos on the web

Get Pocket

The post Joey Amato, Publisher of Pride Journeys, Shares What Brings Him Joy Online appeared first on The Mozilla Blog.

The Mozilla Blog — Understanding Apple’s Private Click Measurement

Private advertising technology proposals could greatly improve privacy for web users. Web advertising has a reputation for poor privacy practices. Firefox, other browsers, and the web community are collaborating on finding ways to support advertising while maintaining strong, technical privacy protections for users.

This series of posts aims to contribute to the ongoing conversation regarding the future of advertising on the web by providing technical analyses on proposals that have been put forward by various players in the ecosystem to address the questions of what might replace third-party cookies. In this installment, we look at Apple’s Private Click Measurement (PCM).

PCM differs from the subject of our other analyses in that it is more than a mere proposal. PCM is available in Safari and iOS today.

The goal of PCM is to support conversion measurement through a new in-browser API. Conversion measurement aims to measure how effective advertising is by observing which advertisements lead to sales or other outcomes like page views and sign-ups. PCM measures the most direct kind of conversion, where a user clicks an ad on one site then later takes an action on the advertiser’s site, like buying a product.

The way that PCM works is that the browser records when clicks and conversions occur. When a click is followed by a conversion, the browser creates a report. PCM aims to safeguard privacy by strictly limiting the information that is included in the report and by submitting reports to the websites through an anonymization service after a delay.

Our analysis concludes that PCM is a poor trade-off between user privacy and advertising utility:

  • Although PCM prevents sites from performing mass tracking, it still allows them to track a small number of users.
  • The measurement capabilities PCM provides are limited relative to the practices that advertisers currently employ, with long delays and too few identifiers for campaigns being the most obvious of the shortcomings.

The poor utility of PCM offers sites no incentive to use it over tracking for browsers that continue to offer cross-site cookies, like Chrome. For browsers that limit the use of cookies — like Firefox or Safari, especially with stricter controls like Total Cookie Protection in Firefox — tracking is already difficult. If Firefox implemented PCM, it would enable a new way to perform cross-site tracking. While some sites might choose to use PCM as intended, nothing in the design of PCM prevents sites from using it for tracking.

Overall, the design choices in PCM that aim to safeguard privacy provide insufficient privacy protections, but they appear to make the API less useful for measurement.

The Private Advertising Technology Community Group (PATCG) in the W3C are currently discussing options for measurement in advertising.

For more on this:

Building a more privacy-preserving ads-based ecosystem

The future of ads and privacy

Privacy analysis of FLoC

Mozilla responds to the UK CMA consultation on Google’s commitments on the Chrome Privacy Sandbox

Privacy analysis of SWAN.community and Unified ID 2.0

Analysis of Google’s Privacy Budget proposal

Privacy Preserving Attribution for Advertising: our Interoperable Private Attribution proposal

The post Understanding Apple’s Private Click Measurement appeared first on The Mozilla Blog.

The Mozilla Blog — Reflecting on 10 years of time well spent with Pocket

Ten years ago, a small, yet mighty team launched Pocket because we felt that people deserved a better way to consume content on the internet. We wanted it to be easy — “as simple an action as putting it in your pocket” — and empowering, giving people the means to engage with the web on their own terms. We championed the save as a fundamental internet action — akin to browse, search and share — but more than any other, allowing you to create your own corner of the internet. 

Right away, Pocket’s save became powerful — a means to grab what you wanted from the waves of posts and tweets, long reads and memes and come back to it when it made the most sense for you. And it continues to lay the groundwork for the future ahead of us.  

A couple of interesting things happened as we started to lean into the save.

First, we remembered the power of stories. People understand the world through stories. The stories that people consume give them power, knowledge, and ideas. Stories allow people to access who they are and who they aspire to be. 

But we get distracted from those stories when too much of the focus from media and platforms tend to be on breaking news — who is first and loudest. In addition, platforms have created incentives all about the click — quality be damned. We have committed Pocket to being a place where quality stories, and those who crave them, can breathe and thrive.

Second, we’ve embraced curation. The modern web is vast, messy and noisy. There is some inherent beauty and innovation in this. As the Mozilla Manifesto states, the open, global internet is the most powerful communication and collaboration resource we have ever seen. It embodies some of our deepest hopes for human progress. It enables new opportunities for learning, building a sense of shared humanity, and solving the pressing problems facing people everywhere. 

But as the web has evolved, it has created a challenging and often overwhelming environment for online content consumers. People rightly feel like they’ve lost agency with their online time. Curation is our way to help readers fight back. We built Pocket’s approach to recommendations with the same intention as the save — content discovery grounded in surfacing stories worthy of your time and attention and taking an approach that brings together algorithms and a human touch so that our commitment to quality is always clear and felt. We often hear from users that “we really get them” even when they are viewing recommendations on non-personalized surfaces — what I believe they are responding to is the quality of what we curate and recommend to our users. 

Since we launched Pocket, people have saved more than 6.5 billion pieces of content. Every month, through Firefox New Tab and our Pocket Hits newsletter, more than 40 million people see our Pocket recommendations. We are strong believers and participants in the broader content ecosystem — and we want to direct users to quality content wherever it happens to live, championing creators and publishers, big and small. Pocket can always be trusted to bring you the good stuff.

In 2020, our recommendations evolved further, with the introduction of Pocket Collections. In the uncertain and disorienting early days of the pandemic, and then after George Floyd was murdered, we saw a need for high-quality content to contextualize and help people navigate events. We also saw an opportunity to elevate voices and use our platform to dig into complicated, systematic issues with room for nuance. Just reading the breaking news or knowing dates and names, the things that pop up in simple Google searches, wasn’t adequate. 

We began digging in deeper and inviting experts to create collections to bring a broader perspective, animated by the idea that some topics require more than just a single article or point of view to bring understanding. Pocket might not be where you come to learn who won an election. But it will be where you come to understand why. Since those initial collections around covid and racial justice, we’ve continued to build and explore where best to use this medium. We now have hundreds of collections, ranging from how to talk to people you disagree with and managing Zoom brain to great long reads on scams and the science of the multiverse. 

This is the future of Pocket we are continuing to expand — improving our ability to find the right piece of content and recommending it to you at that right moment. We are also trying in our own way to elevate unique voices, helping creators and fantastic independent publishers get in front of new audiences. In time, we may even explore opportunities to tap our users as experts in specific topics and pass the mic to them. As the internet and digital content overload economy continue to evolve, so will we. We won’t pretend to be able to solve the internet’s problems, but we do want to help connect people with the hope and talent and wonder we know still exists on the web. 

The internet doesn’t have to be a place that leaves people feeling overwhelmed. For the past 10 years, Pocket has been a corner of the internet where you can joy-scroll and satiate your curiosity about how we got here. Ultimately, our goal is to make sure that using Pocket is time well spent — and will continue to be 10 years from now. 

10 years of fascinating reads

From the top article from each year, to the way Pocket readers kind of predicted the future, these collections will certainly spark your interest

The post Reflecting on 10 years of time well spent with Pocket appeared first on The Mozilla Blog.

Pascal Chevrel — MozFR Transvision Reloaded: 1 year later

Just one year ago, the French Mozilla community was living times of major changes: several key historical contributors were leaving the project, our various community portals were no longer updates or broken, our tools were no longer maintained. At the same time a few new contributors were also popping in our IRC channel asking for ways to get involved in the French Mozilla community.

As a result, Kaze decided to organize the first ever community meetup for the French-speaking community in the Paris office (and we will repeat this meetup in June in the brand new Paris office!) .

IMG_0151

This resulted in a major and successful community reboot. Leaving contributors passed on the torch to other members of the community, newer contributors were meeting in real life for the first time. This is how Clarista officially became our events organizer, this is how Théo replaced Cédric as the main Firefox localizer and this is how I became the new developer for Transvision! :)

What is Transvision? Transvision is a web application created by Philippe Dessantes which was helping the French team finding localized/localizable strings in Mozilla repositories.

Summarized like that, it doesn't sound that great, but believe me, it is! Mozilla applications have big gigantic repos, there are tens of thousands of strings in our mercurial repositories, some of them we have translated like a decade ago, when you decide to change a verb for a better one for example, it is important to be able to find all occurrences of this verb you have used in the past to see if they need update too. When somebody spots a typo or a clumsy wording, it's good to be able to check if you didn't make the same translation mistakes in other parts of the Mozilla applications several years ago and of course, it's good to be able to check that in just a few seconds. Basically, Phillippe had built the QA/assistive technology for our localization process that best fitted our team needs and we just couldn't let it die.

During the MozFR meetup, Philippe showed to me how the application worked and we created a github repository where we put the currently running version of the code. I tagged that code as version 1.0.

Over the summer, I familiarized myself with the code which was mostly procedural PHP, several Bash scripts to maintain copies of our mercurial repos and a Python script used to extract the strings. Quickly, I decided that I would follow the old Open Source strategy of Release early, release often. Since I was doing that on the sidelines of my job at Mozilla,  I needed the changes to be small but frequent incremental steps as I didn't know how much time I could devote to this project. Basically, having frequent releases means that I always have the codebase in mind which is good since I can implement an idea quickly, without having to dive into the code to remember it.

One year and 15 releases later, we are now at version 2.5, so here are the features and achievements I am most proud of:

  1. Transvision is alive and kicking :)
  2. We are now a team! Jesús Perez has been contributing code since last December, a couple more people have shown interest in contributing and Philippe is interested in helping again too. We have also a dynamic community of localizers giving feedback, reporting bugs are asking for immrovements
  3. The project is now organized and if some day I need to step down and pass the torch to another maintainer, he should not have difficulties setting the project up and maintaining it. We have a github repo, release notes, bugs, tagged releases, a beta server, unit testing, basic stats to understand what is used in the app and a mostly cleaned up codebase using much more modern PHP and tools (Atoum, Composer). It's not perfect, but I think that for amateur developers, it's not bad at all and the most important thing is that the code keeps on improving!
  4. There are now more than 3000 searches per week done by localizers on Transvision. That was more like 30 per week a year ago. There are searches in more than 70 languages, although 30 locales are doing the bulk of searches and French is still the biggest consumer with 40% of requests.
  5. Some people are using Transvision in ways I hadn't anticipated, for example our documentation localizers use it to find the translation of UI mentioned in help articles they translate for support.mozilla.org, people in QA use it to point to localized strings in Bugzilla

A quick recap of what we have done, feature-wise, in the last 12 months:

  • Completely redesigned the application to look and feel good
  • Locale to Locale searches, English is not necessarily the locale you want to use as the source (very useful to check differences from languages of the same family, for example Occitan/French/Catalan/Spanish...).
  • Hints and warnings for strings that look too long or too short compare to English, potentially bad typography, access keys that don't match your translation...
  • Possibility for anybody to file a bug in Bugzilla with a pointer to the badly translated string (yes we will use it for QA test days within the French community!)
  • Firefox OS strings are now there
  • Search results are a lot more complete and accurate
  • We now have a stable Json/JsonP API, I know that Pontoon uses it to provide translation suggestions, I heard that the Moses project uses it too. (if you use the Transvision API, ping me, I'd like to know!)
  • We can point any string to the right revision controled file in the source and target repos
  • We have a companion add-on called MozTran for heavy users of the tool provided by Goofy, from our Babelzilla friends.

The above list is of course just a highlight of the main features, you can get more details on the changelog.

If you use Transvision, I hope you enjoy it and that it is useful oo you. If you don't use Transvision (yet), give it a try, it may help you in your translation process, especially if your localization process is similar to the French one (targets Firefox Nighty builds first, work directly on the mercurial repo, focus on QA).

This was the first year of the rebirth of Transvision, I hope that the year to come will be just as good as this one. I learnt a lot with this project and I am happy to see it grow both in terms of usage and community, I am also happy that one tool that was created by a specific localization team is now used by so many other teams in the world :)

Pascal Chevrel — My Q2-2014 report

Summary of what I did last quarter (regular l10n-drivers work such as patch reviews, pushes to production, meetings and past projects maintenance excluded) .

Australis release

At the end of April, we shipped Firefox 29 which was our first major redesign of the Firefox user interface since Firefox 4 (released in 2011). The code name for that was Australis and that meant replacing a lot of content on mozilla.org to introduce this new UI and the new features that go with it. That also means that we were able to delete a lot of old content that now had become really obsolete or that was now duplicated on our support site.

Since this was a major UI change, we decided to show an interactive tour of the new UI to both new users and existing users upgrading to the new version. That tour was fully localized in a few weeks time in close to 70 languages, which represents 97.5% of our user base. For the last locales not ready on time, we either decided to show them a partially translated site (some locales had translated almost everything or some of the non-translated strings were not very visible to most users, such as alternative content to images for screen readers) or to let the page fall back to the best language available (like Occitan falling back to French for example).

Mozilla.org was also updated with 6 new product pages replacing a lot of old content as well as updates to several existing pages. The whole site was fully ready for the launch with 60 languages 100% ready and 20 partially ready, all that done in a bit less than 4 weeks, parallel to the webdev integration work.

I am happy to say that thanks to our webdev team, our amazing l10n community and with the help of my colleagues Francesco Lodolo (also Italian localizer) and my intern Théo Chevalier (also French localizer), we were able to not only offer a great upgrading experience for the quasi totality of our user base, we were also able to clean up a lot of old content, fix many bugs and prepare the site from an l10n perspective for the upcoming releases of our products.

Today, for a big locale spanning all of our products and activities, mozilla.org is about 2,000 strings to translate and maintain (+500 since Q1), for a smaller locale, this is about 800 strings (+200 since Q1). This quarter was a significant bump in terms of strings added across all locales but this was closely related to the Australis launch, we shouldn't have such a rise in strings impacting all locales in the next quarters.

Transvision releases

Last quarter we did 2 releases of Transvision with several features targeting out 3 audiences: localizers, localization tools, current and potential Transvision developers.

For our localizers, I worked on a couple of features, one is quick filtering of search results per component for Desktop repositories (you search for 'home' and with one click, you can filter the results for the browser, for mail or for calendar for example). The other one is providing search suggestions when your search yields no results with the best similar matches ("your search for 'lookmark' yielded no result, maybe you were searching for 'Bookmark'?").

For the localization tools community (software or web apps like Pontoon, Mozilla translator, Babelzilla, OmegaT plugins...), I rewrote entirely our old Json API and extended it to provide more services. Our old API was initially created for our own purposes and basically was just giving the possibility to get our search results as a Json feed on our most popular views. Tools started using it a couple of years ago and we also got requests for API changes from those tool makers, therefore it was time to rewrite it entirely to make it scalable. Since we don't want to break anybody's workflow, we now redirect all the old API calls to the new API ones. One of the significant new service to the API is a translation memory query that gives you results and a quality index based on the Levenshtein distance with the searched terms. You can get more information on the new API in our documentation.

I also worked on improving our internal workflow and make it easier for potential developers wanting to hack on Transvision to install and run it locally. That meant that now we do continuous integration with Travis CI (all of our unit tests are ran on each commit and pull request on PHP 5.4 and 5.5 environments), we have made a lot of improvements to our unit tests suite and coverage, we expose to developers peak memory usage and time per request on all views so as to catch performance problems early, and we also now have a "dev" mode that allows getting Transvision installed and running on the PHP development server in a matter of minutes instead of hours for a real production mode. One of the blockers for new developers was the time required to install Transvision locally. Since it is a spidering tool looking for localized strings in Mozilla source repositories, it needed to first clone all the repositories it indexes (mercurial/git/svn) which is about 20GB of data and takes hours even with a fast connection. We are now providing a snapshot of the final extracted data (still 400MB ;)) every 6 hours that is used by the dev install mode.

Check the release notes for 3.3 and 3.4 to see what other features were added by the team (/ex: on demand TMX generation or dynamic Gaia comparison view added by Théo, my intern).

Web dashboard / Langchecker

The main improvement I brought to the web dashboard is probably this quarter the deadline field to all of our .lang files, which allows to better communicate the urgency of projects and for localizers are an extra parameter allowing them to prioritize their work.

Theo's first project for his internship was to build a 'project' view on the web dashboard that we can use to get an overview of the translation of a set of pages/files, this was used for the Australis release (ex: http://l10n.mozilla-community.org/webdashboard/?project=australis_all) but can be used to any other project we want to define , here is an example for the localization of two Android Add-ons I did for the World Cup that we did and tracked with .lang files.

We brought other improvements to our maintenance scripts for example to be able to "bulk activate" a page for all the locales ready, we improved our locamotion import scripts, started adding unit tests etc. Generally speaking, the Web dashboard keeps improving regularly since I rewrote it last quarter and we regularly experiment using it for more projects, especially for projects which don't fit in the usual web/product categories and that also need tracking. I am pretty happy too that now I co-own the dashboard with Francesco who brings his own ideas and code to streamline our processes.

Théo's internship

I mentionned it before, our main French localizer Théo Chevalier, is doing an internship with me and Delphine Lebédel as mentors, this is the internship that ends his 3rd year of engineering (in a 5 years curriculum). He is based in Montain View, started early April and will be with us until late July.

He is basically working on almost all of the projects I, Delphine and Flod work on.

So far, apart from regular work as an l10n-driver, he has worked for me on 3 projects, the Web Dashboard projects view, building TMX files on demand on Transvision and the Firefox Nightly localized page on mozilla.org. This last project I haven't talked about yet and he blogged about it recently, in short, the first page that is shown to users of localized builds of Firefox Nightly can now be localized, and by localized we don't just mean translated, we mean that we have a community block managed by the local community proposing Nightly users to join their local team "on the ground". So far, we have this page in French, Italian, German and Czech, if your locale workflow is to translate mozilla-central first, this is a good tooll for you to reach a potential technical audience to grow your community .

Community

This quarter, I found 7 new localizers (2 French, 1 Marahati, 2 Portuguese/Portugal, 1 Greek, 1 Albanian) to work with me essentially on mozilla.org content. One of them, Nicolas Delebeque, took the lead on the Australis launch and coordinated the French l10n team since Théo, our locale leader for French, was starting his internship at Mozilla.

For Transvision, 4 people in the French community (after all, Transvision was created initially by them ;)) expressed interest or small patches to the project, maybe all the efforts we did in making the application easy to install and hack are starting to pay, we'll probably see in Q3/Q4 :)

I spent some time trying to help rebuild the Portugal community which is now 5 people (instead of 2 before), we recently resurrected the mozilla.pt domain name to actually point to a server, the MozFR one already hosting the French community and WoMoz (having the French community help the Portuguese one is cool BTW). A mailing list for Portugal was created (accessible also as nntp and via google groups) and the #mozilla-portugal IRC channel was created. This is a start, I hope to have time in Q3 to help launch a real Portugal site and help them grow beyond localization because I think that communities focused on only one activity have no room to grow or renew themselves (you also need coding, QA, events, marketing...).

I also started looking at Babelzilla new platform rewrite project to replace the current aging platform (https://github.com/BabelZilla/WTS/) to see if I can help Jürgen, the only Babelzilla dev, with building a community around his project. Maybe some of the experience I gained through Transvision will be transferable to Babelzilla (was a one man effort, now 4 people commit regularly out of 10 committers). We'll see in the next quarters if I can help somehow, I only had time to far to install the app locally.

In terms of events, this was a quiet quarter, apart from our l10n-drivers work week, the only localization event I was in was the localization sprint over a whole weekend in the Paris office. Clarista, the main organizer blogged about it in French, many thanks to her and the whole community that came over, it was very productive, we will definitely do it again and maybe make it a recurring event.

Summary

This quarter was a good balance between shipping, tooling and community building. The beginning of the quarter was really focused on shipping Australis and as usual with big releases, we created scripts and tools that will help us ship better and faster in the future. Tooling and in particular Transvision work which is probably now my main project, took most of my time in the second part of the quarter.

Community building was as usual a constant in my work, the one thing that I find more difficult now in this area is finding time for it in the evening/week end (when most potential volunteers are available for synchronous communication) basically because it conflicts with my family life a bit. I am trying to be more efficient recruiting using asynchronous communication tools (email, forums…) but as long as I can get 5 to 10 additional people per quarter to work with me, it should be fine with scaling our projects.

Pascal Chevrel — Progressively moving from Sublime Text 3 to Atom

I am currently moving from Sublime Text (I am a happy paying customer) to Atom. I love Sublime but unfortunately, the project is dying and I want to invest in tools that are maintained in the long term.

Sublime 3 is still not release after years, the only developper completely disappeared for months and this is not the first time it happens, this is also not an open source project which means that the project will die if this only developper leaves. The support forum is filled with spam and basically, there is no roadmap nor any official commitment to pursue the project.

Of course, Sublime still works fine, but there are already reports of it getting crashy on newer version of Linux, the addons ecosystem is slowing down and in the meantime, there is Atom created by the vibrant GitHub community with regular releases and attractive features (mainly clearly copied from Sublime).

So far I hadn't switched because Atom was not ready for prime time, despite its 1.0 number. It was just not usable on my machine for any serious dev work.

That said, the recent 1.2beta builds did get improvements and many extensions are actually working around core bugs that were making Atom a no go until now. So today, I am trying to use Atom 50% of the time instead of 10% as before.

So as to get a working setup, I did have to install a few must have extensions that just make the app usable. First and foremost, I needed Atom to work on my X1 carbon which boasts a 2560*1440 resolution and well, hiDPI is not Chrome Forte, I couldn't even read the microscopic UI and using CSS to fix that was a pain and not scalable when you switch your screen to a external monitor or for a presentation with a different resolution, one point for Sublime which just respects my OS DPI settings. Fortunately, this extension fixes it all:

HiDPI

The second thing I needed was decent speed, it's still lagging behind Sublime but the recent Atom builds over the summer did make a significant effort in that respect and the boot up time is now acceptable for my daily use.

The third problem which is probably still a blocker for me is that Atom basically sucks at managing non-US Qwerty keyboards, like seriously, it's bad. Most of the shortcuts didn't work on my French keyboard and this is the same problem for most keyboards in the world. Again this seems to be a Chrome engine limitation according to Github Issues and it should be fixed upstream in a few versions of Chrome from now. In the meantime, this package is an acceptable workaround that makes many non-US keyboards work more or less with Atom:

Keyboard Localization

There is one big caveat and one of my day to day blockers, I can't comment out a single line from a keyboard shortcut, if that simple bug were fixed in the package or Atom, I would certainly use it 75% of the time and not 50%.

In terms of UI and color scheme, it took me some time to find something agreable to the eyes (On Sublime, I tended to like Cobalt color scheme) but Atom Light for the UI and Code Dark for color scheme are rather agreable. Of course I am a PHP developper, so my color scheme tends towards having clear syntax in this language. The 1.2beta builds also stopped opening context menu in silly locations on my screen instead of below my cursor, another pain point gone.

The Zen and minimap extensions are just must haves for Sublime defectors like me:

Other extensions I use a lot are available:

In terms of PHP specific extensions, I found several equivalents to what I use in Sublime:

There are other PHP extensions I installed but didn't have much chance with them yet, some require configuration files, others don't seem to work or conflict with the keyboard shortcuts (stuff like php-cs-fixer).

The one extension I really miss from Sublime is Codeintel that autocompletes method names and gives tootips explaining methods from the data extracted from Dockblocks in classes. I really wish this will be ported some day or that an equivalent feature will be created.

I also need to see how to launch simple scripts on saving files, to launch Atoum unit tests for example, hopefully somebody will hack it and I won't have to dig into doing it myself ;)

On the plus side for Atom, Git/GitHub integration in the UI is obviously out of the box, I didn't have to install any extension for that. The UI is also improving regularly and just more pleasing to the eyes than Sublime's, which is a bit too geeky to my taste. There is a real preference panel where you can manage your extensions and many interesting small features for which you don't have to edit configuration files like in Sublime (seriously, setting your font size in a JSON file is not user friendly).

It does have its share of bugs though, for example color syntaxing seems to choke on very large files (like 2MB) and everything is displayed as text (not cool for XML or long PHP arrays for example). There are also sometimes lock ups of the UI, often when I switch to preferences.

But all in all, the experience is getting better over time and I think that I found a credible alternative to Sublime for 2016. I know there are other options, I actually have PHPStorm for example which is super powerful, but just as with Eclipse, Netbeans and other heavy-weight IDEs, I have better things to do in my life than spend 3 months just learning the tool to be able to edit a couple of files and I don't buy the argument that this is a life investment ;)

The one aspect of Atom that I think is still basic is project management, I need something just as simple as Sublime but I may just have overlooked the right extension for that and anyway, it's such a popular request that I have no doubt it will be in core at some point.

That's it, if you haven't tried Atom in a while and like me are worried about Sublime Text future, maybe that post will make you want to give it a try again.

I still love Sublime, even in its beta form it is a solid and complete product and I would be happy to keep on paying for it and get updates, but unfortunately, its days seem to be doomed because all the eggs are in the same basket of a single developer that may have good reasons to have vanished, but I just need work done and there are enough bus factor people in my own job to afford having also this problem with the main tool I use on a daily basis. I'd be happy to be proven wrong and see a Sublime Renaissance, the only dev is obviously incredibly talented and deserves to make a living out of his work, I just think he got overwelmed by the incredible success he had with his product and just can't keep up. At some point, open source or prioritary software, you need a team to scale up and meet your customers satisfaction. I do hope that if he completely gives up on this project to pursue other adventures, he will open source Sublime Text and not let all this coding beauty disappear in the prioritary world limbos forever :)

PS: And yes, I prepared this blog post in Markdown syntax thanks to the built-in Markdown previewing pane in Atom ;)

Pascal Chevrel — My Q4-2013 report

It's the end of the quarter, just list last quarter I wrote down a summary of what I did this quarter for Mozilla for posterity, here it is ;)

Tools and code

Once again, I spent significant time working on tools this quarter. My activity was focused on Transvision, Langchecker and my FirefoxOS minidashboard.

There were 2 releases of Transvision, 2.9 and 3.0, adding many new features and additional support for Gaia repositories. I also created a #transvision IRC channel on Mozilla IRC server. You can now search strings for 3 locales simultaneaously, check all existing translations for an entity, list all potentially wrong varables in your repo or quickly check  all strings that need some extra QA for Firefox OS.

There were also many improvements to langchecker, the tool that I now maintain with my colleague Francesco Lodolo to track and manage progress of translations for projects using our .lang format. Many views were improved and we added features specific to the metadata used on mozilla.org (page activation and home page promos). We also added checks for UTF8 validity of the source files as well as checks for broken or missing python-style replacement variables in translations. We can also know how much of our l10n user base we cover not only per page but also per string on a page, which allows us to decide when we can activate a minor but visible text change on our pages (recently for example, html meta description and title tag changed for SEO reasons on the Firefox download pages).

As for my FirefoxOS mini dashboard (which syndicates data from the l10n dashboard, the langchecker and also contains some manually maintained data sources), it followed Gaia progresses and now tracks Gaia 1.1 and 1.2 in addition to the master branch.

Community building

This quarter I found 8 new localizers for mozilla.org and key Firefox web parts for the following locales: Afrikaans, Arabic, Basque, Bulgarian, Czech, Slovak and Spanish (Mexico). As usual I tried to focus on helping teams that lack manpower and / or maintain Firefox for a minor language.

I also created / revived IRC channels for these locales to help community building:  Catalan (#mozilla-cat), Slovak (#mozilla-sk) and Serbian (#mozilla.sr).

If we can find 5 to 10 localizers working on key content every quarter, we shouldn't have any problem growing with all of our locales together in the years to come, the thing is that we have to be proactive and look for these people and not wait for them to come to us :),

Events

The only event I went to was the Mozilla Summit, it was great and very productive from a localization point of view, I worked with Dwayne who maintains Locamotion, the Pootle instance focused on Mozilla projects for minor locales (Firefox, mozilla.org content, Firefox OS, Firefox health report) and we worked on improving our collaboration. One of the immediate results this quarter is that now we automate imports of strings for mozilla.org from Locamotion, which takes us a few minutes of work per week  and is faster for both Dwayne and ourselves. We are working on how to make it easier for locales on Locamotion to also follow mozilla.org work as this happens at a much quicker pace than product localization.

I also talked and worked with many localizers either on their specific issues (for example Brano and Wlado from the Slovak team asked me for help finding new localizers) or to check what problems a locale has and how to fix them.

Mozilla.org

A lot of work happened on mozilla.org this quarter. The most visible one is that we now have the home page available in 55 languages, versus 28 at the end of the last quarter. This is a steadily growing number, I hope that we can get the page in 70 locales, the most important is of course maintenance over time. The home page received also several updates promoting various topics and we scaled out l10n work to cover that (end of year donation promo, lightbeam, webmaker, addons). The webdev team implemented a way for us (l10n-drivers) to manage the activation of promos on the home page without code changes thanks to metadata in our lang files, that allowed us to never show mixed language content on the home page and activate these promos for locales as they get done.

Key pages in the main menu (products, mission, about, contribute) are all localizable and localized to about the same scale as the main page. Other key download pages (firefox/channels and firefox/installer-help) pages are also now translated anf maintained at large scale. Lightbeam and State of Mozilla sections were proposed as opt-in projects and have many translations (between 15 and 20 languages). I am happy that we were able to scale the localization of mozilla.org in terms of content (more pages, more content for products), number of locales and locale-specific improvements on the mozilla.org platform (better rtl support, better fonts, l10n friendly templates..), and our management tools to allow us to grow.

Another cool l10n feature that happened on the site was the creation of a 'translation bar' proposing you a mozilla.org page in your language if it exists. This was entirely done by a volunteer Mozillian, Kohei Yoshino, many thanks to him for this cool feature that may expand to other Mozilla sites ! Kohei wrote about it on the webdev blog. It is really cool to see improvements brought by volunteers and it is also cool to see that many people in the webdev team are also acquiring an l10n culture and often spot potential problems before myself or flod get to them !

That's all for this quarter, overall an excellent quarter for mozilla.org and tools which improve steadily. On a side note, it is also an excellent quarter for me at a personal level  as my daughter was born last month (which explains why I am less often connected  these days ;) )

Pascal Chevrel — My Q3/Q4-2014 report

I didn't write the Q3 report, so this report covers both Q3 and Q4.

Regular l10n-drivers work such as patch reviews, pushes to production, meetings and past projects maintenance excluded.

Transvision

New release

We released version 3.5 of Transvision and I worked on these features and bug fixes:

  • Improved the setup process for devs, just launch start.sh after cloning the repository and it will set up everything and launch a Transvision instance locally. Setting up Transvision locally for development takes now about 20mn with a fast DSL connection.
  • Fixed a bunch of regressions in our external API that our Galician localizer Enrique Estévez, aka Keko, reported.
  • Worked with Keko on adding a new translation memory export format which allows importing an existing Mozilla localization effort supported by Transvision into OmegaT (an offline localization tool in Java).
  • Worked with my intern Théo on his Locale status view, our numbers are mostly consistent with the official l10n dashboard but with a different aggregation approach that will allow to expose strings by product and category (/ex, expose the status of devtools localization in Firefox, or group Firefox strings with Firefox content hosted on mozilla.org). Still a lot to do on the perf side for this page (http://transvision.mozfr.org/showrepos/) since it relies heavily on regexes on 10s of thousands of strings (page load is about 15s without caching, 4s with caching). Théo will do another internship next summer with us, so stay tuned.
  • Improved extraction of strings to not escape systematically special characters and html at string extraction. That allows searching for strings with html markup or reserved characters in HTML (for example: &BrandShortName or searching for \n codes in strings.
  • All commits to master are now continuously integrated on the beta site via GitHub webhooks (beta site had to be manually updated before)
  • Many smaller improvements to exiting views

Work in progress related to Transvision

  • Created a standalone library to be used initially in Transvision to extract all changesets from hg/git/svn repositories in the same format, so as to be able to dig contribution data in a uniform way
  • Working on an internal directory built dynamically from our logs, coupled with the vcs library it will allow us to provide our core localizers stats to the CBT team via an external API. Also works as a CLI tool for my own use.
  • Working on a Product class for Transvision that allows to query strings per software and not per repository (important for desktop because there are a lot of strings shared between Firefox and Thunderbird).

Overall, I worked a lot on Transvision during Q3 but very little during Q4. There was a good balance of improvements targetting our different users (localizers, Transvision devs, API consumers) and we did some solid improvements to both the code and our workflow to ship faster in 2015. We also had 8 people contributing to the codebase over the semester and several people popping in our IRC channel asking about the project, which is good. I have several big patches with good potential in branches but unfortunately I didn't have time to work on finishing them in Q4 and it seems I won't be able to work on them either in Q1, maybe in Q2.

Langchecker and other tracking tools we use for management

I found a way to generate data about web parts localization over time from Langchecker (Langchecker was not meant to do that, so I had to be creative), so now we generate graphs that show the history of web parts completion per locale: example with Bulgarian showing what happens when you have a new high performer localizer joining Mozilla

I updated the overview status showing the number of untranslated strings per locales with a summary ranking locales status and also showing locales on locamotion it also now counts strings for all projects on the wbedashboard, not just mozilla.org.

I added the number of strings, words and files to the summary view since this is something we got asked for often from other teams.

Google Play copy localization back into a normal l10n process

Historically, the process of publishing updates to the Firefox page on Google Play has been owned by many different people, most if not all of them no longer working for Mozilla. So there was no real formal process to update the copy and get that copy translated. Sylvestre Ledru (Debian core dev that recently joined the Mozilla Release Management team) decided to establish a process with as much automation as possible via the Google Play API to update our copy in English when we ship a new release and I decided to also help on this front to get that content localized and published simultaneaously with English.

So now the process of translating the copy is back to the l10n-drivers team with our own tools (means tracking, dashboards, integration to the translation platforms we use…).

I created a small app to track and QA translations without pushing to Google Play.

And I also created an associated JSON API that release drivers can use.

I am working with Sylvestre on getting release drivers tools automatically update our localized listings along with the en-US copy. We hope to get that working and functionnal in Q1 and be able to always have an updated copy for all of our supported locale for Firefox on Google Play from now on.

Thunderbird web localization migrating to Bedrock (mozilla.org)

This is localization management work that I am doing as a volunteer as Mozilla does not put paid resources on Thunderbird. My colleague Flod is also helping, as well as Théo (my intern and French localizer) and Kohei Yoshino (ex Mozilla Japan webdev and ex mozilla.org localizer, now living in Canada and still contributing to www.mozilla.org at the code level).

Thunderbird web content is still hosted on the old mozilla.org framework, this content is not actively maintained by anybody and that framework will eventually disappear. The l10n process is also broken there. For Thunderbird 10th anniversary, I volunteered to help manage the key content for Thunderbird on our current platform (Bedrock) so as that Thunderbird in-product pages can get updated and translated. If you use Thunderbird, you may have seen the new version of the start page (some people noticed).

A few notes though:

  • If you are a Mozilla localizer but are not interested in translating Thunderbird content, please don't translate it even if you see that on your locale dashboard, help us find a person to do these translations for Thunderbird in you language instead!
  • I am not volunteering to manage Thunderbird product localization, I just don't have enough time for that. If you want to help on that front, please contact the new Thunderbird team leaders, they will be happy to get help on that front!
  • 23 locales have translated the page, thanks to you guys! If your locale is not done and you want to get involved in Mozilla Web localization, please contact me! (pascal AT mozilla DOT com)

Community building

I try to spend as much time as I can actively finding new localizers and growing our localization community with core contributors that help us ship software and projects. It's great to build tools and we are always happy when we can improve by 20% the productivity of a localizer with a new process or tool, but when we find a new core localizer for a language, we get a lot more than a 20% productivity boost ;). After all, we build tools for people to use them.

Here is a list of countries where I found new core localizers:

Portugal:
  • We now have a webmaster for mozilla.pt (hosted on mozfr.org), Joao, a Portuguese volunteer involved in the Rust language
  • Luis Mourão joined the team as a technical localizer
  • Claudio Esperança, is now an svn committer
Bulgaria
  • New Bulgarian web localizer, Miroslav Yovchev. Miroslav did an awesome job with one of his friends getting mozilla.org which was very outdated in Bulgarian in a decent shape. He also corrected many existing pages for consistency of style.
  • I met with my colleague Pavel Ivanov in Portland who is Bulgarian and works on Gaia so as that we can see how we can help grow the Bulgarian community next year, not just in l10n.
Latvia

New Latvian localizer for webparts: Jānis Marks Gailis. Latvian does not have many speakers and historically we had only one localizer (Raivis Deijus) focused on products, so now we can say that we have a team spanning both products and web sites. A third person would be welcome of course :)

Croatia

New Croatian localizer Antun Koncic. Antun used to be the Croatian dictionary packager for Thunderbird and was no longer involved in Mozilla, welcome back then!

Poland
  • We have a new Polish localizer, Piotr Nalepa that is helping us with web content
  • Joanna Mazgaj, long term Polish contributor in aviary.pl is also now a committer for mozilla.org

We could use the help of a couple more people for Polish, especially to translate products and technical content, if you are interested, please leave a comment :)

Events I attended

I went to the Encontro ibérico de tradutores de software libre a linguas minorizadas in Santiago de Compostela, Spain. I met there several of our Mozilla localizers for Catalan, Galician, Basque, Aragonese and Asturian. We talked about processes, tools and community building. Was very interesting and great fun.

Like a thousand more Mozilla employees and many volunteers, I spent a work week in Portland, USA where I met several localizers we had invited. It was the first time I was meeting some of these localizers IRL, in particular people from India and Bangladesh. That was nice.

Privacy Coach Add-on

In parallel to the 10 years of Mozilla, we created and launched a Firefox for Android extension called Privacy Coach teaching users how to set up privacy settings in their browser.

I worked with Margaret Leibovic on that project and I reused the code and process that I had used for the add-ons we had created during the Australis launch earlier this year to translate the add-on while it was being developped. The idea is that it shouldn't be discruptive for either localizers or developers.

We worked with .lang files that were automatically converted to the right set of dtd/properties files for the extension along with the packaging of the locale into the extension (update of install.rdf/chrome.manifest/locale folder) that were integrated into the extenson via pull requests on github during its development. We got 12 locales done which is more than the 8 locales initially planned. Two locales we wanted to get we didn't in the end (Hungarian and Indonesian) so they are part of the locales I will work with in Q1 to get more people.

Various Web projects

  • Firefox Tiles: prepared the project as a .lang one with Mathjazz, prepopulated translations and launched with Mathjazz for all locales
  • Worked with flod on New home page and new contribute pages for mozilla.org
  • 10 years of Mozilla: Flod was the lead on this project and blogged about it in his own Q4 report, basically my main role there was to help find localizers for the project, all the merit of the project from the l10n POV goes to flod :)
  • Removed 9 locales no longer supported in our products from mozilla.org (ak, csb, lg, mn, nso, sah, sw, ta-LK, wo)

Other

  • I created a GitHub organization for our team and moved several of my projects there. I felt that we needed to have a place for us to have all the tools we use for shipping in one common place where we could create projects whenever we need them instead of using our personal accounts.
  • I transferred the womoz.org domain to Flore Allemandou now leading the Women in Mozilla community (I had been paying for it since 2009) and Flore is working on getting the domain owned and paid by Mozilla
  • I helped the French community (with Théo and Natalia from the Marketing team) set up a promotional Firefox OS site focused on end-users (created the github project, did a temporary version of the site until the French community built a design and content to replace it and took care of setting up some continous integration to allow people to publish code and text updates without me merging and pushing to production).

Pascal Chevrel — My Q1-2014 report

Here is what I did in Q1

Tools and code

This quarter we have made 2 releases of Transvision (3.1 and 3.2). For 3.1, most of the credit goes to my colleague (and Italian localizer) Francesco Lodolo who integrated his productization scripts into the app and worked on redesigning the menus and making the app better in responsive mode, my work on 3.1 was limited on performance patches and release management. Transvision 3.2 received a lot more of my code as it integrated one major new feature (planned since I took the project under my umbrella two years ago), which is indexing all translations of our flagship website www.mozilla.org. So now Transvision is no longer a search engine and QA tool for all of our products translations, it also supports websites using the .lang format (next up is indexing websites using gettext). From a developper perspective, this is also the release in which we automate the generation of the documentation of our classes thanks to phpDocumentor. Currently I am working on a better MVC approach and organization of code so as to work on features such as atom feeds, on the fly generation of TMX files and other more flexible export features, it may be in addition or in replacement of the TMX files we generate at extraction time. Caching is also in the pipe, the app is currently very fast and responsive but caching could help us extend more our external API to more time consuming data generation.

A personal achievement is that I finally rewrote my old Web Dashboard (whose code dated back to 2007) and moved that to github. Two new features, direct query of web localization bugs per locale on Bugzilla and display of the translation status for most of the projects hosted on Verbatim, our web localization platform, the latter was a patch by Francesco. The big feature is that the code went from 6000 lines of spaghetti code to 400 clean code, old features that made sense 7 years ago and now useless were removed and the fact that we can now track www.mozilla.org work in Bugzilla per locale allowed cutting a lot of code. Pretty happy that this was totally transparent to the user and that we are already starting to add new features to help us ship software!

On the Langchecker side, the tool we use to track .lang based projects and deploy lang files on repositories, I added per locale json feeds for the status of the locale and it now warns about invalid metadata and can now removes obsolete or invalid tags that we use on mozilla.org to activate a feature for a locale on a page. We also used it to expose strings from our first project based on Pontoon.

Other small tools of mine got updated, my mass bug filing script can now file bugs marked as mozilla confidential, something we needed for Mobile World Congress pages, I whipped up a script using Transvision data to know the distribution of strings across components in Firefox/FirefoxOS which allows specifically to know how many strings we have for devtools (potentially a future Transvision feature) etc.

All in all, I'd say I spent about 30% of my time this quarter hacking on code, which is cool.

Events and community

This quarter I attended one event outside of Paris, Fosdem. As usual it was interesting in terms of networking with European open source hacktivists. I also organized a one day event in the Paris office with April which mostly consisted of workshops on how to contribute to an open source project (I led two workshops on participating to Mozilla localization). I also attended 3 events in the Paris office, one with the French community to welcome half a dozen new volunteers, one with Rosana and Goofy dedicated  to SUMO localization and a last one which was an impromptu meeting between the French community and Brian King from Reps. With a Paris office tailored for meetings, I expect that I will continue to organize / attend to community meetings locally instead of travelling a lot like in the past, that's more compatible with my recent fatherhood :)

This wasn't a big quarter in terms of finding contributors, I can identify only 2 new people that became active and productive in the Mozilla l10n project directly because of me, that was mostly due to the fact that I spent more time on shipping projects and building tools this quarter. I also had some more paperwork than usual to deal with as I have an intern starting in April and I also worked with my colleagues on a tax refund paper for Research and Development in the Paris office.

I spent some specific time helping the Lithuanian community trying to grow in general, we created an IRC channel, opened a mailing list for Lithuania, did some work on etherpads  and contacted other open source projects in Lithuanians, past contributors, would be contributors through the Contribute page... We found some people interested in helping with l10n but more work will be needed to reboot the Lithuanian community (not just in l10n).

I also closed the mozilla.community.europe which had been created after we closed Mozilla Europe and merged its activities with Mozilla, it unfortunately never attracted any activity and was just collecting spam (which I had to delete on a daily basis).

About the Transvision community, the project reached 9 committers in March, is now listed on What Can I Do for Mozilla (which makes it the only active PHP project listed on the site ;)). It is growing slowly but steadily on all fronts (uses, codebase, contributors, features...). We are also working on making it easier to hack (static and automatic documentation, better installation scripts...) but one of the roadblock is the amount of data that needs to be downloaded first to make it useful locally, the sum of our hg/git/svn repos are about 17GB big. Even after extraction of strings from all of our repos, it is probably like 200MB of text, we may evaluate a downloadable snapshot of that in the install process to make it easier for contributors that want to hack on it.

From a tooling community perspective, big thanks to Enrique Estevez, our galician localizer, who is working on upstreaming patches to OmegaT to make it work better with Mozilla formats (.properties and .lang) and who is also writing an Omega T plugin to get translation suggestions from Transvision. Babelzilla also contacted me for similar purposes, that means that at least four Mozilla tools use it or plan to use it directly (Mozilla Translator, OmegaT, Pontoon, Babelzilla). We will put a focus on improving, documenting and versionning our external API this year to make it easier to the tools community to integrate it.

Mozilla.org localization

On Mozilla.org, the big deal this year was like last year the Mobile World Congress event, we updated all of our Firefox OS pages and home page + tabzilla in 17 languages in time before the event and also added some extra content for a dozen of these locales. As usual with big projects, we exposed feature needs and we now can publish promotions in tabzilla per locale, using the same mecanism as for the home page, so as to target only the languages that we were working on for MWC.

MWC was not all though, we also worked on a new Pricacy page for a few locales, we worked on a landing page for Firefox Metro, but unfortunately the product got cancelled, we updated a lot of older content, made patches in our templates to avoid duplicated strings, shipped new locales on the release channel with all o their web content done (Xhosa and Aragonese) and shipped Malay on the Beta and Aurora channels too.

The amount of strings is now around 1500 for a big locale on mozilla.org and about 600 for a small locale. Of course, since we have frequent content changes (promos, minor string updates on tabzilla, download pages, in-product pages...), this is actually more, especially for the bigger locales and Firefox OS locales, but I am happy that we succeed in keeping all of the locales in good shape, especially the small ones, while not slowing down the creation of English content. The site is definitely getting better and more localized month after month, two years after the switch to Django and the total rewrite, this is starting to look really good again.

And now Q2!

Expect Q2 to be more of the same :) One interesting point is that I am taking Theo Chevalier, our main French localizer, for an internship at Mozilla, which means that he will work on all of the cool stuff I have in mind but never get the time to do (like working on growing our l10n Firefox Nightly userbase which is ridiculously low compared to en-US), he will also of course work on the big Q2 project that is the launch of Firefox Australis!

Karl Dubost — Filter your mail in a dated space server side with Sieve

When it comes to sort out your emails, there are many strategies. Since I have been working at W3C, I'm a fan of dated spaces. I apply this strategy to my emails using Sieve.

mailbox stuffed with old letters and papers.

Dated Space?

A dated space is a way to organize the information by date folders. Here is an example on my folders.

organization by dates of folders.

and here the same type of organization for mails.

organization by dates of folders.

Why would you do that?

  • It creates a unique space. The arrow time is irreversible so there is no issue with creating new folders.
  • It limits the number of items by folders. When a folder has too many items, it becomes harder to find what you need.
  • It is actually helpful, because our memory has a lot of anchors in time and we can find easily stuff by remembering when we created it. (a location-based could be an interesting experiment and has useful applications in some circumstances such as photos for example, but that's for another blog post.)
  • OS search engines: Smart folders based on keywords, types, etc. will make it easier to find stuff. And the files can now belong in multiple contexts. Everything everywhere all at once.

Dated Space For Mail With Sieve

Sieve is a language for filtering e-mail messages. It resides on the server side. So it requires that your mail provider accepts that you can manage sieve rules for your own mail.

I filter all my incoming mails indifferently of their nature to a dated space. Everything is mixed. And I restructure my emails folders with smart mailboxes. Then I can suppress folders which are not valid anymore, create new ones, all of this without touching any emails.

Sieve Script

require ["fileinto", "date", "mailbox", "variables"];
# filter emails based on the date and move them to a folder.
# mail from June 2022, will be saved into /2022/06
# set values for month and year
if currentdate :matches "year" "*" { set "year" "${1}"; };
if currentdate :matches "month" "*" { set "month" "${1}"; };
# mv the message into the right mail folder
fileinto :create "${year}/${month}";
stop;

It's a very simple script and works like a charm. Sieve is a very powerful language. It's possible to do all kind of classifications.

Oh… and yes… my Inbox is obviously by the nature of the script always… 0.

Otsukare!

Frédéric Wang — Update on OpenType MATH fonts

I mentioned in a previous post that Igalia organized the Web Engines Hackfest 2022 last week. As usual, fonts were one of the topic discussed. Dominik Röttsches presented COLRv1 color vector fonts in Chrome and OSS (transcript) and we also settled a breakout session on Tuesday morning. Because one issue raised was the availability of OpenType MATH fonts on operating systems, I believe it’s worth giving an update on the latest status…

There are only a few fonts with an OpenType MATH table. Such fonts can be used for math layout e.g. modern TeX engines to render LaTeX, Microsoft Office to render OMML or Web engines to render MathML. Three of such fonts are interesting to consider, so I’m providing a quick overview together with screenshots generated by XeTeX from the LaTeX formula $${\sqrt{\sum_{n=1}^\infty {\frac{10}{n^4}}}} = {\int_0^\infty \frac{2x dx}{e^x-1}} = \frac{\pi^2}{3} \in {\mathbb R}$$:

Recently, Igalia has been in touch with Myles C. Maxfield who has helped with internal discussion at Apple regarding inclusion of STIX Two Math in the list of fonts on macOS. Last week he came back to us announcing it’s now the case on all the betas of macOS 13 Ventura 🎉 ! I just tested it this morning and indeed STIX Two Math is now showing up as expected in the Font Book. Here is the rendering of the last slide of my hackfest presentation in Safari 16.0:

Screenshot of a math formula rendered with STIX Two Math by Safari

Obviously, this is a good news for Chromium and Firefox too. For the former, we are preparing our MathML intent-to-ship and having decent rendering on macOS by default is important. As for the latter, we could in the future finally get rid of hardcoded tables to support the deprecated STIXGeneral set.

Another interesting platform to consider for Chromium is Android. Last week, there has been new activity on the Noto fonts bug and answers seem more encouraging now… So let’s hope we can get a proper math font on Android soon!

Finally, I’m not exactly sure about the situation on Linux and it may be different for the various distributions. STIX and Latin Modern should generally be available as system packages that can be easily installed. It would be nicer if they were pre-installed by default though…

The Mozilla Blog — Queer Singer/Songwriter Criibaby Finds Pride and Joy at the Intersection of Tech and Music

Here at Mozilla, we are the first to admit the internet isn’t perfect, but we are also quick to point out that the internet is pretty darn magical. The internet opens up doors and opportunities, allows for people to connect with others, and lets everyone find where they belong — their corners of the internet. We all have an internet story worth sharing. In My Corner of the Internet, we talk with people about the online spaces they can’t get enough of, what we should save in Pocket to read later and what sites and forums shaped them.

With Pride celebrations taking place throughout June, we’re featuring LGBTQ+ leaders this month as part of our My Corner of the Internet series. In this next installment, queer music artist Criibaby shares her love of the intersection of music and tech, her process to creating music and how her art extends to shoe design.

What is your favorite corner of the internet?

I have a special place in my heart for a website called soundtrap which is basically an online software that lets you record songs and collaborate with other artists from anywhere you can connect to the internet. Before I had access to a real recording studio and long before I ever released any music as Criibaby, I was writing demos at my kitchen table after work. It reminded me of when I was a kid when I would spend hours after school making mashups and recording songs on garageband. In my early twenties, I was going through a pretty hard time processing grief, and getting in touch with my inner child in that way felt, well, just right. I have tons of voice memos on my phone, probably into the three digits at this point, and soundtrap was one of the first places I could put them all together and get back into multi-track recording. Years ago, I sent a soundtrap demo I’d recorded on my phone to this guy called Pandaraps who was making really inspiring, introspective queer lofi hip hop. Thanks to that demo, I got an invitation to the studio he was recording at, and was introduced to my now-producer Surfer Dave. 

What is an internet deep dive that you can’t wait to jump back into?

I always try to tap into Kiefer and knxwledge’s twitch streams whenever a notification pops up that they’re live. Those are two producers that I really look up to, so it’s cool to go behind the scenes with them and learn about their creative process or hear what they’ve been working on lately. I’ve also been learning a bit about web3 stuff in discords like Mashibeats hosted by Marc De Clive Lowe, and StemsDAO which hosts a producer remix game that I’m hoping to submit to once I finish the LP I’m in the middle of writing. I’m super interested in the intersection of music and tech, so while I’m totally new to that scene, I’m curious to see where it goes.

What is the one tab on your browser you always regret closing?

Any Bill Wurtz video. Watching one always leads to watching 100, so it’s a bit of a rabbit hole. But with those catchy melodies and ridiculous graphics, how could you not get sucked in for like an hour at a time? Someone asked me if I consider myself a Bill Wurtz superfan and I’d say the answer is a resounding yes. A few years ago I recreated the Converse from the Bill Wurtz classic “Christmas isn’t real” and gave a pair to my partner as a birthday present. Not gonna lie, I’m pretty proud of that. That’s peak gifting. 

Who is an LGBTQ+ person with a very online presence that is a role model for you?

Two words: Queer memes. Shout out to instagram meme dealers @hypedykes, @tofuminati, @themmememes and @bi_astrology — I’d call this niche “relatable queer chaos” and their memes are truly some of the best in the game. This breed of hyper-specific, exaggerated surrealist queer commentary that pulls no punches and hits the nail directly on the head brings me endless joy. It’s weirdly affirming to know my emotional rollercoaster of a life is right on track, according to these absurd lil windows into queer life circa 2022.

What can you not stop talking about on the internet right now? 

My new Criibaby EP “crii” came out on June 10 so I’m grateful for these songs to be out in the world! One thing that distinguishes me from other indie artists is that I write all my songs without any gender-specific lyrics. No binary pronouns, no heteronormativity, all written from an openly queer perspective as part of a conscious effort to create more inclusive music. That was the intent behind my debut popsoul EP love songs for everyone and I got some really sweet notes from fellow queer folks who said it really resonated with them and made them feel seen, so I’m excited to be releasing more gender-neutral music. (It also premiered in Billboard and played on BBC 6 Music, which was crazy validating! Turns out critics can be nice??) 

With love songs for everyone I wanted to send a message that no matter how you identify, this feel-good, empowering music is for you — and you are welcome here. Exist! Freely! My new EP is pretty different —- for starters it’s a lot more personal, and some of it comes from a bit of a darker place because that’s just honestly where I was at when I wrote these songs. And I didn’t want to hide from those emotions. I think as queer people, we feel all this pressure to put on a good face as a way to say, hey, I’m queer and I’m thriving! But sometimes you’re just not ok… and that’s ok. I guess that’s my main message with this EP: frankly, you don’t always have to have your sh*t together. Which is something I’m still trying to learn for myself. Sonically it’s inspired by a mix of neo soul, lofi, jazz, and alternative aesthetics, but I wouldn’t really define it as any of those. As someone who is often assumed to be straight based on how I appear and present, I want to show the world that queerness doesn’t have any one look, sound, or genre. They’re some of the best songs I’ve ever made and I’m really proud of them. I hope you check it out and let me know what you think!

What was the first online community you engaged with? 

Yikes, this is really embarrassing, but I think it would have been Yahoo Answers. If you don’t know what that is (good), it was basically like Reddit on training wheels, but … worse than whatever you’re imagining? I can’t remember everything I posted (thank god) but I almost certainly asked strangers on the internet if they thought I might be gay. Good times. 

I also got really into Daytrotter, a website for this studio in Illinois that hosted recording sessions with indie artists, kinda like NPR’s Tiny Desk, before that was a thing. I found a lot of new music from people I’d never heard of that way, and this was during my absolute raging hipster phase, so it fueled a ton of music discovery. In classic hipster fashion, I was really into listening to stuff my friends hadn’t heard of. That and Radiohead, tons of Radiohead. 

What articles and videos are in your Pocket waiting to be read/watched right now?

Them released a new queer music monthly that I’m excited to read up on and listen through. I hope to be included in one of those someday — it would really mean a lot to me to be recognized by that magazine. 

If you could create your own corner of the internet what would it look like?

I’m working on it! Right now I’m building up a Patreon-style group of VIP fans that get free merch, secret listening parties ahead of my release dates, and eventually backstage access to my live shows. DM me on instagram if you want to come along for the ride— it’s free, and shaping up to be a really sweet community!

Rising queer artist Criibaby invites listeners to look deeper within themselves with her genre-bending, intimate music. Using a unique songwriting method she describes as possessing “no names, no genders, only the feelings, only the bare truths,” Criibaby’s songs contain only lyrics without binary pronouns, written from an openly queer perspective in a conscious effort to create more inclusive music. Drawing upon neo soul, lofi, jazz, and alternative aesthetics to form something entirely her own, Criibaby’s intricate layers of delicate, emotional vocals and emotionally bare lyricism come together to form a new kind of introspective sonic escapism. You can find her on Instagram, Twitter, LinkedIn and Facebook and can listen to her new EP “crii” on all streaming platforms.  

Save and discover the best articles, stories and videos on the web

Get Pocket

The post Queer Singer/Songwriter Criibaby Finds Pride and Joy at the Intersection of Tech and Music appeared first on The Mozilla Blog.

The Mozilla Blog — Firefox Presents: Redefining love and identity, one viral video at a time

Every Monday and Thursday, queer comedy writer and actor Brandon Kyle Goodman asks their 176,000 Instagram followers to tell them something good or messy. 

People who tune into the Instagram Stories series, called “Messy Mondays” and “Thot Thursdays,” send in submissions that are indeed good, messy or both. Some share their kinks. Others inquire about techniques they want to try in bed. One asked for advice for people who struggle with getting in the mood. Another confessed to having slept with their therapist.

In turn, Brandon asks their own questions (“what’s your favorite phrase to tell someone you want them?”), shares musings (“I think ‘Babe’ was a gay movie. I think it was my gay awakening”), gives smart advice (“find a new therapist”) and does demonstrations. NSFW memes are deployed. 

One can’t help but laugh along with Brandon, who also writes for and voices the queer character Walter the Lovebug in the Netflix animated show “Big Mouth” and its spinoff, “Human Resources.” They’re also the author of “You Gotta Be You,” part memoir and part humorous self-help guide out this September, that Brandon hopes will help people like them — who’s nonbinary and Black — feel less alone. 

This intention is also what’s behind Brandon’s social media series, which, while irreverently funny, can also turn tender. 

Once, Brandon’s husband made an appearance and shared his new fondness for being kissed in his stomach, an area he used to be sensitive about. 

Sitting next to him, Brandon said “this is why I love talking about sex, because of the ability to reclaim your body and your sexuality.” 

Brandon makes periodic reminders that their account is a safe space, where people can be joyous about queer sex. 

Brandon Kyle Goodman and their husband Matthew sit next to each other, smiling in front of a phone and a ring light.<figcaption>Photo: Nita Hong for Mozilla</figcaption>

The 35-year-old first gained notice on social media during a particularly dark time in the world. In 2020, the comedy writer had virtually gone back to work after a lunch break, during which they watched the video of a white police officer kill George Floyd.

“I logged back onto my Zoom and sat there for the next three or four hours pitching jokes with my non-Black and white colleagues,” Brandon said. “No one had mentioned it. It had just happened, and so they might not have seen it. But it was wild to me that I was sitting there, not speaking about it and continuing my day as if I didn’t just watch this Black man get murdered for nothing.”

Afterwards, Brandon posted a 7-minute video on Instagram addressed to “my white friends.”

“These are my thoughts and I hope it will compel you and yours to be ACTIVE in the fight for our Black lives,” the caption says. 

Brandon went to sleep after making the video. When they woke up, the clip had nearly a million views. Their followers had jumped to 30,000. 

“I remember being in my living room and feeling my grandmother, who was a minister, and feeling ancestors, whose names I don’t know,” Brandon said. “It was a question of, you can completely ignore this and go about your business. Or, you can step into this. I heard very clearly, ‘Step in. You can do this.’”

At first, Brandon hesitated identifying as an activist, “not wanting to take away” from organizers who march and lead rallies on the streets. But they realized that activism has always been laced into their work. 

“Viola Davis said, ‘Me just showing up as a dark-skinned, Black woman on your camera is activism.’ I think that’s true, too,” Brandon said. 

They continued to make videos for social media and started “Messy Mondays” because they needed a break from the heaviness.

“When you think about sex education, it’s trash in our country,” Brandon said. “It doesn’t include queer people at all. Here was this chance to redefine that.”

They added that when it comes to sex, it’s about the whole person. 

“Your Blackness, your queerness, your womanhood, your nonbinaryhood, your manhood. It’s about you,” Brandon said. “This is part of my activism, my relationship to sex. And so for me to be a part of a collective of influencers and educators and activists and people who are boldly talking about it is so liberating. I wish that young Brandon had that.”

Brandon Kyle Goodman dances.<figcaption>Photo: Nita Hong for Mozilla</figcaption>

Brandon was raised by their mother and grandmother in Queens, New York. They attended boarding school in Georgia and the Tisch School of the Arts at New York University before moving to Los Angeles to pursue a writing and acting career. They didn’t come out as gay until they were 21 years old. 

“Growing up Black, queer, nonbinary, there’s so much messaging that there’s something wrong with me,” Brandon said. “That’s in addition to the fact that when you turn on TV or read books in school, there’s just not a Black protagonist, let alone a Black queer protagonist. So it became really important to reclaim and refocus and reframe that to ‘I am enough. There’s nothing wrong with me. Even if the world is not ready to see me or hold me, I have to see and hold myself.’”

Brandon said their mother, who became deeply religious after their grandmother died, could not accept their queerness. The two haven’t spoken in 10 years.

Blood family may not always be capable of taking you through your human existence, Brandon said. But they found their own family and community, including online. 

The internet has its dark side, Brandon acknowledged, but it can also be a glorious place. Their advice: “Just remember that you are enough.”

Firefox is exploring all the ways the internet makes our planet an awesome place. Almost everything we do today ties back to the online world in some way — so, join us in highlighting the funny, weird, inspiring and courageous stories that remind us why we love the world wide web.

Brandon Kyle Goodman smiles at the camera.

Get the browser that makes a difference

Download Firefox

The post Firefox Presents: Redefining love and identity, one viral video at a time appeared first on The Mozilla Blog.

Frédéric Wang — Short blog post from Madrid's hotel room

This week, I finally went back to A Coruña for the Web Engines Hackfest and internal company meetings. These were my first on-site events since the COVID-19 pandemic. After two years of non-super-exciting virtual conferences I was so glad to finally be able to meet with colleagues and other people from the Web.

Igalia has grown considerably and I finally get to know many new hires in person. Obviously, some people were still not able to travel despite the effort we put to settle strong sanitary measures. Nevertheless, our infrastructure has also improved a lot and we were able to provide remote communication during these events, in order to give people a chance to attend and participate !

Work on the Madrid–Galicia high-speed rail line finally completed last December, meaning one can now travel with fast trains between Paris - Barcelona - Madrid - A Coruña. This takes about one day and a half though and, because I’m voting for the Legislative elections in France, I had to shorten a bit my stay and miss nice social activities 😥… That’s a pity, but I’m looking forward to participating more next time!

Finally on the technical side, my main contribution was to present our upcoming plan to ship MathML in Chromium. The summary is that we are happy with this first implementation and will send the intent-to-ship next week. There are minor issues to address, but the consensus from the conversations we had with other attendees (including folks from Google and Mozilla) is that they should not be a blocker and can be refined depending on the feedback from API owners. So let’s do it and see what happens…

There is definitely a lot more to write and nice pictures to share, but it’s starting to be late here and I have a train back to Paris tomorrow. 😉

The Mozilla Blog — Kids are growing up in a very online world. What’s a concerned parent to do?

Technology is easy to blame for the mental health crisis that kids are facing. But according to experts, it’s not that simple. 

A rare public advisory from the U.S. surgeon general in December 2021 warned that young people are facing unprecedented challenges that have had a “devastating” effect on their mental health. These difficulties were already widespread before the pandemic started in 2020 — with up to 1 in 5 people in the U.S. ages 3 to 17 having a reported mental, emotional, developmental or behavioral disorder. 

We often attribute the crisis to technology, particularly social media. After all, young people today are spending much of their time glued to screens like no generation before them. One study conducted in 2021 found that teens spent 7.7 hours per day in front of screens for activities unrelated to school. But there is not a definitive correlation between mental health and social media use.

Over 7 hours of screen time may sound excessive. But as more and more of life moves online for people of all ages, it is to be expected. Technology brings with it challenges and opportunities for today’s youngest generations. Even as its use rises and youth mental health declines, researchers haven’t found a clear link between social media and mental health but instead a lot of factors of modern life, including technology, that interconnect.

What researchers are learning about social media and kids’ mental health

There’s been a lot of research on the subject over the last decade, but we still haven’t learned anything conclusive.

Amanda Lenhart, a researcher who focuses on how technology affects families and children for the Data & Society Research Institute, said studies generally show a slight negative impact of social media on people’s well-being, or none at all.

“We have this narrative about it being a negative thing, but we’re not seeing it in the data,” Lenhart said.

In a project by the Universities of Amsterdam and Tilburg, researchers have been exploring the relationship between adolescents’ social media use and their well-being. Early data from that project, Lenhart said, suggests that about 70% don’t feel any different after using social media; 19% feel a lot better; and 10% feel worse.

“I think one big takeaway is that there’s a subgroup of people for whom using these platforms is not great,” Lenhart said. More research is now focusing on identifying this group and what their experience is like.

Lenhart said results are showing a model called differential susceptibility, meaning different people can have different responses to the same platform.

For example, a child of color may have a more negative experience on a social media platform than others because he or she faces harassment. A platform could also worsen feelings of depression for someone who’s already depressed.

“It’s not that a platform always makes people feel worse or better, but that it exacerbates their highs and exacerbates their lows,” Lenhart explained.

Experts are concerned that social media platforms are designed with adults as the user in mind, grabbing their attention and keeping them scrolling. Tech companies can do a lot better with content moderation and ensuring age-appropriate and health-promoting content, said Michael Robb, who for the last 15 years has been researching the effects of media and technology on children’s development. He’s ​​currently the senior director of research at Common Sense Media, a U.S. nonprofit that advocates for internet safety for families.

“Expecting that you could iterate and iterate when you’re dealing with a sensitive population without having given real care to what’s going on developmentally can be irresponsible,” Robb said, adding that the concept of “move fast and break things” in the tech space should not be applied to children. 

Lenhart expressed the same sentiment, pointing to Snapchat’s Snapstreaks feature, which encourages two people to send each other a photo or video snap everyday to keep up a streak. 

“I think when they were built, the idea was this was going to be a really fun thing that would be a chance for people to feel connected, get people coming back every day, “ Lenhart said. “But I think they failed to realize that in particular contexts, particularly among young people, peer relationships are very important and intense.”

In some instances, the feature resulted in unproductive behavior and an obsession with making sure to keep up the streaks. People would give other people their passwords before they couldn’t send snaps when they go places without reliable internet or when they were sick.

“It became a thing that was very agitating and disturbing for a subset of young people,” Lenhart said. 

How social media affects people’s well-being could also depend by age, according to a large study conducted in the U.K. from 2011 and 2018. It identified two periods when heavy social media use predicted a drop in “life satisfaction” ratings a year later: 14-15 and 19 years old for males, and 11-13 and 19 years old for females. The inverse was also true: Lower social media use in those periods predicted an increase in life satisfaction ratings. 

The study argues that by looking at the issue in a developmental lens, research could make “much needed progress in understanding how social media use affects well-being.”

A ‘lifeline’ for many young people

During the pandemic, social media has played an outsized role among young people seeking connection and help with their mental health. 

A study from Common Sense Media found that social media and other online tools concluded that they’ve “proven to be the lifeline that many young people needed to get through this last year.”

The study surveyed 14- to 22-year-olds across the U.S. from September to November 2020. More than 85% of them went online in search of health information, 69% used mobile apps related to health issues, 47% connected with a provider online and 40% sought people experiencing similar health concerns. 

From 2018 to 2020, more teens and young adults reported relying on social media platforms for support, community and self-expression, the study found.

A table shows the results of a survey by Common Sense Media on the importance of social media during the coronavirus pandemic. <figcaption>A table shows the results of a survey by Common Sense Media on the importance of social media during the coronavirus pandemic. </figcaption>

Social media can pose real problems among young people, especially if they’re already vulnerable, like those experiencing depression or those who come from marginalized communities, said Robb, who co-authored the report. “But many kids are also using social media and the internet to look up things related to their mental health, or look for other people who may be experiencing similar things.”

He added, “If you’re a teen who’s experiencing symptoms of depression or anxiety looking for others who have had similar experiences, being able to take solace in the fact that you’re not alone, or perhaps get tips or advice or other kinds of support from others who are in your community – I don’t want to undervalue social media in those respects, because that’s the other side of the coin that I think does not get talked about quite as often.”

Tips for families

Current research suggests that there’s no clear, direct line between internet use and screen time to mental health concerns with children. Like everything in life, context matters.

“We worry a lot about how much screen time every kid is having, and there are no conclusive studies saying how much screen time exactly is good or bad,” said Melanie Pinola, who spoke to various experts in writing a guide to how and when to limit kids’ tech use for The New York Times. She now covers tech and privacy at Consumer Reports.

She noted that even the American Association of Pediatrics has changed its recommendations on screen time a couple of times.

With a new generation of people who never lived in a world without the constant presence of technology, there’s still a lot of unknowns.

“We’re always learning and trying to adapt to what we know,” Pinola said. 

An illustration shows a phone surrounded by emojis.<figcaption>Illustration: Nick Velazquez / Mozilla</figcaption>

How to help kids have a better internet experience

While there’s no consensus as to how, exactly, children can be raised to have a good relationship with technology, here are what families can do:

1. Be open

Lenhart suggests parents learn about online platforms and use them with their children. “Have an attitude of openness about them, remembering that even as you limit your child’s use of these platforms, you’re also potentially limiting their access to social connection, learning and relationships, because that’s where a lot of these things happen,” Lenhart said. 

She acknowledged that there’s a lot of good reasons why some platforms shouldn’t be used by children as young as 12 or 13, but Lenhart said ideally, adults should figure it out with their kids. She suggested families ask themselves: Is the platform something that you can use together, or that children can use independently?

2. Find good content

Robb noted that there’s plenty of content online, from YouTube channels to video games, that are great for younger kids. 

Common Sense Media rates movies, TV shows, books, games and other media for age appropriateness and “learning potential.” 

“The PBS Kids apps are a lifesaver,” said Lucile Vareine, who works with Mozilla’s communications team and has two young kids. “We go to the public library to research about animals we just discovered on PBS Shows.”

The Mozilla Foundation also creates a guide that analyzes the online security of different products, from mental health apps to toys and games.

3. Think of your child’s well-being outside of technology

Instead of just dialing down on children’s screen time, Robb suggests focusing on things that are essential to children’s health development. Think about whether they’re getting quality sleep, enough socialization with friends, family time and good nutrition. 

“There are lots of things that are really important that are much better supported by data in terms of kids’ mental and physical health than just how many hours of screen use,” Robb said. “I wouldn’t worry so much, if it’s an hour or three hours. I’d look over the course of the week and see, ‘Is my kid doing the things that I hoped that they would be doing?’”

4. Set boundaries

Pinola said it helps, just like with other aspects of parenting, to set some boundaries. She suggests starting slowly, like having a “no tech dinner” rule.

“When I tried that with my [16-year-old] daughter, it worked,” Pinola said. “Now, we’re actually having conversations over dinner, which is what I was used to growing up. If you start small like that, you start to introduce the feeling for kids that they can be off their devices, and there might be a better experience for them.”

5. Use parental controls with your child’s consent, but give them room to grow

Whether it’s time limits or filters, there’s a lot of tools within platforms that can be used by parents who have younger children. Lenhart recommends using these tools with your child’s knowledge, making sure they understand why you’re using them and having a plan to ease oversight.

“We need to teach them slowly but surely how to manage themselves in these spaces,” Lenhart said. “Giving them opportunities to fail with you around to help pick them back up and talk about it with is good. There can be false starts. But it’s definitely something that we have to do.”

Adults shouldn’t be surprised if their kids can go around these tools.

“Young people are much more adaptive than we would think,” Pinola said.

“When we try to limit their tech use too restrictively or try to monitor them a lot, that can be counterproductive because they are so good at [technology], they’re going to find ways to bypass whatever barriers you put,” she said. “It’s just a matter of balance between you and your child.”

The internet is a great place for families. It gives us new opportunities to discover the world, connect with others and just generally make our lives easier and more colorful. But it also comes with new challenges and complications for parents raising the next generation. Mozilla wants to help parents make the best online decisions for their kids, whatever that looks like, in our latest series, Parental Control. 

Firefox browser logo

Get Firefox

Get the browser that protects what’s important

The post Kids are growing up in a very online world. What’s a concerned parent to do? appeared first on The Mozilla Blog.

Hacks.Mozilla.Org — Hacks Decoded: Bikes and Boomboxes with Samuel Aboagye

Welcome to our Hacks: Decoded Interview series!

Once a month, Mozilla Foundation’s Xavier Harding speaks with people in the tech industry about where they’re from, the work they do and what drives them to keep going forward. Make sure you follow Mozilla’s Hacks blog to find more articles in this series and make sure to visit the Mozilla Foundation site to see more of our org’s work.

Meet Samuel Aboagye!

Samuel Aboagye is a genius. Aboagye is 17 years old. In those 17 years, he’s crafted more inventions than you have, probably. Among them: a solar-powered bike and a Bluetooth speaker, both using recycled materials. We caught up with Ghanaian inventor Samuel Aboagye over video chat in hopes that he’d talk with us about his creations, and ultimately how he’s way cooler than any of us were at 17.

 

Samuel, you’ve put together lots of inventions like an electric bike and Bluetooth speaker and even a fan. What made you want to make them?

For the speaker, I thought of how I could minimize the rate at which yellow plastic containers pollute the environment.  I tried to make good use of it after it served its purpose. So, with the little knowledge, I acquired in my science lessons, instead of the empty container just lying down and polluting the environment, I tried to create something useful with it.  

After the Bluetooth speaker was successful, I realized there was more in me I could show to the universe. More importantly, we live in a very poor ventilated room and we couldn’t afford an electric fan so the room was unbearably hot. As such, this situation triggered and motivated me to manufacture a fan to solve this family problem.

With the bike, I thought it would be wise to make life easier for the physically challenged because I was always sad to see them go through all these challenges just to live their daily lives. Electric motors are very expensive and not common in my country, so I decided to do something to help.

Since solar energy is almost always readily available in my part of the world and able to renew itself, I thought that if I am able to make a bike with it, it would help the physically challenged to move from one destination to another without stress or thinking of how to purchase a battery or fuel.  

So how did you go about making them? Did you run into any trouble?

I went around my community gathering used items and old gadgets like radio sets and other electronics and then removed parts that could help in my work. With the electrical energy training given to me by my science teacher after discovering me since JHS1, I was able to apply this and also combined with my God-given talent.

Whenever I need some sort of technical guidance, I call on my teacher Sir David. He has also been my financial help for all my projects.  Financing projects has always been my biggest struggle and most times I have to wait on him to raise funds for me to continue.

The tricycle: Was it much harder to make than a bike?

​​Yes, it was a little bit harder to make the tricycle than the bike. It’s time-consuming and also cost more than a bike. It needs extra technical and critical thinking too. 

You made the bike and speaker out of recycled materials. This answer is probably obvious but I’ve gotta ask: why recycled materials?  Is environment-friendly tech important to you?

I used recycled materials because they were readily available and comparable to cheap and easy to get. With all my inventions I make sure they are all environmentally friendly so as not to pose any danger now or future to the beings on Earth.  But also, I want the world to be a safe and healthy place to be. 

 

The post Hacks Decoded: Bikes and Boomboxes with Samuel Aboagye appeared first on Mozilla Hacks - the Web developer blog.

Niko Matsakis — What it feels like when Rust saves your bacon

You’ve probably heard that the Rust type checker can be a great “co-pilot”, helping you to avoid subtle bugs that would have been a royal pain in the !@#!$! to debug. This is truly awesome! But what you may not realize is how it feels in the moment when this happens. The answer typically is: really, really frustrating! Usually, you are trying to get some code to compile and you find you just can’t do it.

As you come to learn Rust better, and especially to gain a bit of a deeper understanding of what is happening when your code runs, you can start to see when you are getting a type-check error because you have a typo versus because you are trying to do something fundamentally flawed.

A couple of days back, I had a moment where the compiler caught a really subtle bug that would’ve been horrible had it been allowd to compile. I thought it would be fun to narrate a bit how it played out, and also take the moment to explain a bit more about temporaries in Rust (a common source of confusion, in my observations).

Code available in this repository

All the code for this blog post is available in a github repository.

Setting the scene: lowering the AST

In the compiler, we first represent Rust programs using an Abstract Syntax Tree (AST). I’ve prepared a standalone example that shows roughly how the code looks today (of course the real thing is a lot more complex). The AST in particular is found in the ast module containing various data structures that map closely to Rust syntax. So for example we have a Ty type that represents Rust types:

pub enum Ty {
    ImplTrait(TraitRef),
    NamedType(String, Vec<Ty>),
    // ...
}

pub struct Lifetime {
    // ...
}

The impl Trait notation references a TraitRef, which stores the Trait part of things:

pub struct TraitRef {
    pub trait_name: String,
    pub parameters: Parameters,
}

pub enum Parameters {
    AngleBracket(Vec<Parameter>),
    Parenthesized(Vec<Ty>),
}

pub enum Parameter {
    Ty(Ty),
    Lifetime(Lifetime),
}

Note that the parameters of the trait come in two varieties, angle-bracket (e.g., impl PartialEq<T> or impl MyTrait<'a, U>) and parenthesized (e.g., impl FnOnce(String, u32)). These two are slightly different – parenthesized parameters, for example, only accept types, whereas angle-bracket accept types or lifetimes.

After parsing, this AST gets translated to something called High-level Intermediate Representation (HIR) through a process called lowering. The snippet doesn’t include the HIR, but it includes a number of methods like lower_ty that take as input an AST type and produce the HIR type:

impl Context {
    fn lower_ty(&mut self, ty: &ast::Ty) -> hir::Ty {
        match ty {
            // ... lots of stuff here
            // A type like `impl Trait`
            ast::Ty::ImplTrait(trait_ref) => {
                do_something_with(trait_ref);
            }

            // A type like `Vec<T>`, where `Vec` is the name and
            // `[T]` are the `parameters`
            ast::Ty::NamedType(name, parameters) => {
                for parameter in parameters {
                    self.lower_ty(parameter);
                }
            }
        }
        // ...
    }
}

Each method is defined on this Context type that carries some common state, and the methods tend to call one another. For example, lower_signature invokes lower_ty on all of the input (argument) types and on the output (return) type:

impl Context {
    fn lower_signature(&mut self, sig: &ast::Signature) -> hir::Signature {
        for input in &sig.inputs {
            self.lower_ty(input);
        }

        self.lower_ty(&sig.output);

        ...
    }
}

Our story begins

Santiago Pastorino is working on a refactoring to make it easier to support returning impl Trait values from trait functions. As part of that, he needs to collect all the impl Trait types that appear in the function arguments. The challenge is that these types can appear anywhere, and not just at the top level. In other words, you might have fn foo(x: impl Debug), but you might also have fn foo(x: Box<(impl Debug, impl Debug)>). Therefore, we decided it would make sense to add a vector to Context and have lower_ty collect the impl Trait types into it. That way, we can find the complete set.

To do this, we started by adding the vector into this Context. We’ll store the TraitRef from each impl Trait type:

struct Context<'ast> {
    saved_impl_trait_types: Vec<&'ast ast::TraitRef>,
    // ...
}

To do this, we had to add a new lifetime parameter, 'ast, which is meant to represent the lifetime of the AST structure itself. In other words, saved_impl_trait_types stores references into the AST. Of course, once we did this, the compiler got upset and we had to go modify the impl block that references Context:

impl<'ast> Context<'ast> {
    ...
}

Now we can modify the lower_ty to push the trait ref into the vector:

impl<'ast> Context<'ast> {
    fn lower_ty(&mut self, ty: &ast::Ty) {
        match ty {
            ...
            
            ast::Ty::ImplTrait(...) => {
                // 👇 push the types into the vector 👇
                self.saved_impl_trait_types.push(ty);
                do_something();
            }

            ast::Ty::NamedType(name, parameters) => {
                ... // just like before
            }
            
            ...
        }
    }
}

At this point, the compiler gives us an error:

error[E0621]: explicit lifetime required in the type of `ty`
   --> examples/b.rs:125:42
    |
119 |     fn lower_ty(&mut self, ty: &ast::Ty) -> hir::Ty {
    |                                -------- help: add explicit lifetime `'ast` to the type of `ty`: `&'ast ast::Ty`
...
125 |                 self.impl_trait_tys.push(trait_ref);
    |                                          ^^^^^^^^^ lifetime `'ast` required

Pretty nice error, actually! It’s pointing out that we are pushing into this vector which needs references into “the AST”, but we haven’t declared in our signature that the ast::Ty must actually from “the AST”. OK, let’s fix this:

impl<'ast> Context<'ast> {
    fn lower_ty(&mut self, ty: &'ast ast::Ty) {
        // had to add 'ast here 👆, just like the error message said
        ...
    }
}

Propagating lifetimes everywhere

Of course, now we start getting errors in the functions that call lower_ty. For example, lower_signature says:

error[E0621]: explicit lifetime required in the type of `sig`
  --> examples/b.rs:71:18
   |
65 |     fn lower_signature(&mut self, sig: &ast::Signature) -> hir::Signature {
   |                                        --------------- help: add explicit lifetime `'ast` to the type of `sig`: `&'ast ast::Signature`
...
71 |             self.lower_ty(input);
   |                  ^^^^^^^^ lifetime `'ast` required

The fix is the same. We tell the compiler that the ast::Signature is part of “the AST”, and that implies that the ast::Ty values owned by the ast::Signature are also part of “the AST”:

impl<'ast> Context<'ast> {
    fn lower_signature(&mut self, sig: &'ast ast::Signature) -> hir::Signature {
        //        had to add 'ast here 👆, just like the error message said
        ...
    }
}

Great. This continues for a bit. But then… we hit this error:

error[E0597]: `parameters` does not live long enough
  --> examples/b.rs:92:53
   |
58 | impl<'ast> Context<'ast> {
   |      ---- lifetime `'ast` defined here
...
92 |                 self.lower_angle_bracket_parameters(&parameters);
   |                 ------------------------------------^^^^^^^^^^^-
   |                 |                                   |
   |                 |                                   borrowed value does not live long enough
   |                 argument requires that `parameters` is borrowed for `'ast`
93 |             }
   |             - `parameters` dropped here while still borrowed

What’s this about?

Uh oh…

Jumping to that line, we see this function lower_trait_ref:

impl Context<'ast> {
    // ...
    fn lower_trait_ref(&mut self, trait_ref: &'ast ast::TraitRef) -> hir::TraitRef {
        match &trait_ref.parameters {
            ast::Parameters::AngleBracket(parameters) => {
                self.lower_angle_bracket_parameters(&parameters);
            }
            ast::Parameters::Parenthesized(types) => {
                let parameters: Vec<_> = types.iter().cloned().map(ast::Parameter::Ty).collect();
                self.lower_angle_bracket_parameters(&parameters); // 👈 error is on this line
                
            }
        }

        hir::TraitRef
    }
    // ...
}

So what’s this about? Well, the purpose of this code is a bit clever. As we saw before, Rust has two syntaxes for trait-refs, you can use parentheses like FnOnce(u32), in which case you only have types, or you can use angle brackets like Foo<'a, u32>, in which case you could have either lifetimes or types. So this code is normalizing to the angle-bracket notation, which is more general, and then using the same lowering helper function.

Wait! Right there! That was the moment!

What?

That was the moment that Rust saved you a world of pain!

It was? It just kind of seemed like an annoying, and I will say, kind of confusing compilation error. What the heck is going on? The problem here is that parameters is a local variable. It is going to be freed as soon as lower_trait_ref returns. But it could happen that lower_trait_ref calls lower_ty which takes a reference to the type and stores it into the saved_impl_trait_types vector. Then, later, some code would try to use that reference, and access freed memory. That would sometimes work, but often not – and if you forgot to test with parenthesized trait refs, the code would work fine for ever, so you’d never even notice.

How to fix it

Maybe you’re wondering: great, Rust saved me a world of pain, but how do I fix it? Do I just have to copy the lower_angle_bracket_parameters and have two copies? ‘Cause that’s kind of unfortunate.

Well, there are a variety of ways you might fix it. One of them is to use an arena, like the typed-arena crate. An arena is a memory pool. Instead of storing the temporary Vec<Parameter> vector on the stack, we’ll put it in an arena, and that way it will live for the entire time that we are lowering things. Example C in the repo takes this approach. It starts by adding the arena field to the Context:

struct Context<'ast> {
    impl_trait_tys: Vec<&'ast ast::TraitRef>,

    // Holds temporary AST nodes that we create during lowering;
    // this can be dropped once lowering is complete.
    arena: &'ast typed_arena::Arena<Vec<ast::Parameter>>,
}

This actually makes a subtle change to the meaning of 'ast. It used to be that the only things with 'ast lifetime were “the AST” itself, so having that lifetime implied being a part of the AST. But now that same lifetime is being used to tag the arena, too, so if we hae &'ast Foo it means the data comes is owned by either the arena or the AST itself.

Side note: despite the name lifetimes, which I now rather regret, more and more I tend to think of lifetimes like 'ast in terms of “who owns the data”, which you can see in my description in the previous paragraph. You could instead think of 'ast as a span of time (a “lifetime”), in which case it refers to the time that the Context type is valid, really, which must be a subset of the time that the arena is valid and the time that the AST itself is valid, since Context stores references to data owned by both of those.

Now we can rewrite lower_trait_ref to call self.arena.alloc():

impl Context<'ast> {
    fn lower_trait_ref(&mut self, trait_ref: &'ast ast::TraitRef) -> hir::TraitRef {
        match &trait_ref.parameters {
            // ...
            ast::Parameters::Parenthesized(types) => {
                let parameters: Vec<_> = types.iter().cloned().map(ast::Parameter::Ty).collect();
                let parameters = self.arena.alloc(parameters); // 👈 added this line!
                self.lower_angle_bracket_parameters(parameters);
            }
        }
        // ...
    }
}

Now the parameters variable is not stored on the stack but allocated in the arena; the arena has 'ast lifetime, so that’s fine, and everything works!

Calling the lowering code and creating the context

Now that we added, the arena, creating the context will look a bit different. It’ll look something like:

let arena = TypedArena::new();
let context = Context::new(&arena);
let hir_signature = context.lower_signature(&signature);

The nice thing about this is that, once we are done with lowering, the context will be dropped and all those temporary nodes will be freed.

Another way to fix it

The other obvious option is to avoid lifetimes altogether and just “clone all the things”. Given that the AST is immutable once constructed, you can just clone them into the vector:

struct Context {
    impl_trait_tys: Vec<ast::TraitRef>, // just clone it!
}

If that clone is too expensive (possible), then use Rc<ast::TraitRef> or Arc<ast::TraitRef> (this will require deep-ish changes to the AST to put all the things into Rc or Arc that might need to be individually referenced). At this point you’ve got a feeling a lot like garbage collection (if less ergonomic).

Yet another way

The way I tend to write compilers these days is to use the “indices as pointers”. In this approach, all the data in the AST is stored in vectors, and references between things use indices, kind of like I described here.

Conclusion

Compilation errors are pretty frustrating, but they may also be a sign that the compiler is protecting us from ourselves. In this case, when we embarked on this refactoring, I was totally sure it was going to work fine, because I didn’t realize we ever created “temporary AST” nodes, so I assumed that all the data was owned by the original AST. In a language like C or C++, it would have been very easy to have a bug here, and it would have been a horrible pain to find. With Rust, that’s not a problem.

Of course, not everything is great. For me, doing these kinds of lifetime transformations is old-hat. But for many people it’s pretty non-obvious how to start when the compiler is giving you error messages. When people come to me for help, the first thing I try to do is to suss out: what are the ownership relationships, and where do we expect these references to be coming form? There’s also various heuristics that I use to decide: do we need a new lifetime parameter? Can we re-use an existing one? I’ll try to write up more stories like this to clarify that side of things. Honestly, my main point here was that I was just so grateful that Rust prevented us from spending hours and hours debugging a subtle crash!

Looking forward a bit, I see a lot of potential to improve things about our notation and terminology. I think we should be able to make cases like this one much slicker, hopefully without requiring named lifetime parameters and so forth, or as many edits. But I admit I don’t yet know how to do it! :) My plan for now is to keep an eye out for the tricks I am using and the kinds of analysis I am doing in my head and write out blog posts like this one to capture those narratives. I encourage those of you who know Rust well (or who don’t!) to do the same.

Appendix: why not have Context own the TypedArena?

You may have noticed that using the arena had a kind of annoying consequence: people who called Context::new now had to create and supply an area:

let arena = TypedArena::new();
let context = Context::new(&arena);
let hir_signature = context.lower_signature(&signature);

This is because Context<'ast> stores a &'ast TypedArena<_>, and so the caller must create the arena. If we modified Context to own the arena, then the API could be better. So why didn’t I do that? To see why, check out example D (which doesn’t build). In that example, the Context looks like…

struct Context<'ast> {
    impl_trait_tys: Vec<&'ast ast::TraitRef>,

    // Holds temporary AST nodes that we create during lowering;
    // this can be dropped once lowering is complete.
    arena: typed_arena::Arena<Vec<ast::Parameter>>,
}

You then have to change the signatures of each function to take an &'ast mut self:

impl Context<'ast> {
    fn lower_signature(&'ast mut self, sig: &'ast ast::Signature) -> hir::Signature {...}
}

This is saying: the 'ast parameter might refer to data owned by self, or maybe by sig. Seems sensible, but if you try to build Example D, though, you get lots of errors. Here is one of the most interesting to me:

error[E0502]: cannot borrow `*self` as mutable because it is also borrowed as immutable
  --> examples/d.rs:98:17
   |
62 | impl<'ast> Context<'ast> {
   |      ---- lifetime `'ast` defined here
...
97 |                 let parameters = self.arena.alloc(parameters);
   |                                  ----------------------------
   |                                  |
   |                                  immutable borrow occurs here
   |                                  argument requires that `self.arena` is borrowed for `'ast`
98 |                 self.lower_angle_bracket_parameters(parameters);
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ mutable borrow occurs here

What is this all about? This is actually pretty subtle! This is saying that parameters was allocated from self.arena. That means that parameters will be valid as long as self.arena is valid.

But self is an &mut Context, which means it can mutate any of the fields of the Context. When we call self.lower_angle_bracket_parameters(), it’s entirely possible that lower_angle_bracket_parameters could mutate the arena:

fn lower_angle_bracket_parameters(&'ast mut self, parameters: &'ast [ast::Parameter]) {
    self.arena = TypedArena::new(); // what if we did this?
    // ...
}

Of course, the code doesn’t do that now, but what if it did? The answer is that the parameters would be freed, because the arena that owns them is freed, and so we’d have dead code. D’oh!

All things considered, I’d like to make it possible for Context to own the arena, but right now it’s pretty challenging. This is a good example of code patterns we could enable, but it’ll require language extensions.

The Mozilla Blog — How to easily switch from Chrome to Firefox

There’s never been a better time to switch from Chrome to Firefox, if we do say so ourselves. 

Some of the internet’s most popular ad blockers, such as uBlock Origin — tools that save our digital sanity from video ads that auto-play, banners that take up half the screen and pop-up windows with infuriatingly tiny “x” buttons — will become less effective on Google’s web browser thanks to a set of changes in Chrome’s extensions platform. 

At Mozilla, we’re all about protecting your privacy and security – all while offering add-ons and features that enhance performance and functionality so you can experience the very best of the web. We know that content blockers are very important to Firefox users, so rest assured that despite changes to Chrome’s new extensions platform, we’ll continue to ensure access to the best privacy tools available – including content-blocking extensions that not only stop creepy ads from following you around the internet, but also allows for a faster and more seamless browsing experience. In addition, Firefox has recently enabled Total Cookie Protection as default for all users, making Firefox the most private and secure major browser available across Windows, Mac and Linux.  

Longtime Chrome user? We know change can be hard. But we’re here to help you make the move with any data you want to bring along, including your bookmarks, saved passwords and browsing history.  

Here’s how to easily switch from Chrome to Firefox as your desktop browser in five steps:

Step 1: Download and install Firefox from Mozilla’s download page. 

Step 2: If you have Chrome running, quit the app. But don’t delete it just yet.

Step 3: Open Firefox. The import tool should pop up. 

In case it doesn’t, click the menu button Fx57Menu > Bookmarks > Manage Bookmarks > Import import button > Import Data from another Browser. 

Step 4: In the Import Settings and Data window, choose Chrome. Then in the next screen, select what you want to import:

  • Cookies: Bits of data stored on your computer by some websites. They’re used to remember information like your log-in, language preference and even items you added in an online shopping cart. With Total Cookies Protection on by default on Firefox, cookies are confined to the site where they were created — preventing tracking companies from using your data.
  • Browsing History: A list of web pages you’ve visited. If there’s an article you didn’t quite finish last week, bring over your browsing history so you can find it later. (Pro tip: Save it in your Pocket list next time!) 
  • Saved Passwords: Usernames and passwords you saved in Chrome. Here’s why you can trust Firefox with your passwords.
  • Bookmarks: Web pages that you bookmarked in Chrome. 

Step 5: Once you pick which data to bring to Firefox, click Continue > Done. 

If you imported your bookmarks, you’ll find them in a folder named “From Google Chrome” in the Firefox Bookmarks Toolbar. 

In case the toolbar is hidden, click the menu button Fx89menuButton> More Tools… > choose Customize Toolbar > Toolbars > Bookmarks Toolbar > set to Always Show, Never Show or Only Show on New Tab > Done.

We may be a little biased, but we truly believe that Mozilla’s commitment to privacy helps make the internet better and safer for everyone. We wouldn’t be able to do it without the support of our community of Firefox users, so we’d love for you to join us.  

Related stories:

The post How to easily switch from Chrome to Firefox appeared first on The Mozilla Blog.

This Week In Rust — This Week in Rust 447

Hello and welcome to another issue of This Week in Rust! Rust is a programming language empowering everyone to build reliable and efficient software. This is a weekly summary of its progress and community. Want something mentioned? Tweet us at @ThisWeekInRust or send us a pull request. Want to get involved? We love contributions.

This Week in Rust is openly developed on GitHub. If you find any errors in this week's issue, please submit a PR.

Updates from Rust Community

Official
Foundation
Newsletters
Project/Tooling Updates
Observations/Thoughts
Rust Walkthroughs
Research
Miscellaneous

Crate of the Week

This week's crate is dxf-rs, a library to parse AutoCAD files.

Thanks to cosj for the suggestion!

Please submit your suggestions and votes for next week!

Call for Participation

Always wanted to contribute to open-source projects but didn't know where to start? Every week we highlight some tasks from the Rust community for you to pick and get started!

Some of these tasks may also have mentors available, visit the task page for more information.

If you are a Rust project owner and are looking for contributors, please submit tasks here.

Updates from the Rust Project

368 pull requests were merged in the last week

Rust Compiler Performance Triage

A mixed week. I suppose it is best to focus on the fact we made some big improvements to a large number of primary benchmarks, at the cost of some smaller regressions to a smaller number of primary benchmarks.

Triage done by @pnkfelix. Revision range: bb55bd44..edab34ab

Summary:

mean max count
Regressions 😿
(primary)		 0.6% 1.6% 35
Regressions 😿
(secondary)		 2.1% 8.1% 23
Improvements 🎉
(primary)		 -0.8% -3.5% 72
Improvements 🎉
(secondary)		 -0.8% -2.9% 62
All 😿🎉 (primary) -0.4% -3.5% 107

4 Regressions, 3 Improvements, 5 Mixed; 4 of them in rollups 47 artifact comparisons made in total

Full report here.

Call for Testing

An important step for RFC implementation is for people to experiment with the implementation and give feedback, especially before stabilization. The following RFCs would benefit from user testing before moving forward:

  • No RFCs issued a call for testing this week.

If you are a feature implementer and would like your RFC to appear on the above list, add the new call-for-testing label to your RFC along with a comment providing testing instructions and/or guidance on which aspect(s) of the feature need testing.

Approved RFCs

Changes to Rust follow the Rust RFC (request for comments) process. These are the RFCs that were approved for implementation this week:

Final Comment Period

Every week the team announces the 'final comment period' for RFCs and key PRs which are reaching a decision. Express your opinions now.

RFCs
  • No RFCs entered Final Comment Period this week.
Tracking Issues & PRs
New and Updated RFCs

Upcoming Events

Rusty Events between 2022-06-15 - 2022-07-13 🦀

Virtual
North America
Europe
Oceania

If you are running a Rust event please add it to the calendar to get it mentioned here. Please remember to add a link to the event too. Email the Rust Community Team for access.

Rust Jobs

Audiotool

Kollider

Disney

Stockly

Stellate

Enso

Spire Global

Tweet us at @ThisWeekInRust to get your job offers listed here!

Quote of the Week

Because lower-level software has more operational constraints than higher-level software (e.g. it typically cannot tolerate a runtime or memory management via garbage collection), developing a memory safe language suitable for systems software is particularly challenging. The Rust language has met that challenge, however, and is an excellent candidate for replacing C in many systems applications.

We plan to invest in the tools that allow systems engineers to move their software to Rust. This means investing in improving package management, compilers, and Foreign Function Interface (FFI) generators. In many cases this will include providing interfaces compatible with existing widely-used components to enable transition. With these tools, adoption of a memory safe alternative will scale much faster without replication of efforts.

– The White House Open Source Software Mobilization Plan, multiple authors (PDF link)

Thanks to Brian Kung for the suggestion.

Please submit quotes and vote for next week!

This Week in Rust is edited by: nellshamrell, llogiq, cdmistman, ericseppanen, extrawurst, andrewpollack, U007D, kolharsam, joelmarcey, mariannegoldin.

Email list hosting is sponsored by The Rust Foundation

Discuss on r/rust

The Mozilla Blog — How to set Firefox as your default browser on Windows

During a software update, your settings can sometimes change or revert back to their original state. For example, if your computer has multiple browsers installed, you may end up with a different default browser than Firefox. That’s easy to fix so that Firefox is there for you when you expect it, like when you click on web links in email and other apps.

With Firefox set as your default Windows browser, you’ll be automatically guarded from invasive tracking methods like fingerprinting and cryptomining, thanks to Firefox’s technology that blocks more than 10,000,000,000 trackers every day including Total Cookie Protection. (See what Firefox has blocked for you.) Your bookmarks, history, open tabs, form information and passwords are accessible wherever you’re logged into Firefox, from your PC to your phone to your tablet.

If you’re using a PC — like an HP, Dell, Lenovo or Acer brand computer — that runs the Microsoft Windows operating system, here’s how to set Firefox as your default:

  1. Click the menu button (three horizontal lines) and select Options.
  2. In the General panel, click Make Default
  3. The Windows Settings app will open with the Choose default apps screen.
  4. Scroll down and click the entry under Web browser.
  5. Click on Firefox in the dialog that opens with a list of available browsers.

Firefox is now listed as your default browser. Close the Settings window by clicking the X in the upper right to save your changes.

Another option is to go through the Windows 10 operating systems settings:

  1. Go to the Windows Start menu and click the Settings icon.
  2. Click Apps, then choose Default Apps on the left pane.
  3. Scroll down and click the entry under Web browser.
  4. Click on Firefox in the dialog that opens with a list of available browsers.
  5. Firefox is now listed as your default browser. Close the Settings window to save your changes.

Originally published January 20, 2020.

See also:
Make Firefox as your default browser on Mac
Make Firefox the default browser on Android
Make Firefox the default browser for iOS

The post How to set Firefox as your default browser on Windows appeared first on The Mozilla Blog.

Hacks.Mozilla.Org — Everything Is Broken: Shipping rust-minidump at Mozilla – Part 1

Everything Is Broken: Shipping rust-minidump at Mozilla

For the last year I’ve been leading the development of rust-minidump, a pure-Rust replacement for the minidump-processing half of google-breakpad.

Well actually in some sense I finished that work, because Mozilla already deployed it as the crash processing backend for Firefox 6 months ago, it runs in half the time, and seems to be more reliable. (And you know, isn’t a terrifying ball of C++ that parses and evaluates arbitrary input from the internet. We did our best to isolate Breakpad, but still… yikes.)

This is a pretty fantastic result, but there’s always more work to do because Minidumps are an inky abyss that grows deeper the further you delve… wait no I’m getting ahead of myself. First the light, then the abyss. Yes. Light first.

What I can say is that we have a very solid implementation of the core functionality of minidump parsing+analysis for the biggest platforms (x86, x64, ARM, ARM64; Windows, MacOS, Linux, Android). But if you want to read minidumps generated on a PlayStation 3 or process a Full Memory dump, you won’t be served quite as well.

We’ve put a lot of effort into documenting and testing this thing, so I’m pretty confident in it!

Unfortunately! Confidence! Is! Worth! Nothing!

Which is why this is the story of how we did our best to make this nightmare as robust as we could and still got 360 dunked on from space by the sudden and incredible fuzzing efforts of @5225225.

This article is broken into two parts:

  1. what minidumps are, and how we made rust-minidump
  2. how we got absolutely owned by simple fuzzing

You are reading part 1, wherein we build up our hubris.

Background: What’s A Minidump, and Why Write rust-minidump?

Your program crashes. You want to know why your program crashed, but it happened on a user’s machine on the other side of the world. A full coredump (all memory allocated by the program) is enormous — we can’t have users sending us 4GB files! Ok let’s just collect up the most important regions of memory like the stacks and where the program crashed. Oh and I guess if we’re taking the time, let’s stuff some metadata about the system and process in there too.

Congratulations you have invented Minidumps. Now you can turn a 100-thread coredump that would otherwise be 4GB into a nice little 2MB file that you can send over the internet and do postmortem analysis on.

Or more specifically, Microsoft did. So long ago that their docs don’t even discuss platform support. MiniDumpWriteDump’s supported versions are simply “Windows”. Microsoft Research has presumably developed a time machine to guarantee this.

Then Google came along (circa 2006-2007) and said “wouldn’t it be nice if we could make minidumps on any platform”? Thankfully Microsoft had actually built the format pretty extensibly, so it wasn’t too bad to extend the format for Linux, MacOS, BSD, Solaris, and so on. Those extensions became google-breakpad (or just Breakpad) which included a ton of different tools for generating, parsing, and analyzing their extended minidump format (and native Microsoft ones).

Mozilla helped out with this a lot because apparently, our crash reporting infrastructure (“Talkback”) was miserable circa 2007, and this seemed like a nice improvement. Needless to say, we’re pretty invested in breakpad’s minidumps at this point.

Fast forward to the present day and in a hilarious twist of fate, products like VSCode mean that Microsoft now supports applications that run on Linux and MacOS so it runs breakpad in production and has to handle non-Microsoft minidumps somewhere in its crash reporting infra, so someone else’s extension of their own format is somehow their problem now!

Meanwhile, Google has kind-of moved on to Crashpad. I say kind-of because there’s still a lot of Breakpad in there, but they’re more interested in building out tooling on top of it than improving Breakpad itself. Having made a few changes to Breakpad: honestly fair, I don’t want to work on it either. Still, this was a bit of a problem for us, because it meant the project became increasingly under-staffed.

By the time I started working on crash reporting, Mozilla had basically given up on upstreaming fixes/improvements to Breakpad, and was just using its own patched fork. But even without the need for upstreaming patches, every change to Breakpad filled us with dread: many proposed improvements to our crash reporting infrastructure stalled out at “time to implement this in Breakpad”.

Why is working on Breakpad so miserable, you ask?

Parsing and analyzing minidumps is basically an exercise in writing a fractal parser of platform-specific formats nested in formats nested in formats. For many operating systems. For many hardware architectures. And all the inputs you’re parsing and analyzing are terrible and buggy so you have to write a really permissive parser and crawl forward however you can.

Some specific MSVC toolchain that was part of Windows XP had a bug in its debuginfo format? Too bad, symbolicate that stack frame anyway!

The program crashed because it horribly corrupted its own stack? Too bad, produce a backtrace anyway!

The minidump writer itself completely freaked out and wrote a bunch of garbage to one stream? Too bad, produce whatever output you can anyway!

Hey, you know who has a lot of experience dealing with really complicated permissive parsers written in C++? Mozilla! That’s like the core functionality of a web browser.

Do you know Mozilla’s secret solution to writing really complicated permissive parsers in C++?

We stopped doing it.

We developed Rust and ported our nastiest parsers to it.

We’ve done it a lot, and when we do we’re always like “wow this is so much more reliable and easy to maintain and it’s even faster now”. Rust is a really good language for writing parsers. C++ really isn’t.

So we Rewrote It In Rust (or as the kids call it, “Oxidized It”). Breakpad is big, so we haven’t actually covered all of its features. We’ve specifically written and deployed:

  • dump_syms which processes native build artifacts into symbol files.
  • rust-minidump which is a collection of crates that parse and analyze minidumps. Or more specifically, we deployed minidump-stackwalk, which is the high-level cli interface to all of rust-minidump.

Notably missing from this picture is minidump writing, or what google-breakpad calls a client (because it runs on the client’s machine). We are working on a rust-based minidump writer, but it’s not something we can recommend using quite yet (although it has sped up a lot thanks to help from Embark Studios).

This is arguably the messiest and hardest work because it has a horrible job: use a bunch of native system APIs to gather up a bunch of OS-specific and Hardware-specific information about the crash AND do it for a program that just crashed, on a machine that caused the program to crash.

We have a long road ahead but every time we get to the other side of one of these projects it’s wonderful.

 

Background: Stackwalking and Calling Conventions

One of rust-minidump’s (minidump-stackwalk’s) most important jobs is to take the state for a thread (general purpose registers and stack memory) and create a backtrace for that thread (unwind/stackwalk). This is a surprisingly complicated and messy job, made only more complicated by the fact that we are trying to analyze the memory of a process that got messed up enough to crash.

This means our stackwalkers are inherently working with dubious data, and all of our stackwalking techniques are based on heuristics that can go wrong and we can very easily find ourselves in situations where the stackwalk goes backwards or sideways or infinite and we just have to try to deal with it!

It’s also pretty common to see a stackwalker start hallucinating, which is my term for “the stackwalker found something that looked plausible enough and went on a wacky adventure through the stack and made up a whole pile of useless garbage frames”. Hallucination is most common near the bottom of the stack where it’s also least offensive. This is because each frame you walk is another chance for something to go wrong, but also increasingly uninteresting because you’re rarely interested in confirming that a thread started in The Same Function All Threads Start In.

All of these problems would basically go away if everyone agreed to properly preserve their cpu’s PERFECTLY GOOD DEDICATED FRAME POINTER REGISTER. Just kidding, turning on frame pointers doesn’t really work either because Microsoft invented chaos frame pointers that can’t be used for unwinding! I assume this happened because they accidentally stepped on the wrong butterfly while they were traveling back in time to invent minidumps. (I’m sure it was a decision that made more sense 20 years ago, but it has not aged well.)

If you would like to learn more about the different techniques for unwinding, I wrote about them over here in my article on Apple’s Compact Unwind Info. I’ve also attempted to document breakpad’s STACK WIN and STACK CFI unwind info formats here, which are more similar to the  DWARF and PE32 unwind tables (which are basically tiny programming languages).

If you would like to learn more about ABIs in general, I wrote an entire article about them here. The end of that article also includes an introduction to how calling conventions work. Understanding calling conventions is key to implementing unwinders.

 

How Hard Did You Really Test Things?

Hopefully you now have a bit of a glimpse into why analyzing minidumps is an enormous headache. And of course you know how the story ends: that fuzzer kicks our butts! But of course to really savor our defeat, you have to see how hard we tried to do a good job! It’s time to build up our hubris and pat ourselves on the back.

So how much work actually went into making rust-minidump robust before the fuzzer went to work on it?

Quite a bit!

I’ll never argue all the work we did was perfect but we definitely did some good work here, both for synthetic inputs and real world ones. Probably the biggest “flaw” in our methodology was the fact that we were only focused on getting Firefox’s usecase to work. Firefox runs on a lot of platforms and sees a lot of messed up stuff, but it’s still a fairly coherent product that only uses so many features of minidumps.

This is one of the nice benefits of our recent work with Sentry, which is basically a Crash Reporting As A Service company. They are way more liable to stress test all kinds of weird corners of the format that Firefox doesn’t, and they have definitely found (and fixed!) some places where something is wrong or missing! (And they recently deployed it into production too! 🎉)

But hey don’t take my word for it, check out all the different testing we did:

Synthetic Minidumps for Unit Tests

rust-minidump includes a synthetic minidump generator which lets you come up with a high-level description of the contents of a minidump, and then produces an actual minidump binary that we can feed it into the full parser:

// Let’s make a synth minidump with this particular Crashpad Info…

let module = ModuleCrashpadInfo::new(42, Endian::Little)
    .add_list_annotation("annotation")
    .add_simple_annotation("simple", "module")
    .add_annotation_object("string", AnnotationValue::String("value".to_owned()))
    .add_annotation_object("invalid", AnnotationValue::Invalid)
    .add_annotation_object("custom", AnnotationValue::Custom(0x8001, vec![42]));

let crashpad_info = CrashpadInfo::new(Endian::Little)
    .add_module(module)
    .add_simple_annotation("simple", "info");

let dump = SynthMinidump::with_endian(Endian::Little).add_crashpad_info(crashpad_info);

// convert the synth minidump to binary and read it like a normal minidump
let dump = read_synth_dump(dump).unwrap();

// Now check that the minidump reports the values we expect…

minidump-synth intentionally avoids sharing layout code with the actual implementation so that incorrect changes to layouts won’t “accidentally” pass tests.

A brief aside for some history: this testing framework was started by the original lead on this project, Ted Mielczarek. He started rust-minidump as a side project to learn Rust when 1.0 was released and just never had the time to finish it. Back then he was working at Mozilla and also a major contributor to Breakpad, which is why rust-minidump has a lot of similar design choices and terminology.

This case is no exception: our minidump-synth is a shameless copy of the synth-minidump utility in breakpad’s code, which was originally written by our other coworker Jim Blandy. Jim is one of the only people in the world that I will actually admit writes really good tests and docs, so I am totally happy to blatantly copy his work here.

Since this was all a learning experiment, Ted was understandably less rigorous about testing than usual. This meant a lot of minidump-synth was unimplemented when I came along, which also meant lots of minidump features were completely untested. (He built an absolutely great skeleton, just hadn’t had the time to fill it all in!)

We spent a lot of time filling in more of minidump-synth’s implementation so we could write more tests and catch more issues, but this is definitely the weakest part of our tests. Some stuff was implemented before I got here, so I don’t even know what tests are missing!

This is a good argument for some code coverage checks, but it would probably come back with “wow you should write a lot more tests” and we would all look at it and go “wow we sure should” and then we would probably never get around to it, because there are many things we should do.

On the other hand, Sentry has been very useful in this regard because they already have a mature suite of tests full of weird corner cases they’ve built up over time, so they can easily identify things that really matter, know what the fix should roughly be, and can contribute pre-existing test cases!

Integration and Snapshot Tests

We tried our best to shore up coverage issues in our unit tests by adding more holistic tests. There’s a few checked in Real Minidumps that we have some integration tests for to make sure we handle Real Inputs properly.

We even wrote a bunch of integration tests for the CLI application that snapshot its output to confirm that we never accidentally change the results.

Part of the motivation for this is to ensure we don’t break the JSON output, which we also wrote a very detailed schema document for and are trying to keep stable so people can actually rely on it while the actual implementation details are still in flux.

Yes, minidump-stackwalk is supposed to be stable and reasonable to use in production!

For our snapshot tests we use insta, which I think is fantastic and more people should use. All you need to do is assert_snapshot! any output you want to keep track of and it will magically take care of the storing, loading, and diffing.

Here’s one of the snapshot tests where we invoke the CLI interface and snapshot stdout:

#[test]
fn test_evil_json() {
    // For a while this didn't parse right
    let bin = env!("CARGO_BIN_EXE_minidump-stackwalk");
    let output = Command::new(bin)
        .arg("--json")
        .arg("--pretty")
        .arg("--raw-json")
        .arg("../testdata/evil.json")
        .arg("../testdata/test.dmp")
        .arg("../testdata/symbols/")
        .stdout(Stdio::piped())
        .stderr(Stdio::piped())
        .output()
        .unwrap();

    let stdout = String::from_utf8(output.stdout).unwrap();
    let stderr = String::from_utf8(output.stderr).unwrap();

    assert!(output.status.success());
    insta::assert_snapshot!("json-pretty-evil-symbols", stdout);
    assert_eq!(stderr, "");
}

Stackwalker Unit Testing

The stackwalker is easily the most complicated and subtle part of the new implementation, because every platform can have slight quirks and you need to implement several different unwinding strategies and carefully tune everything to work well in practice.

The scariest part of this was the call frame information (CFI) unwinders, because they are basically little virtual machines we need to parse and execute at runtime. Thankfully breakpad had long ago smoothed over this issue by defining a simplified and unified CFI format, STACK CFI (well, nearly unified, x86 Windows was still a special case as STACK WIN). So even if DWARF CFI has a ton of complex features, we mostly need to implement a Reverse Polish Notation Calculator except it can read registers and load memory from addresses it computes (and for STACK WIN it has access to named variables it can declare and mutate).

Unfortunately, Breakpad’s description for this format is pretty underspecified so I had to basically pick some semantics I thought made sense and go with that. This made me extremely paranoid about the implementation. (And yes I will be more first-person for this part, because this part was genuinely where I personally spent most of my time and did a lot of stuff from scratch. All the blame belongs to me here!)

The STACK WIN / STACK CFI parser+evaluator is 1700 lines. 500 of those lines are a detailed documentation and discussion of the format, and 700 of those lines are an enormous pile of ~80 test cases where I tried to come up with every corner case I could think of.

I even checked in two tests I knew were failing just to be honest that there were a couple cases to fix! One of them is a corner case involving dividing by a negative number that almost certainly just doesn’t matter. The other is a buggy input that old x86 Microsoft toolchains actually produce and parsers need to deal with. The latter was fixed before the fuzzing started.

And 5225225 still found an integer overflow in the STACK WIN preprocessing step! (Not actually that surprising, it’s a hacky mess that tries to cover up for how messed up x86 Windows unwinding tables were.)

(The code isn’t terribly interesting here, it’s just a ton of assertions that a given input string produces a given output/error.)

Of course, I wasn’t satisfied with just coming up with my own semantics and testing them: I also ported most of breakpad’s own stackwalker tests to rust-minidump! This definitely found a bunch of bugs I had, but also taught me some weird quirks in Breakpad’s stackwalkers that I’m not sure I actually agree with. But in this case I was flying so blind that even being bug-compatible with Breakpad was some kind of relief.

Those tests also included several tests for the non-CFI paths, which were similarly wobbly and quirky. I still really hate a lot of the weird platform-specific rules they have for stack scanning, but I’m forced to work on the assumption that they might be load-bearing. (I definitely had several cases where I disabled a breakpad test because it was “obviously nonsense” and then hit it in the wild while testing. I quickly learned to accept that Nonsense Happens And Cannot Be Ignored.)

One major thing I didn’t replicate was some of the really hairy hacks for STACK WIN. Like there are several places where they introduce extra stack-scanning to try to deal with the fact that stack frames can have mysterious extra alignment that the windows unwinding tables just don’t tell you about? I guess?

There’s almost certainly some exotic situations that rust-minidump does worse on because of this, but it probably also means we do better in some random other situations too. I never got the two to perfectly agree, but at some point the divergences were all in weird enough situations, and as far as I was concerned both stackwalkers were producing equally bad results in a bad situation. Absent any reason to prefer one over the other, divergence seemed acceptable to keep the implementation cleaner.

Here’s a simplified version of one of the ported breakpad tests, if you’re curious (thankfully minidump-synth is based off of the same binary data mocking framework these tests use):

#[test]
fn test_x86_frame_pointer() {
    let mut f = TestFixture::new();
    let frame0_ebp = Label::new();
    let frame1_ebp = Label::new();
    let mut stack = Section::new();

    // Setup the stack and registers so frame pointers will work
    stack.start().set_const(0x80000000);
    stack = stack
        .append_repeated(12, 0) // frame 0: space
        .mark(&frame0_ebp)      // frame 0 %ebp points here
        .D32(&frame1_ebp)       // frame 0: saved %ebp
        .D32(0x40008679)        // frame 0: return address
        .append_repeated(8, 0)  // frame 1: space
        .mark(&frame1_ebp)      // frame 1 %ebp points here
        .D32(0)                 // frame 1: saved %ebp (stack end)
        .D32(0);                // frame 1: return address (stack end)
    f.raw.eip = 0x4000c7a5;
    f.raw.esp = stack.start().value().unwrap() as u32;
    f.raw.ebp = frame0_ebp.value().unwrap() as u32;

    // Check the stackwalker's output:
    let s = f.walk_stack(stack).await;
    assert_eq!(s.frames.len(), 2);
    {
        let f0 = &s.frames[0];
        assert_eq!(f0.trust, FrameTrust::Context);
        assert_eq!(f0.context.valid, MinidumpContextValidity::All);
        assert_eq!(f0.instruction, 0x4000c7a5);
    }
    {
        let f1 = &s.frames[1];
        assert_eq!(f1.trust, FrameTrust::FramePointer);
        assert_eq!(f1.instruction, 0x40008678);
    }
}

A Dedicated Production Diffing, Simulating, and Debugging Tool

Because minidumps are so horribly fractal and corner-casey, I spent a lot of time terrified of subtle issues that would become huge disasters if we ever actually tried to deploy to production. So I also spent a bunch of time building socc-pair, which takes the id of a crash report from Mozilla’s crash reporting system and pulls down the minidump, the old breakpad-based implementation’s output, and extra metadata.

It then runs a local rust-minidump (minidump-stackwalk) implementation on the minidump and does a domain-specific diff over the two inputs. The most substantial part of this is a fuzzy diff on the stackwalks that tries to better handle situations like when one implementation adds an extra frame but the two otherwise agree. It also uses the reported techniques each implementation used to try to identify whose output is more trustworthy when they totally diverge.

I also ended up adding a bunch of mocking and benchmarking functionality to it as well, as I found more and more places where I just wanted to simulate a production environment.

Oh also I added really detailed trace-logging for the stackwalker so that I could easily post-mortem debug why it made the decisions it made.

This tool found so many issues and more importantly has helped me quickly isolate their causes. I am so happy I made it. Because of it, we know we actually fixed several issues that happened with the old breakpad implementation, which is great!

Here’s a trimmed down version of the kind of report socc-pair would produce (yeah I abused diff syntax to get error highlighting. It’s a great hack, and I love it like a child):

comparing json...

: {
    crash_info: {
        address: 0x7fff1760aca0
        crashing_thread: 8
        type: EXCEPTION_BREAKPOINT
    }
    crashing_thread: {
        frames: [
            0: {
                file: wrappers.cpp:1750da2d7f9db490b9d15b3ee696e89e6aa68cb7
                frame: 0
                function: RustMozCrash(char const*, int, char const*)
                function_offset: 0x00000010
-               did not match
+               line: 17
-               line: 20
                module: xul.dll

.....

    unloaded_modules: [
        0: {
            base_addr: 0x7fff48290000
-           local val was null instead of:
            code_id: 68798D2F9000
            end_addr: 0x7fff48299000
            filename: KBDUS.DLL
        }
        1: {
            base_addr: 0x7fff56020000
            code_id: DFD6E84B14000
            end_addr: 0x7fff56034000
            filename: resourcepolicyclient.dll
        }
    ]
~   ignoring field write_combine_size: "0"
}

- Total errors: 288, warnings: 39

benchmark results (ms):
    2388, 1986, 2268, 1989, 2353, 
    average runtime: 00m:02s:196ms (2196ms)
    median runtime: 00m:02s:268ms (2268ms)
    min runtime: 00m:01s:986ms (1986ms)
    max runtime: 00m:02s:388ms (2388ms)

max memory (rss) results (bytes):
    267755520, 261152768, 272441344, 276131840, 279134208, 
    average max-memory: 258MB (271323136 bytes)
    median max-memory: 259MB (272441344 bytes)
    min max-memory: 249MB (261152768 bytes)
    max max-memory: 266MB (279134208 bytes)

Output Files: 
    * (download) Minidump: b4f58e9f-49be-4ba5-a203-8ef160211027.dmp
    * (download) Socorro Processed Crash: b4f58e9f-49be-4ba5-a203-8ef160211027.json
    * (download) Raw JSON: b4f58e9f-49be-4ba5-a203-8ef160211027.raw.json
    * Local minidump-stackwalk Output: b4f58e9f-49be-4ba5-a203-8ef160211027.local.json
    * Local minidump-stackwalk Logs: b4f58e9f-49be-4ba5-a203-8ef160211027.log.txt

Staging and Deploying to Production

Once we were confident enough in the implementation, a lot of the remaining testing was taken over by Will Kahn-Greene, who’s responsible for a lot of the server-side details of our crash-reporting infrastructure.

Will spent a bunch of time getting a bunch of machinery setup to manage the deployment and monitoring of rust-minidump. He also did a lot of the hard work of cleaning up all our server-side configuration scripts to handle any differences between the two implementations. (Although I spent a lot of time on compatibility, we both agreed this was a good opportunity to clean up old cruft and mistakes.)

Once all of this was set up, he turned it on in staging and we got our first look at how rust-minidump actually worked in ~production:

Terribly!

Our staging servers take in about 10% of the inputs that also go to our production servers, but even at that reduced scale we very quickly found several new corner cases and we were getting tons of crashes, which is mildly embarrassing for the thing that handles other people’s crashes.

Will did a great job here in monitoring and reporting the issues. Thankfully they were all fairly easy for us to fix. Eventually, everything smoothed out and things seemed to be working just as reliably as the old implementation on the production server. The only places where we were completely failing to produce any output were for horribly truncated minidumps that may as well have been empty files.

We originally did have some grand ambitions of running socc-pair on everything the staging servers processed or something to get really confident in the results. But by the time we got to that point, we were completely exhausted and feeling pretty confident in the new implementation.

Eventually Will just said “let’s turn it on in production” and I said “AAAAAAAAAAAAAAA”.

This moment was pure terror. There had always been more corner cases. There’s no way we could just be done. This will probably set all of Mozilla on fire and delete Firefox from the internet!

But Will convinced me. We wrote up some docs detailing all the subtle differences and sent them to everyone we could. Then the moment of truth finally came: Will turned it on in production, and I got to really see how well it worked in production:

*dramatic drum roll*

It worked fine.

After all that stress and anxiety, we turned it on and it was fine.

Heck, I’ll say it: it ran well.

It was faster, it crashed less, and we even knew it fixed some issues.

I was in a bit of a stupor for the rest of that week, because I kept waiting for the other shoe to drop. I kept waiting for someone to emerge from the mist and explain that I had somehow bricked Thunderbird or something. But no, it just worked.

So we left for the holidays, and I kept waiting for it to break, but it was still fine.

I am honestly still shocked about this!

But hey, as it turns out we really did put a lot of careful work into testing the implementation. At every step we found new problems but that was good, because once we got to the final step there were no more problems to surprise us.

And the fuzzer still kicked our butts afterwards.

But that’s part 2! Thanks for reading!

 

The post Everything Is Broken: Shipping rust-minidump at Mozilla – Part 1 appeared first on Mozilla Hacks - the Web developer blog.

The Mozilla Blog — Firefox rolls out Total Cookie Protection by default to all users worldwide

Take back your privacy

Starting today, Firefox is rolling out Total Cookie Protection by default to all Firefox users worldwide, making Firefox the most private and secure major browser available across Windows, Mac and Linux. Total Cookie Protection is Firefox’s strongest privacy protection to date, confining cookies to the site where they were created, thus preventing tracking companies from using these cookies to track your browsing from site to site.

Whether it’s applying for a student loan, seeking treatment or advice through a health site, or browsing an online dating app, massive amounts of your personal information is online — and this data is leaking all over the web. The hyper-specific-to-you ads you so often see online are made possible by cookies that are used to track your behavior across sites and build an extremely sophisticated profile of who you are.

Recent stories (including an excellent Last Week Tonight episode) have shown how robust, yet under-the-radar, the data selling economy is and how easy it is for anyone to buy your data, combine it with more data about you and use it for a variety of purposes, even beyond advertising.

It’s an alarming reality — the possibility that your every move online is being watched, tracked and shared — and one that’s antithetical to the open web we at Mozilla have strived to build. That’s why we developed Total Cookie Protection to help keep you safe online.

What is Total Cookie Protection?

Total Cookie Protection offers strong protections against tracking without affecting your browsing experience.

Total Cookie Protection creates a separate cookie jar for each website you visit. (Illustration: Meghan Newell)

Total Cookie Protection works by creating a separate “cookie jar” for each website you visit. Instead of allowing trackers to link up your behavior on multiple sites, they just get to see behavior on individual sites. Any time a website, or third-party content embedded in a website, deposits a cookie in your browser, that cookie is confined to the cookie jar assigned to only that website. No other websites can reach into the cookie jars that don’t belong to them and find out what the other websites’ cookies know about you — giving you freedom from invasive ads and reducing the amount of information companies gather about you. 

This approach strikes the balance between eliminating the worst privacy properties of third-party cookies – in particular the ability to track you – and allowing those cookies to fulfill their less invasive use cases (e.g. to provide accurate analytics). With Total Cookie Protection in Firefox, people can enjoy better privacy and have the great browsing experience they’ve come to expect. 

Total Cookie Protection offers additional privacy protections beyond those provided by our existing anti-tracking features. Enhanced Tracking Protection (ETP), which we launched in 2018, works by blocking trackers based on a maintained list. If a party is on that list, they lose the ability to use third-party cookies. ETP was a huge privacy win for Firefox users, but we’ve known this approach has some shortcomings. If a tracker for some reason isn’t on that list, they can still track users and violate their privacy. And if an attacker wants to thwart ETP, they can set up a new tracking domain that isn’t on the list. Total Cookie Protection avoids these problems by restricting the functionality for all cookies, not just for those on a defined list.  

A Culmination of Years of Anti-Tracking Work

Total Cookie Protection is the culmination of years of work to fight the privacy catastrophe that stems from online trackers. We first began to block tracking in 2015 with the release of Tracking Protection, a feature people could turn on by going into Private Browsing mode. We recognized at that time that browser makers couldn’t just sit back and let their users be abused. In 2018, we introduced Enhanced Tracking Protection and turned it on by default for all Firefox users in 2019, reflecting our commitment to actively protect our users rather than expect them to protect themselves. Since then, we have continued to make progress towards blocking trackers and ending cross-site tracking by introducing protections against fingerprinting and supercookies. 

Today’s release of Total Cookie Protection is the result of experimentation and feature testing, first in ETP Strict Mode and Private Browsing windows, then in Firefox Focus earlier this year. We’re now making it a default feature for all Firefox desktop users worldwide.

Our long history of fighting online tracking manifests itself in our advocacy to policy makers and other tech companies to reinforce their own privacy protections. We also push to make privacy an industry priority through our efforts in industry standards bodies when shaping the future of online advertising. Furthermore, we created the Privacy Not Included guide to simplify the very complicated privacy landscape and help consumers shop smarter and safer for products that connect to the internet.

Over more than a decade, Mozilla has proudly been leading the fight to build a more private internet. Bringing Total Cookie Protection to all Firefox users is our next step towards creating a better internet, one where your privacy is not optional.

Take back your privacy by downloading Firefox today.

Firefox browser logo

Get Firefox

Get the browser that protects what’s important

The post Firefox rolls out Total Cookie Protection by default to all users worldwide appeared first on The Mozilla Blog.

The Mozilla Blog — Calling for Antitrust Reform

Mozilla supports the American Innovation and Choice Online Act (AICOA). The time for change is now.

It’s time for governments to address the reality that five tech companies—not everyday consumers—control our online experiences today. Updated competition laws are essential for the internet to be private, secure, interoperable, open, accessible, transparent, and a balance between commercial profit and public benefit. This is Mozilla’s vision for the internet. For a number of years, we have shared our views supporting government competition efforts globally to achieve it. 

One such proposal now under discussion in the US Congress is the American Innovation and Choice Online Act (AICOA). This bill is an important step in correcting two decades of digital centralization by creating a level playing field for smaller, independent software companies to compete. We support this bipartisan effort led by Senators Amy Klobuchar and Chuck Grassley and Representatives David Cicilline and Ken Buck. 

We believe that AICOA will facilitate innovation and consumer choice by ensuring that big tech companies cannot give preference to their own products and services over the rich diversity of competitive options offered by others. Mozilla—and many other independent companies—cannot effectively compete without this antitrust law. We are disadvantaged by the fact that current and future Firefox users, many of whom are privacy and security focused, cannot easily install and keep Firefox as their preferred browser because of confusing operating system messages and settings. We are further challenged by app store rules designed to keep out Gecko, our independent browser engine that powers Firefox, Tor and other browsers. We are stuck when big tech companies do not offer us and other developers open APIs and other functionality needed for true interoperability. 

A fair playing field is vital to ensure that Mozilla and other independent companies can continue to act as a counterweight to big tech and shape the future of the internet to be more private and more secure. We understand that the bill sponsors intend AICOA to regulate only gatekeeper companies and their controlled products. It is not intended to regulate or impact the agreements or product offerings of non-regulated independent companies like Mozilla that partner with gatekeepers for critical services. Nor does it require trading off privacy and security in order to enhance competition.

We disagree with the position taken by opponents to AICOA that competition legislation will undermine privacy and security. It is true that companies like Apple and Google offer key privacy features and security services that protect millions; for example, Apple’s App Tracking Transparency (ATT) approach and Google’s Safebrowsing service. Mozilla advocated for Apple to implement ATT, and Firefox (and all major browsers) use Safebrowsing. We do not believe these technologies would be impacted by AICOA because they can be provided without engaging in problematic self-preferencing behavior and because the bill includes clear protections for privacy and security.

Our view is that self-preferencing is preventing internet development from being more private and secure than it is today. For example, Mozilla was at the forefront of developing technology against cross-site tracking. Yet we have never released this technology to Firefox users on iOS because of App Store rules preferring Apple’s own browser engine over alternatives. As another example, Android’s affiliated browser Chrome does not offer anti-tracking technology. This leaves the majority of people on the planet without effective privacy protections. Real browser competition would empower millions to choose freely.

This year marks the 15th anniversary of the Mozilla Manifesto and two decades of our advocacy for a better internet. There has been progress in many areas, but the time has come for government action. 15 years of every major platform deciding for you that you should use their software is far too long. Enabling a level playing field for independent options is good for people’s online experiences, good for innovation and the economy, and ultimately good for a healthy, open internet.  We applaud those leading the charge on antitrust reform in the US and across the globe. The time for change is now.

The post Calling for Antitrust Reform appeared first on The Mozilla Blog.

Niko Matsakis — Async cancellation: a case study of pub-sub in mini-redis

Lately I’ve been diving deep into tokio’s mini-redis example. The mini-redis example is a great one to look at because it’s a realistic piece of quality async Rust code that is both self-contained and very well documented. Digging into mini-redis, I found that it exemplifies the best and worst of async Rust. On the one hand, the code itself is clean, efficient, and high-level. On the other hand, it relies on a number of subtle async conventions that can easily be done wrong – worse, if you do them wrong, you won’t get a compilation error, and your code will “mostly work”, breaking only in unpredictable timing conditions that are unlikely to occur in unit tests. Just the kind of thing Rust tries to avoid! This isn’t the fault of mini-redis – to my knowledge, there aren’t great alterantive patterns available in async Rust today (I go through some of the alternatives in this post, and their downsides).

Context: evaluating moro

We’ve heard from many users that async Rust has a number of pitfalls where things can break in subtle ways. In the Async Vision Doc, for example, the Barbara battles buffered streams and solving a deadlock stories discuss challenges with FuturesUnordered (wrapped in the buffered combinator); the Barbara gets burned by select and Alan tries to cache requests, which doesn’t always happen stories talk about cancellation hazards and the select! or race combinators.

In response to these stories, I created an experimental project called moro that explores structured concurrency in Rust. I’ve not yet blogged about moro, and that’s intentional. I’ve been holding off until I gain more confidence in moro’s APIs. In the meantime, various people (including myself) have been porting different bits of code to moro to get a better sense for what works and what doesn’t. GusWynn, for example, started changing bits of the materialize.io codebase to use moro and to have a safer alternative to cancellation. I’ve been poking at mini-redis, and I’ve also been working with some folks within AWS with some internal codebases.

What I’ve found so far is that moro absolutely helps, but it’s not enough. Therefore, instead of the triumphant blog post I had hoped for, I’m writing this one, which does a kind of deep-dive into the patterns that mini-redis uses: both how they work well when done right, but also how they are tedious and error-prone. I’ll be posting some follow-up blog posts that explore some of the ways that moro can help.

What is mini-redis?

If you’ve not seen it, mini-redis is a really cool bit of example code from the tokio project. It implements a “miniature” version of the redis in-memory data store, focusing on the key-value and pub-sub aspects of redis. Specifically, clients can connect to mini-redis and issue a subset of the redis commands. In this post, I’m going to focus on the “pub-sub” aspect of redis, in which clients can publish messages to a topic which are then broadcast to everyone who has subscribed to that topic. Whenever a client publishes a message, it receives in response the number of other clients that are currently subscribed to that topic.

Here is an example workflow involving two clients. Client 1 is subscribing to things, and Client 2 is publishing messages.

Core data structures

To implement this, the redis server maintains a struct State that is shared across all active clients. Since it is shared across all clients, it is maintained in a Mutex (source):

struct Shared {
    /// The shared state is guarded by a mutex. […]
    state: Mutex<State>,
    …
}

Within this State struct, there is a pub_sub field (source):

pub_sub: HashMap<String, broadcast::Sender<Bytes>>,

The pub_sub field stores a big hashmap. The key is the topic and the value is the broadcast::Sender, which is the “sender half” of a tokio broadcast channel. Whenever a client issues a publish command, it ultimately calls Db::publish, which winds up invoking send on this broadcast channel:

pub(crate) fn publish(&self, key: &str, value: Bytes) -> usize {
        let state = self.shared.state.lock().unwrap();
        state
            .pub_sub
            .get(key)
            // On a successful message send on the broadcast channel, the number
            // of subscribers is returned. An error indicates there are no
            // receivers, in which case, `0` should be returned.
            .map(|tx| tx.send(value).unwrap_or(0))
            // If there is no entry for the channel key, then there are no
            // subscribers. In this case, return `0`.
            .unwrap_or(0)
}

The subscriber loop

We just saw how, when clients publish data to a channel, that winds up invoking send on a broadcast channel. But how do the clients who are subscribed to that channel receive those messages? The answer lies in the Subscribe command.

The idea is that the server has a set subscriptions of subscribed channels for the client (source):

let mut subscriptions = StreamMap::new();

This is implemented using a tokio StreamMap, which is a neato data structure that takes multiple streams which each yield up values of type V, gives each of them a key K, and combines them into one stream that yields up (K, V) pairs. In this case, the streams are the “receiver half” of those broadcast channels, and the keys are the channel names.

When it receives a subscribe command, then, the server wants to do the following:

  • Add the receivers for each subscribed channel into subscriptions.
  • Loop:
    • If a message is published to subscriptions, then send it to the client.
    • If the client subscribes to new channels, add those to subscriptions and send an acknowledgement to client.
    • If the client unsubscribes from some channels, remove them from subscriptions and send an acknowledgement to client.
    • If the client terminates, end the loop and close the connection.

“Show me the state”

Learning to write Rust code is basically an exercise in asking “show me the state” — i.e., the key to making Rust code work is knowing what data is going to be modified and when1. In this case, there are a few key pieces of state…

  • The set subscriptions of “broadcast receivers” from each subscribed stream
    • There is also a set self.channels of “pending channel names” that ought to be subscribed to, though this is kind of an implementation detail and not essential.
  • The connection connection used to communicate with the client (a TCP socket)

And there are three concurrent tasks going on, each of which access that same state…

  • Looking for published messages from subscriptions and forwarding to connection (reads subscriptions, writes to connection)
  • Reading client commands from connection and then either…
    • subscribing to new channels (writes to subscriptions) and sending a confirmation (writes to connection);
    • or unsubscribing from channels (writes to subscriptions) and sending a confirmation (writes to connection).
  • Watching for termination and then cancelling everything (drops the broadcast handles in connections).

You can start to see that this is going to be a challenge. There are three conceptual tasks, but they are each needing mutable access to the same data:

If you tried to do this with normal threads, it just plain wouldn’t work…

let mut subscriptions = vec![]; // close enough to a StreamMap for now
std::thread::scope(|s| {
   s.spawn(|| subscriptions.push("key1"));
   s.spawn(|| subscriptions.push("key2"));
});

If you try this on the playground, you’ll see it gets an error because both closures are trying to access the same mutable state. No good. So how does it work in mini-redis?

Enter select!, our dark knight

Mini-redis is able to juggle these three threads through careful use of the select! macro. This is pretty cool, but also pretty error-prone — as we’ll see, there are a number of subtle points in the way that select! is being used here, and it’s easy to write the code wrong and have surprising bugs. At the same time, it’s pretty neat that we can use select! in this way, and it begs the question of whether we can find safer patterns to achieve the same thing. I think right now you can find safer ones, but they require less efficiency, which isn’t really living up to Rust’s promise (though it might be a good idea). I’ll cover that in a follow-up post, though, for now I just want to focus on explaining what mini-redis is doing and the pros and cons of this approach.

The main loop looks like this (source):

let mut subscriptions = StreamMap::new();
loop {
    …
    select! {
        Some((channel_name, msg)) = subscriptions.next() => ...
        //                          -------------------- future 1
        res = dst.read_frame() => ...
        //    ---------------- future 2
        _ = shutdown.recv() => ...
        //  --------------- future 3
    }
}

select! is kind of like a match statement. It takes multiple futures (underlined in the code above) and continues executing them until one of them completes. Since the select! is in a loop, and in this case each of the features are producing a series of events, this setup effectively runs the three futures concurrently, processing events as they arrive:

  • subscriptions.next() – the future waiting for the next message to arise to the StreamMap
  • dst.read_frame() – the async method read_frame is defined on the conection, dst. It reads data from the client, parses it into a complete command, and returns that command. We’ll dive into this function in a bit – it turns out that it is written in a very careful way to account
  • shutdown.recv() – the mini-redis server signals a global shutdown by threading a tokio channel to every connection; when a message is sent to that channel, all the loops cleanup and stop.

How select! works

So, select! runs multiple futures concurrently until one of them completes. In practice, this means that it iterates down the futures, one after the other. Each future gets awoken and runs until it either yields (meaning, awaits on something that isn’t ready yet) or completes. If the future yields, then select! goes to the next future and tries that one.

Once a future completes, though, the select! gets ready to complete. It begins by dropping all the other futures that were selected. This means that they immediately stop executing at whatever await point they reached, running any destructors for things on the stack. As I described in a previous blog post, in practice this feels a lot like a panic! that is injected at the await point. And, just like any other case of recovering from an exception, it requires that code is written carefully to avoid introducing bugs – tomaka describes one such example in his blog post. These bugs are what gives async cancellation in Rust a reputation for being difficult.

Cancellation and mini-redis

Let’s talk through what cancellation means for mini-redis. As we saw, the select! here is effectively running two distinct tasks (as well as waiting for shutdown):

  • Waiting on subscriptions.next() for a message to arrive from subscribed channels, so it can be forwarded to the client.
  • Waiting on dst.read_frame() for the next comand from the client, so that we can modify the set of subscribed channels.

We’ll see that mini-redis is coded carefully so that, whichever of these events occurs first, everything keeps working correctly. We’ll also see that this setup is fragile – it would be easy to introduce subtle bugs, and the compiler would not help you find them.

Take a look back at the sample subscription workflow at the start of this post. After Client1 has subscribed to A, the server is effectively waiting for Client1 to send further messages, or for other clients to publish.

The code that checks for further messages from Client1 is an async function called read_frame. It has to read the raw bytes sent by the client and assemble them into a “frame” (a single command). The read_frame in mini-redis is written in particular way:

  • It loops and, for each iteration…
    • tries to parse from a complete frame from self.buffer,
    • if self.buffer doesn’t contain a complete frame, then it reads more data from the stream into the buffer.

In pseudocode, it looks like (source):

impl Connection {
    async fn read_frame(&mut self) -> Result<Option<Frame>> {
        loop {
            if let Some(f) = parse_frame(&self.buffer) {
                return Ok(Some(f));
            }
            
            read_more_data_into_buffer(&mut self.buffer).await;
        }
    }
}

The key idea is that the function buffers up data until it can read an entire frame (i.e., successfully complete) and then it removes that entire frame at once. It never removes part of a frame from the buffer. This ensures that if the read_frame function is canceled while awaiting more data, nothing gets lost.

Ways to write a broken read_frame

There are many ways to a version of read_frame that is NOT cancel-safe. For example, instead of storing the buffer in self, one could put the buffer on the stack:

impl Connection {
    async fn read_frame(&mut self) -> Result<Option<Frame>> {
        let mut buffer = vec![];
        
        loop {
            if let Some(f) = parse_frame(&buffer) {
                return Ok(Some(f));
            }
            
            read_more_data_into_buffer(&mut buffer).await;
            //                                      -----
            //                If future is canceled here,
            //                buffer is lost.
        }
    }
}

This setup is broken because, if the future is canceled when awaiting more data, the buffered data is lost.

Alternatively, read_frame could intersperse reading from the stream and parsing the frame itself:

impl Connection {
    async fn read_frame(&mut self) -> Result<Option<Frame>> {
        let mut buffer = vec![];
        
        let command_name = self.read_command_name().await 
        match command_name {
            "subscribe" => self.parse_subscribe_command().await,
            "unsubscribe" => self.parse_unsubscribe_command().await,
            "publish" => self.parse_publish_command().await,
            ...
        }
    }
}

The problem here is similar: if we are canceled while awaiting one of the parse_foo_command futures, then we will forget the fact that we read the command_name already.

Comparison with JavaScript

It is interesting to compare Rust’s Future model with Javascript’s Promise model. In JavaScript, when an async function is called, it implicitly creates a new task. This task has “independent life”, and it keeps executing even if nobody ever awaits it. In Rust, invoking an async fn returns a Future, but that is inert. A Future only executes when some task awaits it. (You can create a task by invoking a suitable spawn method your runtime, and then it will execute on its own.)

There are really good reasons for Rust’s model: in particular, it is a zero-cost abstraction (or very close to it). In JavaScript, if you have one async function, and you factor out a helper function, you just went from one task to two tasks, meaning twice as much load on the scheduler. In Rust, if you have an async fn and you factor out a helper, you still have one task; you also still allocate basically the same amount of stack space. This is a good example of the “performant” (“idiomatic code runs efficiently”) Rust design principle in action.

However, at least as we’ve currently set things up, the Rust model does have some sharp edges. We’ve seen three ways to write read_frame, and only one of them works. Interestingly, all three of them would work in JavaScript, because in the JS model, an async function always starts a task and hence maintains its context.

I would argue that this represents a serious problem for Rust, because it represents a failure to maintain the “reliability” principle (“if it compiles, it works”), whigh ought to come first and foremost for us. The result is that async Rust feels a bit more like C or C++, where performant and versatile take top rank, and one has to have a lot of experience to know how to avoid sharp edges.

Now, I am not arguing Rust should adopt the “Promises” model – I think the Future model is better. But I think we need to tweak something to recover that reliability.

Comparison with threads

It’s interesting to compare how mini-redis with async Rust would compare to a mini-redis implemented with threads. It turns out that it would also be challenging, but in different ways. To start, let’s write up some pseudocode for what we are trying to do:

let mut subscriptions = StreamMap::new();

spawn(async move {
    while let Some((channel_name, msg)) = subscriptions.next().await {
        connection.send_message(channel_name, msg);
    }
});

spawn(async move {
    while let Some(frame) = connection.read_frame().await {
        match frame {
            Subscribe(new_channel) => subscribe(&mut connection, new_channel),
            Unsubscribe(channel) => unsubscribe(&mut connection, channel),
            _ => ...,
        }
    }
});

Here we have spawned out two threads, one of which is waiting for new messages from the subscriptions, and one of which is processing incoming client messages (which may involve adding channels the subscriptions map).

There are two problems here. First, you may have noticed I didn’t handle server shutdown! That turns out to be kind of a pain in this setup, because tearing down those spawns tasks is harder than you might think. For simplicity, I’m going to skip that for the rest of the post – it turns out that moro’s APIs solve this problem in a really nice way by allowing shutdown to be imposed externally without any deep changes.

Second, those two threads are both accessing subscriptions and connection in a mutable way, which the Rust compiler will not accept. This is a key problem. Rust’s type system works really well when you can breakdown your data such that every task accesses distinct data (i.e., “spatially disjoint”), either because each task owns the data or because they have &mut references to different parts of it. We have a much harder time dealing with multiple tasks accessing the same data but at different points in time (i.e., “temporally disjoint”).

Use an arc-mutex?

The main way to manage multiple tasks sharing access to the same data is with some kind of interior mutability, typically an Arc<Mutex<T>>. One problem with this is that it fails Rust’s performant design principle (“idiomatic code runs efficiently”), because there is runtime overhead (even if it is minimal in practice, it doesn’t feel good). Another problem with Arc<Mutex<T>> is that it hits on a lot of Rust’s ergonomic weak points, failing our “supportive” principle (“the language, tools, and community are here to help”):

  • You have to allocate the arcs and clone references explicitly, which is annoying;
  • You have to invoke methods like lock, get back lock guards, and understand how destructors and lock guards interact;
  • In Async code in particular, thanks to #57478, the compiler doesn’t understand very well when a lock guard has been dropped, resulting in annoying compiler errors – though Eric Holk is close to landing a fix for this one! :tada:

Of course, people who remember the “bad old days” of async Rust before async-await are very familiar with this dynamic. In fact, one of the big selling points of adding async await sugar into Rust was getting rid of the need to use arc-mutex.

Deeper problems

But the ergonomic pitfalls of Arc<Mutex> are only the beginning. It’s also just really hard to get Arc<Mutex> to actually work for this setup. To see what I mean, let’s dive a bit deeper into the state for mini-redis. There are two main bits of state we have to think about:

  • the tcp-stream to the client
  • the StreamMap of active connections

Managing access to the tcp-stream for the client is actually relatively easy. For one thing, tokio streams support a split operation, so it is possible to take the stream and split out the “sending half” (for sending messages to the client) and the “receiving half” (for receiving messages from the client). All the active threads can send data to the client, so they all need the sending half, and presumably it’ll be have to be wrapped in an (async aware) mutex. But only one active thread needs the receiving half, so it can own that, and avoid any locks.

Managing access to the StreamMap of active connections, though, is quite a bit more difficult. Imagine we were to put that StreamMap itself into a Arc<Mutex>, so that both tasks can access it. Now one of the tasks is going to be waiting for new messages to arrive. It’s going to look something like this:

let mut subscriptions = Arc::new(Mutex::new(StreamMap::new()));

spawn(async move {
    while let Some((channel_name, msg)) = subscriptions.lock().unwrap().next().await {
        connection.send_message(channel_name, msg);
    }
});

However, this code won’t compile (thankfully!). The problem is that we are acquiring a lock but we are trying to hold onto that lock while we await, which means we might switch to other tasks with the lock being held. This can easily lead to deadlock if those other tasks try to acquire the lock, since the tokio scheduler and the O/S scheduler are not cooprerating with one another.

An alternative would be to use an async-aware mutex like tokio::sync::Mutex, but that is also not great: we can still wind up with a deadlock, but for another reason. The server is now prevented from adding a new subscription to the list until the lock is released, which means that if Client1 is trying to subscribe to a new channel, it has to wait for some other client to send a message to an existing channel to do so (because that is when the lock is released). Not great.

Actually, this whole saga is covered under another async vision doc “status quo” story, Alan thinks he needs async locks.

A third alternative: actors

Recognizing the problems with locks, Alice Ryhl some time ago wrote a nice blog post, “Actors with Tokio”, that explains how to setup actors. This problem actually helps to address both our problems around mutable state. The idea is to move the connections array so that it belongs solely to one actor. Instead of directly modifying collections, the other tasks will communicate with this actor by exchanging messages.

So basically there could be two actors, or even three:

  • Actor A, which owns the connections (list of subscribed streams). It receives messages that are either publishing new messages to the streams or messages that say “add this stream” to the list.
  • Actor B, which owns the “read half” of the client’s TCP stream. It reads bytes and parses new frames, then sends out requests to the other actors in response. For example, when a subscribe message comes in, it can send a message to Actor A saying “subscribe the client to this channel”.
  • Actor C, which owns the “write half” of the client’s TCP stream. Both actors A and B will send messages to it when there are things to be sent to client.

To see how this would be implemented, take a look at Alice’s post. The TL;DR is that you would model connections between actors as tokio channels. Each actor is either spawned or otherwise setup to run independently. You still wind up using select!, but you only use it to receive messages from multiple channels at once. This doesn’t present any cancelation hazards because the channel code is carefully written to avoid them.

This setup works fine, and is even elegant in its own way, but it’s also not living up to Rust’s concept of performant or the goal of “zero-cost abstractions” (ZCA). In particular, the idea with ZCA is that it is supposed to give you a model that says “if you wrote this by hand, you couldn’t do any better”. But if you wrote a mini-redis server in C, by hand, you probably wouldn’t adopt actors. In some sense, this is just adopting something much closer to the Promise model. (Plus, the most obvious way to implement actors in tokio is largely to use tokio::spawn, which definitely adds overhead, or to use FuturesUnordered, which can be a bit subtle as well – moro does address these problems by adding a nice API here.)

(The other challenge with actors implemented this way is coordinating shutdown, though it can certainly be done: you just have to remember to thread the shutdown handler around everywhere.)

Cancellation as the “dark knight”: looking again at select!

Taking a step back, we’ve now seen that trying to use distinct tasks introduces this interesting problem that we have shared data being accessed by all the tasks. That either pushes us to locks (broken) or actors (works), but either way, it raises the question: why wasn’t this a problem with select!? After all, select! is still combining various logical tasks, and those tasks are still touching the same variables, so why is the compiler ok with it?

The answer is closely tied to cancellation: the select! setup works because

  • the things running concurrently are not touching overlapping state:
    • one of them is looking at subscriptions (waiting for a message);
    • another is looking at connection;
    • and the last one is receiving the termination message.
  • and once we decide which one of these paths to take, we cancel all the others.

This last part is key: if we receive an incoming message from the client, for example, we drop the future that was looking at subscriptions, canceling it. That means subscriptions is no longer in use, so we can push new subscriptions into it, or remove things from it.

So, cancellation is both what enables the mini-redis example to be performant and a zero-cost abstraction, but it is also the cause of our reliability hazards. That’s a pickle!

Conclusions

We’ve seen a lot of information, so let me try to sum it all up for you:

  • Fine-grained cancellation in select! is what enables async Rust to be a zero-cost abstraction and to avoid the need to create either locks or actors all over the place.
  • Fine-grained cancellation in select is the root cause for a LOT of reliability problems.

You’ll note that I wrote fine-grained cancellation. What I mean by that is specifically things like how select! will cancel the other futures. This is very different from coarse-grained cancellation like having the entire server shutdown, for which I think structured concurrency solves the problem very well.

So what can we do about fine-grained cancellation? Well, the answer depends.

In the short term, I value reliability above all, so I think adopting an actor-like pattern is a good idea. This setup can be a nice architecture for a lot of reasons2, and while I’ve described it as “not performant”, that assumes you are running a really high-scale server that has to handle a ton of load. For most applications, it will perform very well indeed.

I think it makes sense to be very judiciouis in what you select!! In the context of Materialize, GusWynn was experimenting with a Selectable trait for precisely this reason; that trait just permits select from a few sources, like channels. It’d be nice to support some convenient way of declaring that an async fn is cancel-safe, e.g. only allowing it to be used in select! if it is tagged with #[cancel_safe]. (This might be something one could author as a proc macro.)

But in the longer term, I’m interested if we can come up with a mechanism that will allow the compiler to get smarter. For example, I think it’d be cool if we could share one &mut across two async fn that are running concurrently, so long as that &mut is not borrowed across an await point. I have thoughts on that but…not for this post.

  1. My experience is that being forced to get a clear picture on this is part of what makes Rust code reliable in practice. ↩

  2. It’d be fun to take a look at Reactive Design Patterns and examine how many of them apply to Rust. I enjoyed that book a lot. ↩

Mozilla Thunderbird — Frequently Asked Questions: Thunderbird Mobile and K-9 Mail

Today, we announced our detailed plans for Thunderbird on mobile. We also welcomed the open-source Android email client K-9 Mail into the Thunderbird family. Below, you’ll find an evolving list of frequently asked questions about this collaboration and our future plans.

Revealed: Our Plans For Thunderbird On Android
Why not develop your own mobile client?

The Thunderbird team had many discussions on how we might provide a great mobile experience for our users. In the end, we didn’t want to duplicate effort if we could combine forces with an existing open-source project that shared our values. Over years of discussing ways K-9 and Thunderbird could collaborate, we decided it would best serve our users to work together.

Should I install K-9 Mail now or wait for Thunderbird?

If you want to help shape the future of Thunderbird on Android, you’re encouraged to install K-9 Mail right now. Leading up to the first official release of Thunderbird for Android, the user interface will probably change a few times. If you dislike somewhat frequent changes in apps you use daily, you might want to hold off.

Will this affect desktop Thunderbird? How?

Many Thunderbird users have asked for a Thunderbird experience on mobile, which we intend to provide by helping make K-9 amazing (and turning it into Thunderbird on Android). K-9 will supplement the Thunderbird experience and enhance where and how users are able to have a great email experience. Our commitment to desktop Thunderbird is unchanged, most of our team is committed to making that a best-in-class email client and it will remain that way.

What will happen to K-9 Mail once the official Thunderbird for Android app has been released?

K-9 Mail will be brought in-line with Thunderbird from a feature perspective, and we will ensure that syncing between Thunderbird and K-9/Thunderbird on Android is seamless. Of course, Thunderbird on Android and Thunderbird on Desktop are both intended to serve very different form factors, so there will be UX differences between the two. But we intend to allow similar workflows and tools on both platforms.

Will I be able to sync my Thunderbird accounts with K-9 Mail?

Yes. We plan to offer Firefox Sync as one option to allow you to securely sync accounts between Thunderbird and K-9 Mail. We expect this feature to be implemented in the summer of 2023.

Will Thunderbird for Android support calendars, tasks, feeds, or chat like the desktop app?

We are working on an amazing email experience first. We are looking at the best way to provide Thunderbird’s other functionality on Android but currently are still debating how best to achieve that. For instance, one method is to simply sync calendars, and then users are able to use their preferred calendar application on their device. But we have to discuss this within the team, and the Thunderbird and K-9 communities, then decide what the best approach is.

Going forward, how will K-9 Mail donations be used?

Donations made towards K-9 will be allocated to the Thunderbird project. Of course, Thunderbird in turn will provide full support for K-9 Mail’s development and activities that support the advancement and sustainability of the app.

Is a mobile Thunderbird app in development for iOS?

Thunderbird is currently evaluating the development of an iOS app.

How can I get involved?

1) Participate in our discussion and planning forum.
2) Developers are encouraged to visit https://developer.thunderbird.net to get started.
3) Obtain Thunderbird source code by visiting https://developer.thunderbird.net/thunderbird-development/getting-started.
4) K-9 Mail source code is available at: https://github.com/thundernest/k-9
5) You can financially support Thunderbird and K-9 Mail’s development by donating via this link: https://mzla.link/k9-give.

Thunderbird is the leading open-source, cross-platform email and calendaring client, free for business and personal use. We want it to stay secure and become even better. A donation will allow us to hire more developers, pay for infrastructure, expand our userbase, and continue to improve.

Click here to make a donation. 

The post Frequently Asked Questions: Thunderbird Mobile and K-9 Mail appeared first on The Thunderbird Blog.

Mozilla Thunderbird — Revealed: Our Plans For Thunderbird On Android

For years, we’ve wanted to extend Thunderbird beyond the desktop, and the path to delivering a great Thunderbird on Android™ experience started in 2018.

That’s when Thunderbird Product Manager Ryan Lee Sipes first met up with Christian Ketterer (aka “cketti”), the project maintainer for open-source Android email client K-9 Mail. The two instantly wanted to find a way for the two projects to collaborate. Throughout the following few years, the conversation evolved into how to create an awesome, seamless email experience across platforms.

But Ryan and cketti both agreed that the final product had to reflect the shared values of both projects. It had to be open source, respect the user, and be a perfect fit for power users who crave customization and a rich feature set.

“Ultimately,” Sipes says, “it made sense to work together instead of developing a mobile client from scratch.”

K-9 Mail Joins The Thunderbird Family

To that end, we’re thrilled to announce that today, K-9 Mail officially joins the Thunderbird family. And cketti has already joined the full-time Thunderbird staff, bringing along his valuable expertise and experience with mobile platforms.

Ultimately, K-9 Mail will transform into Thunderbird on Android.

That means the name itself will change and adopt Thunderbird branding. Before that happens, we need to reach certain development milestones that will bring K-9 Mail into alignment with Thunderbird’s feature set and visual appearance.

To accomplish that, we’ll devote finances and development time to continually improving K-9 Mail. We’ll be adding brand new features and introducing quality-of-life enhancements.

K-9 Mail and Thunderbird are both community-funded projects. If you want to help us improve and expand K-9 Mail faster, please consider donating at https://mzla.link/k9-give

Here’s a glimpse into our features roadmap:

  • Account setup using Thunderbird account auto-configuration.
  • Improved folder management.
  • Support for message filters.
  • Syncing between desktop and mobile Thunderbird.

“Joining the Thunderbird family allows K-9 Mail to become more sustainable and gives us the resources to implement long-requested features and fixes that our users want,” cketti says. “In other words, K-9 Mail will soar to greater heights with the help of Thunderbird.”

Thunderbird On Android: Join The Journey

Thunderbird users have long been asking for Thunderbird on their Android and iOS devices. This move allows Thunderbird users to have a powerful, privacy-respecting email experience today on Android. Plus, it lets the community help shape the transition of K-9 Mail into a fully-featured mobile Thunderbird experience.

This is only the beginning, but it’s a very exciting first step.

Want to talk directly with the Thunderbird team about it? Join us for a Twitter Spaces chat (via @MozThunderbird) on Wednesday, June 15 at 10am PDT / 1pm EDT / 7pm CEST). I’ll be there alongside cketti and Ryan to answer your questions, and discuss the future of Thunderbird on mobile devices.

Additional Links And Resources

FAQ: Frequently Asked Questions

Frequently Asked Questions: Thunderbird Mobile and K-9 Mail

We’ve published a separate FAQ here, addressing many of the community’s questions and concerns. Check back there from time to time, as we plan to update the FAQ as this collaboration progresses.

Thunderbird is the leading open-source, cross-platform email and calendaring client, free for business and personal use. We want it to stay secure and become even better. A donation will allow us to hire more developers, pay for infrastructure, expand our userbase, and continue to improve.

Click here to make a donation. 

The post Revealed: Our Plans For Thunderbird On Android appeared first on The Thunderbird Blog.

Will Kahn-Greene — Dennis v1.0.0 released! Retrospective! Handing it off!

What is it?

Dennis is a Python command line utility (and library) for working with localization. It includes:

  • a linter for finding problems in strings in .po files like invalid Python variable syntax which leads to exceptions

  • a template linter for finding problems in strings in .pot files that make translator's lives difficult

  • a statuser for seeing the high-level translation/error status of your .po files

  • a translator for strings in your .po files to make development easier

v1.0.0 released!

It's been 5 years since I released Dennis v0.9. That's a long time.

This brings several minor things and clean up. Also, I transferred the repository from "willkg" to "mozilla" in GitHub.

  • b38a678 Drop Python 3.5/3.6; add Python 3.9/3.10 (#122, #123, #124, #125)

  • b6d34d7 Redo tarrminal printin' and colorr (#71)

    There's an additional backwards-incompatible change here in which we drop the --color and --no-color arguments from dennis-cmd lint.

  • 658f951 Document dubstep (#74)

  • adb4ae1 Rework CI so it uses a matrix

  • transfer project from willkg to mozilla for ongoing maintenance and support

Retrospective

I worked on Dennis for 9 years.

It was incredibly helpful! It eliminated an entire class of bugs we were plagued with for critical Mozilla sites like AMO, MDN, SUMO, Input 1, and others. It did it in a way that supported and was respectful of our localization community.

It was pretty fun! The translation transforms are incredibly helpful for fixing layout issues. Some of them also produce hilarious results:

1

Input has gone to the happy hunting ground in the sky.

/images/sumo_dubstep1.thumbnail.png

SUMO in dubstep.

/images/sumo-pirate.thumbnail.png

SUMO in Pirate.

/images/sumo_zombie1.thumbnail.png

SUMO in Zombie.

There were a variety of dennis recipes including using it in a commit hook to translate commit messages. https://github.com/mozilla/dennis/commits/main

I enjoyed writing silly things at the bottom of all the release blog posts.

I learned a lot about gettext, localization, and languages! Learning about the nuances of plurals was fascinating.

The code isn't great. I wish I had redone the tokenization pipeline. I wish I had gotten around to adding support for other gettext variable formats.

Regardless, this project had a significant impact on Mozilla sites which I covered briefly in my Dennis Retrospective (2013).

Handing it off

It's been 6 years since I worked on sites that have localization, so I haven't really used Dennis in a long time and I'm no longer a stakeholder for it.

I need to reduce my maintenance load, so I looked into whether to end this project altogether. Several Mozilla projects still use it for linting PO files for deploys, so I decided not to end the project, but instead hand it off.

Welcome @diox and @akatsoulas who are picking it up!

Where to go for more

For more specifics on this release, see here: https://dennis.readthedocs.io/en/latest/changelog.html#version-1-0-0-june-10th-2022

Documentation and quickstart here: https://dennis.readthedocs.io/en/latest/

Source code and issue tracker here: https://github.com/mozilla/dennis

39 of 7,952,991,938 people were aware that Dennis existed but tens--nay, hundreds!--of millions were affected by it.

Steve Fink — Ephemeron Tables aka JavaScript WeakMaps and How They Work

Introduction I read Ephemerons explained today after finding it on Hacker News, and it was good but lengthy. It was also described in terms of the Squeak language and included post-mortem finalization, which is unavailable in JavaScript (and frankly sounds terrifying from an implementation point of view!) I thought I’d try my hand at writing […]

Karl Dubost — Get browsers version number on macOS (zsh)

I'm not sure why I had not written this before, but it kind of hit me when doing testing this week, that I could optimize a bit more my time.

statues in the forest with a red beanny.

This is a shell (zsh) script and macOS only. It reads the version information of a list of browsers and spills them out in a nice and ready to be copied and pasted in a bug report.

#!/bin/zsh

APP_PATH="/Applications/"
INFO_PATH=".app/Contents/Info.plist"
browsers=("Safari Technology Preview" "Firefox Nightly" "Google Chrome Canary" "Safari" "Firefox" "Google Chrome" "Microsoft Edge Canary")

for browser_name in ${(@k)browsers}; do
  full_path="${APP_PATH}${browser_name}${INFO_PATH}" ;
  if test -f "$full_path"; then
    browser_version=$(defaults read "$full_path" CFBundleShortVersionString);
    echo "${browser_name} ${browser_version}";
  fi
done

What it looks like once rendered. I need to update a couple of things.

Safari Technology Preview 15.4
Firefox Nightly 103.0a1
Safari 15.5
Firefox 99.0
Microsoft Edge Canary 104.0.1285.0

Otsukare!

Mozilla Addons Blog — Manifest V3 Firefox Developer Preview — how to get involved

Firefox logoWhile MV3 is still in development, many major features are already included in the Developer Preview, which provides an opportunity to expose functionality for testing and feedback. With strong developer feedback, we’re better equipped to quickly address critical bug fixes, provide clear developer documentation, and reorient functionality.

Some features, such as a well defined and documented lifecycle for Event Pages, are still works in progress. As we complete features, they’ll land in future versions of Firefox and you’ll be able to test and progress your extensions into MV3 compatibility. In most ways Firefox is committed to MV3 cross browser compatibility. However in some cases Firefox will offer distinct extension functionality.

Developer Preview is not available to regular users; it requires you to change preferences in about:config. Thus you will not be able to upload MV3 extensions to addons.mozilla.org (AMO) until we have an official release available to users.

The following are key considerations about migration at this time and areas we’d greatly appreciate developer feedback.

  1. Read the MV3 migration guide. MV3 contains many changes and our migration guide covers the major necessary steps, as well as linking to documentation to help understand further details.
  2. Update your extension to be compatible with Event Pages. One major difference in Firefox is our use of Event Pages, which provides an alternative to the existing Background Pages that allows idle timeouts and page restarts. This adds resilience to the background, which is necessary for resource constraints and mobile devices. For the most part, Event Pages are compatible with existing Background Pages, requiring only minor changes. We plan to release Event Pages for MV2 in an upcoming Firefox release, so preparation to use Event Pages can be included in MV2 addons soon. Many extensions may not need all the capabilities available in Event Pages. The background scripts are easily transferable to the Service Worker background when it becomes available in a future release. In the meantime, extensions attempting to support both Chrome and Firefox can take advantage of Event Pages in Firefox.
  3. Test your content scripts with MV3. There are multiple changes that will impact content scripts, ranging from tighter restrictions on CORS, CSP, remote code execution, and more. Not all extensions will run into issues in these cases, and some may only require minor modifications that will likely work within MV2 as well.
  4. Understand and consider your migration path for APIs that have changed or deprecated. Deprecated APIs will require code changes to utilize alternate or new APIs. Examples include New Scripting API (which will be part of MV2 in a future release), changing page and browser actions to the action API, etc.
  5. Test and plan migration for permissions. Most permissions are already available as optional permissions in MV2. With MV3, we’re making host permissions optional — in many cases by default. While we do not yet have the primary UI for user control in Developer Preview, developers should understand how these changes will affect their extensions.
  6. Let us know how it’s going! Your feedback will help us make the transition from MV2 to MV3 as smooth as possible. Through Developer Preview we anticipate learning about MV3 rough edges, documentation needs, new features to be fleshed out, and bugs to be fixed. We have a host of community channels you can access to ask questions, help others, report problems, or whatever else you desire to communicate as it relates to the MV3 migration process.

Stay in touch with us on any of these forums…

 

The post Manifest V3 Firefox Developer Preview — how to get involved appeared first on Mozilla Add-ons Community Blog.

This Week In Rust — This Week in Rust 446

Hello and welcome to another issue of This Week in Rust! Rust is a programming language empowering everyone to build reliable and efficient software. This is a weekly summary of its progress and community. Want something mentioned? Tweet us at @ThisWeekInRust or send us a pull request. Want to get involved? We love contributions.

This Week in Rust is openly developed on GitHub. If you find any errors in this week's issue, please submit a PR.

Updates from Rust Community

Official
Newsletters
Project/Tooling Updates
Observations/Thoughts
Rust Walkthroughs
Research
Miscellaneous

Crate of the Week

This week's crate is osmpbf an OpenStreetMap pbf-file reader.

Thanks to Kornel for the suggestion.

Please submit your suggestions and votes for next week!

Call for Participation

Always wanted to contribute to open-source projects but didn't know where to start? Every week we highlight some tasks from the Rust community for you to pick and get started!

Some of these tasks may also have mentors available, visit the task page for more information.

If you are a Rust project owner and are looking for contributors, please submit tasks here.

Updates from the Rust Project

385 pull requests were merged in the last week

Rust Compiler Performance Triage

A busy week in compiler performance, but fortunately improvements outweighed regressions. The biggest improvements came from @nnethercote's work on making the decoding of SourceFile::lines lazy which significantly cuts the costs of decoding crate metadata. The biggest regressions came from the removal of json handling in rustc_serialize which has been a multi-month effort to improve the maintainability of json (de-)serialization in the compiler.

Triage done by @rylev. Revision range: 0a43923a..bb55bd

Summary:

mean max count
Regressions 😿
(primary)		 0.5% 3.2% 36
Regressions 😿
(secondary)		 0.3% 0.9% 15
Improvements 🎉
(primary)		 -1.3% -15.1% 124
Improvements 🎉
(secondary)		 -2.7% -13.5% 182
All 😿🎉 (primary) -0.9% -15.1% 160

2 Regression, 6 Improvements, 5 Mixed; 4 of them in rollups 48 artifact comparisons made in total

Full report here

Call for Testing

An important step for RFC implementation is for people to experiment with the implementation and give feedback, especially before stabilization. The following RFCs would benefit from user testing before moving forward:

If you are a feature implementer and would like your RFC to appear on the above list, add the new call-for-testing label to your RFC along with a comment providing testing instructions and/or guidance on which aspect(s) of the feature need testing.

Approved RFCs

Changes to Rust follow the Rust RFC (request for comments) process. These are the RFCs that were approved for implementation this week:

  • No RFCs were approved this week.
Final Comment Period

Every week the team announces the 'final comment period' for RFCs and key PRs which are reaching a decision. Express your opinions now.

RFCs
Tracking Issues & PRs
New and Updated RFCs
  • No New or Updated RFCs were created this week.

Upcoming Events

Rusty Events between 2022-06-08 - 2022-07-06 🦀

Virtual
North America
Europe
Oceania

If you are running a Rust event please add it to the calendar to get it mentioned here. Please remember to add a link to the event too. Email the Rust Community Team for access.

Rust Jobs

Spire Global

Meilisearch

Stockly

Nikulipe

GMEX Group

Enso

Kraken

Tweet us at @ThisWeekInRust to get your job offers listed here!

Quote of the Week

I wrote a bespoke time-series database in Rust a few years ago, and it has had exactly one issue since I stood it up in production, and that was due to pessimistic filesystem access patterns, rather than the language. This thing is handling hundreds of thousands of inserts per second, and it's even threaded.

Given that I've been programming professionally for over a decade in Python, Perl, Ruby, C, C++, Javascript, Java, and Rust, I'll pick Rust absolutely any time that I want something running that I won't get called at 3 AM to fix. It probably took me 5 times as long to write it as if I did it in Go or Python, but I guarantee it's saved me 10 times as much time I would have otherwise spent triaging, debugging, and running disaster recovery.

– Taywee on hacker news

Thanks to Erich Gubler for the suggestion.

Please submit quotes and vote for next week!

This Week in Rust is edited by: nellshamrell, llogiq, cdmistman, ericseppanen, extrawurst, andrewpollack, U007D, kolharsam, joelmarcey, mariannegoldin.

Email list hosting is sponsored by The Rust Foundation

Discuss on r/rust

Hacks.Mozilla.Org — Training efficient neural network models for Firefox Translations

Machine Translation is an important tool for expanding the accessibility of web content. Usually, people use cloud providers to translate web pages. State-of-the-art Neural Machine Translation (NMT) models are large and often require specialized hardware like GPUs to run inference in real-time.

If people were able to run a compact Machine Translation (MT) model on their local machine CPU without sacrificing translation accuracy it would help to preserve privacy and reduce costs.

The Bergamot project is a collaboration between Mozilla, the University of Edinburgh, Charles University in Prague, the University of Sheffield, and the University of Tartu with funding from the European Union’s Horizon 2020 research and innovation programme. It brings MT to the local environment, providing small, high-quality, CPU optimized NMT models. The Firefox Translations web extension utilizes proceedings of project Bergamot and brings local translations to Firefox.

In this article, we will discuss the components used to train our efficient NMT models. The project is open-source, so you can give it a try and train your model too!

Architecture

NMT models are trained as language pairs, translating from language A to language B. The training pipeline was designed to train translation models for a language pair end-to-end, from environment configuration to exporting the ready-to-use models. The pipeline run is completely reproducible given the same code, hardware and configuration files.

The complexity of the pipeline comes from the requirement to produce an efficient model. We use Teacher-Student distillation to compress a high-quality but resource-intensive teacher model into an efficient CPU-optimized student model that still has good translation quality. We explain this further in the Compression section.

The pipeline includes many steps: compiling of components, downloading and cleaning datasets, training teacher, student and backward models, decoding, quantization, evaluation etc (more details below). The pipeline can be represented as a Directly Acyclic Graph (DAG).

 

Firfox Translation training pipeline DAG

The workflow is file-based and employs self-sufficient scripts that use data on disk as input, and write intermediate and output results back to disk.

We use the Marian Neural Machine Translation engine. It is written in C++ and designed to be fast. The engine is open-sourced and used by many universities and companies, including Microsoft.

Training a quality model

The first task of the pipeline is to train a high-quality model that will be compressed later. The main challenge at this stage is to find a good parallel corpus that contains translations of the same sentences in both source and target languages and then apply appropriate cleaning procedures.

Datasets

It turned out there are many open-source parallel datasets for machine translation available on the internet. The most interesting project that aggregates such datasets is OPUS. The Annual Conference on Machine Translation also collects and distributes some datasets for competitions, for example, WMT21 Machine Translation of News. Another great source of MT corpus is the Paracrawl project.

OPUS dataset search interface:

OPUS dataset search interface

It is possible to use any dataset on disk, but automating dataset downloading from Open source resources makes adding new language pairs easy, and whenever the data set is expanded we can then easily retrain the model to take advantage of the additional data. Make sure to check the licenses of the open-source datasets before usage.

Data cleaning

Most open-source datasets are somewhat noisy. Good examples are crawled websites and translation of subtitles. Texts from websites can be poor-quality automatic translations or contain unexpected HTML, and subtitles are often free-form translations that change the meaning of the text.

It is well known in the world of Machine Learning (ML) that if we feed garbage into the model we get garbage as a result. Dataset cleaning is probably the most crucial step in the pipeline to achieving good quality.

We employ some basic cleaning techniques that work for most datasets like removing too short or too long sentences and filtering the ones with an unrealistic source to target length ratio. We also use bicleaner, a pre-trained ML classifier that attempts to indicate whether the training example in a dataset is a reversible translation. We can then remove low-scoring translation pairs that may be incorrect or otherwise add unwanted noise.

Automation is necessary when your training set is large. However, it is always recommended to look at your data manually in order to tune the cleaning thresholds and add dataset-specific fixes to get the best quality.

Data augmentation

There are more than 7000 languages spoken in the world and most of them are classified as low-resource for our purposes, meaning there is little parallel corpus data available for training. In these cases, we use a popular data augmentation strategy called back-translation.

Back-translation is a technique to increase the amount of training data available by adding synthetic translations. We get these synthetic examples by training a translation model from the target language to the source language. Then we use it to translate monolingual data from the target language into the source language, creating synthetic examples that are added to the training data for the model we actually want, from the source language to the target language.

The model

Finally, when we have a clean parallel corpus we train a big transformer model to reach the best quality we can.

Once the model converges on the augmented dataset, we fine-tune it on the original parallel corpus that doesn’t include synthetic examples from back-translation to further improve quality.

Compression

The trained model can be 800Mb or more in size depending on configuration and requires significant computing power to perform translation (decoding). At this point, it’s generally executed on GPUs and not practical to run on most consumer laptops. In the next steps we will prepare a model that works efficiently on consumer CPUs.

Knowledge distillation

The main technique we use for compression is Teacher-Student Knowledge Distillation. The idea is to decode a lot of text from the source language into the target language using the heavy model we trained (Teacher) and then train a much smaller model with fewer parameters (Student) on these synthetic translations. The student is supposed to imitate the teacher’s behavior and demonstrate similar translation quality despite being significantly faster and more compact.

We also augment the parallel corpus data with monolingual data in the source language for decoding. This improves the student by providing additional training examples of the teacher’s behavior.

Ensemble

Another trick is to use not just one teacher but an ensemble of 2-4 teachers independently trained on the same parallel corpus. It can boost quality a little bit at the cost of having to train more teachers. The pipeline supports training and decoding with an ensemble of teachers.

Quantization

One more popular technique for model compression is quantization. We use 8-bit quantization which essentially means that we store weights of the neural net as int8 instead of float32. It saves space and speeds up matrix multiplication on inference.

Other tricks

Other features worth mentioning but beyond the scope of this already lengthy article are the specialized Neural Network architecture of the student model, half-precision decoding by the teacher model to speed it up, lexical shortlists, training of word alignments, and finetuning of the quantized student.

Yes, it’s a lot! Now you can see why we wanted to have an end-to-end pipeline.

How to learn more

This work is based on a lot of research. If you are interested in the science behind the training pipeline, check out reference publications listed in the training pipeline repository README and across the wider Bergamot project. Edinburgh’s Submissions to the 2020 Machine Translation Efficiency Task is a good academic starting article. Check this tutorial by Nikolay Bogoychev for a more practical and operational explanation of the steps.

Results

The final student model is 47 times smaller and 37 times faster than the original teacher model and has only a small quality decrease!

Benchmarks for en-pt model and Flores dataset:

Model Size Total number of parameters Dataset decoding time on 1 CPU core Quality, BLEU
Teacher 798Mb 192.75M 631s 52.5
Student quantized 17Mb 15.7M 17.9s 50.7

We evaluate results using MT standard BLEU scores that essentially represent how similar translated and reference texts are. This method is not perfect but it has been shown that BLEU scores correlate well with human judgment of translation quality.

We have a GitHub repository with all the trained models and evaluation results where we compare the accuracy of our models to popular APIs of cloud providers. We can see that some models perform similarly, or even outperform, the cloud providers which is a great result taking into account our model’s efficiency, reproducibility and open-source nature.

For example, here you can see evaluation results for the English to Portuguese model trained by Mozilla using open-source data only.

Evaluation results en-pt

Anyone can train models and contribute them to our repo. Those contributions can be used in the Firefox Translations web extension and other places (see below).

Scaling

It is of course possible to run the whole pipeline on one machine, though it may take a while. Some steps of the pipeline are CPU bound and difficult to parallelize, while other steps can be offloaded to multiple GPUs. Most of the official models in the repository were trained on machines with 8 GPUs. A few steps, like teacher decoding during knowledge distillation, can take days even on well-resourced single machines. So to speed things up, we added cluster support to be able to spread different steps of the pipeline over multiple nodes.

Workflow manager

To manage this complexity we chose Snakemake which is very popular in the bioinformatics community. It uses file-based workflows, allows specifying step dependencies in Python, supports containerization and integration with different cluster software. We considered alternative solutions that focus on job scheduling, but ultimately chose Snakemake because it was more ergonomic for one-run experimentation workflows.

Example of a Snakemake rule (dependencies between rules are inferred implicitly):

rule train_teacher:
    message: "Training teacher on all data"
    log: f"{log_dir}/train_teacher{{ens}}.log"
    conda: "envs/base.yml"
    threads: gpus_num*2
    resources: gpu=gpus_num
    input:
        rules.merge_devset.output, 
        train_src=f'{teacher_corpus}.{src}.gz',
        train_trg=f'{teacher_corpus}.{trg}.gz',
        bin=ancient(trainer), 
        vocab=vocab_path
    output: model=f'{teacher_base_dir}{{ens}}/{best_model}'
    params: 
        prefix_train=teacher_corpus, 
        prefix_test=f"{original}/devset", 
        dir=directory(f'{teacher_base_dir}{{ens}}'),
        args=get_args("training-teacher-base")
    shell: '''bash pipeline/train/train.sh \
                teacher train {src} {trg} "{params.prefix_train}" \
                "{params.prefix_test}" "{params.dir}" \
                "{input.vocab}" {params.args} >> {log} 2>&1'''

Cluster support

To parallelize workflow steps across cluster nodes we use Slurm resource manager. It is relatively simple to operate, fits well for high-performance experimentation workflows, and supports Singularity containers for easier reproducibility. Slurm is also the most popular cluster manager for High-Performance Computers (HPC) used for model training in academia, and most of the consortium partners were already using or familiar with it.

How to start training

The workflow is quite resource-intensive, so you’ll need a pretty good server machine or even a cluster. We recommend using 4-8 Nvidia 2080-equivalent or better GPUs per machine.

Clone https://github.com/mozilla/firefox-translations-training and follow the instructions in the readme for configuration.

The most important part is to find parallel datasets and properly configure settings based on your available data and hardware. You can learn more about this in the readme.

How to use the existing models

The existing models are shipped with the Firefox Translations web extension, enabling users to translate web pages in Firefox. The models are downloaded to a local machine on demand. The web extension uses these models with the bergamot-translator Marian wrapper compiled to Web Assembly.

Also, there is a playground website at https://mozilla.github.io/translate where you can input text and translate it right away, also locally but served as a static website instead of a browser extension.

If you are interested in an efficient NMT inference on the server, you can try a prototype HTTP service that uses bergamot-translator natively compiled, instead of compiled to WASM.

Or follow the build instructions in the bergamot-translator readme to directly use the C++, JavaScript WASM, or Python bindings.

Conclusion

It is fascinating how far Machine Translation research has come in recent years. Local high-quality translations are the future and it’s becoming more and more practical for companies and researchers to train such models even without access to proprietary data or large-scale computing power.

We hope that Firefox Translations will set a new standard of privacy-preserving, efficient, open-source machine translation accessible for all.

Acknowledgements

I would like to thank all the participants of the Bergamot Project for making this technology possible, my teammates Andre Natal and Abhishek Aggarwal for the incredible work they have done bringing Firefox Translations to life, Lonnen for managing the project and editing this blog post and of course awesome Mozilla community for helping with localization of the web-extension and testing its early builds.

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 825303 🇪🇺

The post Training efficient neural network models for Firefox Translations appeared first on Mozilla Hacks - the Web developer blog.

Mozilla Thunderbird — Welcome To The Thunderbird 102 Beta! Resources, Links, And Guides

The wait for this year’s major new Thunderbird release is almost over! But you can test-drive many of the new features like the brand new Address Book, Matrix Chat support, import/export wizard, and refreshed visuals right now with the Thunderbird 102 Beta. Better still, you might be directly responsible for improving the final product via your feedback and bug reports.

Below, you’ll find all the resources you need for testing the Thunderbird 102 Beta. From technical guides to a community feedback forum to running the beta side-by-side with your existing stable version, we’ve got you covered.

Do you feel something is missing from this list? Please leave a comment here, or email me personally (jason at thunderbird dot net). I’ll make sure we get it added!

Thunderbird 102 Beta: First Steps

Here are some first steps and important considerations to take into account before deciding to install the beta.

Thunderbird 102 Beta: Guides, Links, And Resources

We want you to have the smoothest beta experience possible. Whether you’re reporting bugs, seeking solutions, or trying to run beta side-by-side with an existing Thunderbird installation, these resources should help.

From all of us here at Thunderbird: Have fun, and happy testing!

The post Welcome To The Thunderbird 102 Beta! Resources, Links, And Guides appeared first on The Thunderbird Blog.

Cameron Kaiser — macOS Oxnard

Those of us in southern California are shaking our heads. I like the City of (San Buena)Ventura fine, but Ventura isn't exactly in the same snobbish class as, you know, Big Sur or Monterey. I mean, if they really wanted they could have had macOS Camarillo, or macOS Thousand Oaks, or maybe even macOS Calabasas even though that sounds those gourd urns you buy at Pier 1 or a euphemism for unmentionable body parts, but anyway. (macOS Malibu! Buy all her friends!) There was a lot of buzz over the possibility this could have been macOS Mammoth, and even Wikipedia went all-in. I can see why they didn't because the jokes would have flown thick and heavy if it turns out as big and ponderous as the name, but when you give up the chance at Point Mugu or Oxnard (or La Conchita: the operating system that collapses on itself!) for a pretty if less ostentatious coastal community, someone at Apple just isn't thinking big. Or, for that matter, mammoth.

Support.Mozilla.Org — Introducing Ryan Johnson

Hi folks,

Please join me to welcome Ryan Johnson to the Customer Experience team as a Staff Software Engineer. He will be working closely with Tasos to maintain and improve the Mozilla Support platform.

Here’s a short intro from Ryan:

Hello everyone! I’m Ryan Johnson, and I’m joining the SUMO engineering team as a Staff Software Engineer. This is a return to Mozilla for me, after a brief stint away, and I’m excited to work with Tasos and the rest of the Customer Experience team in reshaping SUMO to better serve the needs of Mozilla and all of you. In my prior years at Mozilla, I was fortunate to work on the MDN team, and with many of its remarkable supporters, and this time I look forward to meeting, working with, and learning from many of you.

Once again, please join me to congratulate and welcome Ryan to our team!

Mozilla Performance Blog — Prioritized Task Scheduling API is Prototyped in Nightly

In bug 1734997, we prototyped the Prioritized Task Scheduling API. Having the API prototyped empowers us to experiment with the benefits of this API. It is enabled by default in Nightly at the moment.

Motivation

Prioritized Task Scheduling API allows web developers to control and schedule prioritized tasks in a united and flexible way. We believe this API is going to benefit the web because firstly it has valid use cases as large web frameworks (eg. React has attempted to implement a custom version of it before the spec is standardized). Secondly, the current task scheduling primitives (eg. setTimeout and requestIdleCallback) are inadequate. We think this API is going to provide a more robust solution for scheduling tasks.

All About the API

The scheduler interface is the centralized controller for this API which lives in every window and worker scope. The scheduler interface provides the postTask function that allows us to post tasks with not only different priorities but AbortSignal to allow us to cancel tasks when they are not needed.

Let’s look at some usage of this API. To post a task, we can do something like:

function task() {}
scheduler.postTask(task); // post a task with `user-visible` priority

Prioritized Task Scheduling API provides 3 levels of priorities, they are

  • user-blocking
  • user-visible
  • background

user-blocking is the highest priority, user-visible is the second-highest priority and background is the lowest priority. Each task has a priority associated and user-visible is the default priority.

To post a task with different priorities, we can do something like

function task() {}
await scheduler.postTask(task, {priority: ‘user-blocking’}); const promise = scheduler.postTask(task, {priority: ‘background’});

postTask function also allows us to delay the running of a task for x milliseconds. Sample usage of that is

scheduler.postTask(task, {delay: 100}); // delay the running of the task for 100 milliseconds

Another key concept in postTask is AbortSignal. When using the postTask function, we can pass an AbortSignal instance to allow us to cancel the task after the task is posted.

To pass an AbortSignal and cancel the task afterwards, we could do it as

function task() {}
const aboutController = new AbortController(); // AbortSignal is created via AbortContoller
const promise = scheduler.postTask(task, {signal: aboutController.signal});
aboutController.abort();
// promise is rejected

In addition to canceling the task, we could also pass a TaskSignal to change the priority afterwards. This can be done as

function task() {}
const taskController = new TaskController({priority: 'background'}); // TaskSignal is created via TaskControllerconst promise = scheduler.postTask(task, {signal: taskController.signal});
// TaskController inherits AbortController, so taskContoller.abort() also can be used to cancel the tasktaskController.setPriority("user-blocking");
// A prioritychange event is fired on taskController.signal

Again, this API is behind the dom.enable_web_task_scheduling pref and has only been enabled by default in Nightly since Firefox 101.

What’s Next?

So far we’ve learnt that Airbnb has been using this API with success stories. We will continue to learn and experiment how this API is used in the wild and make decisions based on our learnings. Thanks for your reading and please stay tuned for our future Performance API updates.

Firefox Nightly — These Weeks In Firefox: Issue 117

Highlights

Friends of the Firefox team

Introductions/Shout-Outs
  • Welcome Janvi [:janvi01] from Outreachy, who will be helping us improve Picture-in-Picture!
Resolved bugs (excluding employees)
Volunteers that fixed more than one bug
  • Itiel
  • Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]
  • Shane Hughes [:aminomancer]
New contributors (🌟 = first patch)

Project Updates

Add-ons / Web Extensions
WebExtensions Framework
  • wasm-unsafe-eval CSP directive and extensions:
    • Behaviors expected for manifest_version 2 and manifest_version 3 extensions has been described in Bug 1766027
      • “manifest_version: 2” extensions will still be able to use wasm unconditionally, “manifest_version: 3”  extensions will have to specify `wasm-unsafe-eval` in their own custom CSP
    • A regression on manifest_version 2 extensions using their own custom CSP has been fixed in Bug 1770468 (fixed in Nightly 102 itself)
  • Fixed a bug related to optional permissions changes not propagated to preloaded child processes – Bug 1766822
  • Fixed a fission-related regression on extension pages moved to BFCache as a side-effect of a cross-process navigation – Bug 1768522

 

WebExtension APIs
  • Igorlogius fixed menus API support for registering context menus items on the bookmarks. Thanks to Igorlogius for contributing this fix! – Bug 1768371
Fluent
  • Fewer than 190 DTD strings left in the entirety of mozilla-central!
Form Autofill
PDFs & Printing
  • Hanna fixed an issue with margins being invalid on small paper sizes (like receipt printers) Bug 1756169
  • Emilio fixed an issue where you margins and headers/footers couldn’t be changed when printing a PDF Bug 1752379
Picture-in-Picture
Performance Tools (aka Firefox Profiler)
Privacy/Security
  • Hannah landed a patch to migrate Nightly users from the network cookie lifetime policy to the Sanitizer (Bug 1681498)
    • This is the first step for removing the deprecated cookie lifetime policy feature. We want to remove the feature entirely in Fx 103. It will be replaced by the sanitize-on-shutdown mechanism, previously under about:preferences#privacy -> History
Search and Navigation
ESMification status
  • This is a project to switch us away from the Mozilla-specific JSMs to ES6 modules in our privileged code. The infrastructure has been landing slowly but surely into mozilla-central to help with the transition.
    • Here is the metabug for the effort
    • After much deliberation, it seems these types of modules will have the file extension “.sys.mjs”. Example: “PictureInPicture.sys.mjs”.
    • Expect a mailing list post soon, and instructions on how you can help with getting us onto ES6 modules.

Mozilla Open Policy & Advocacy Blog — Enhancing trust and security on the internet – browsers are the first line of defence

Enhancing trust and security online is one of the defining challenges of our time – in the EU alone, 37% of residents do not believe they can sufficiently protect themselves from cybercrime. Individuals need assurance that their credit card numbers, social media logins, and other sensitive data are protected from cybercriminals when browsing. With that in mind, we’ve just unveiled an update to the security policies that protect people from cybercrime, demonstrating again the critical role Firefox plays in ensuring trust and security online.

Browsers like Firefox use encryption to protect individuals’ data from eavesdroppers when they navigate online (e.g. when sending credit card details to an online marketplace). But protecting data from cybercriminals when it’s on the move is only part of the risk we mitigate. Individuals also need assurance that they are sending data to the correct domain (e.g., “amazon.com”). If someone sends their private data to a cybercriminal instead of to their bank, for example, it is of little consolation that the data was encrypted while getting there.

To address this we rely on cryptographic website certificates, which allow a website to prove that it controls the domain name that the individual has navigated to. Websites obtain these certificates from certificate authorities, organisations that run checks to verify that websites are not compromised. Certificate authorities are a critical pillar of trust in this ecosystem – if they mis-issue certificates to cybercriminals or other malicious actors, the consequences for individuals can be catastrophic.

To keep Firefox users safe, we ensure that only certificate authorities that maintain high standards of security and transparency are trusted in the browser (i.e., included in our ‘root certificate store’). We also continuously monitor and review the behaviour of certificate authorities that we opt to trust to ensure that we can take prompt action to protect individuals in cases where a trusted certificate authority has been compromised.

Properly maintaining a root certificate store is a significant undertaking, not least because the cybersecurity threat landscape is constantly evolving. We aim to ensure our security standards are always one step ahead, and as part of that effort, we’ve just finalised an important policy update that will increase transparency and security in the certificate authority ecosystem. This update introduces new standards for how audits of certificate authorities should be conducted and by whom; phases out legacy encryption standards that some certificate authorities still deploy today; and requires more transparency from certificate authorities when they revoke certificates. We’ve already begun working with certificate authorities to ensure they can properly transition to the new higher security standards.

The policy update is the product of a several-month process of open dialogue and debate amongst various stakeholders in the website security space. It is a further case-in-point of our belief in the value of transparent, community-based processes across the board for levelling-up the website security ecosystem. For instance, before accepting a certificate authority in Firefox we process lots of data and perform significant due diligence, then publish our findings and hold a public discussion with the community. We also maintain a public security incident reporting process to encourage disclosure and learning from experts in the field.

Ultimately, this update process highlights once again how operating an independent root certificate store allows us to drive the website security ecosystem towards ever-higher standards, and to serve as the first line of defence for when web certificates are misused. It’s a responsibility we take seriously and we see it as critical to enhancing trust on the internet.

It’s also why we’re so concerned about draft laws under consideration in the EU (Article 45 of the ‘eIDAS regulation’) that would forbid us from applying our security standards to certain certificate authorities and block us from taking action if and when those certificate authorities mis-issue certificates. If adopted in its current form by the EU, Article 45 would be a major step back for security on the internet, because of how it would restrict browser security efforts and because of the global precedent it would set. A broad movement of digital rights organisations; consumer groups; and numerous independent cybersecurity experts (here, here, and here) has begun to raise the alarm and to encourage the EU to change course on Article 45. We are working hard to do so too.

We’re proud of our root certificate store and the role it plays in enhancing trust and security online. It’s part of our contribution to the internet – we’ll continue to invest in it with security updates like this one and work with lawmakers on ensuring legal frameworks continue to support this critical work.

 

Thumbnail photo credit:

Creative Commons Attribution-Share Alike 4.0 International license.
Attribution: Santeri Viinamäki

 

The post Enhancing trust and security on the internet – browsers are the first line of defence appeared first on Open Policy & Advocacy.

This Week In Rust — This Week in Rust 445

Hello and welcome to another issue of This Week in Rust! Rust is a programming language empowering everyone to build reliable and efficient software. This is a weekly summary of its progress and community. Want something mentioned? Tweet us at @ThisWeekInRust or send us a pull request. Want to get involved? We love contributions.

This Week in Rust is openly developed on GitHub. If you find any errors in this week's issue, please submit a PR.

Updates from Rust Community

Official
Project/Tooling Updates
Observations/Thoughts
Rust Walkthroughs
Research
Miscellaneous

Crate of the Week

This week's crate is pgfplots, a crate to generate publication-quality figures (with or without LaTeX).

Thanks to Daniel Duque for the self-suggestion!

Please submit your suggestions and votes for next week!

Call for Participation

Always wanted to contribute to open-source projects but didn't know where to start? Every week we highlight some tasks from the Rust community for you to pick and get started!

Some of these tasks may also have mentors available, visit the task page for more information.

If you are a Rust project owner and are looking for contributors, please submit tasks here.

Updates from the Rust Project

361 pull requests were merged in the last week

Rust Compiler Performance Triage

A good week: The regressions were small; some have follow-up PR's in flight to address them; and we saw a big improvement from PR #97345, which adds more fast paths for quickly exiting comparisons between two types (such as BitsImpl<M> and BitsImpl<N> for const integers M and N). This improved compile-times for the bitmaps benchmark by 50-65% in some cases (including the trunk nalgebra, according to independent investigation from nnethercote). That same PR had more modest improvements (1% to 2%) to the compile-times for a number of other crates. Many thanks to lcnr and nnethercote for some excellent work here!

Triage done by @pnkfelix. Revision range: 43d9f385..0a43923a

3 Regressions, 1 Improvements, 9 Mixed; 0 of them in rollups 59 artifact comparisons made in total

Full report here

Call for Testing

An important step for RFC implementation is for people to experiment with the implementation and give feedback, especially before stabilization. The following RFCs would benefit from user testing before moving forward:

  • No RFCs issued a call for testing this week.

If you are a feature implementer and would like your RFC to appear on the above list, add the new call-for-testing label to your RFC along with a comment providing testing instructions and/or guidance on which aspect(s) of the feature need testing.

Approved RFCs

Changes to Rust follow the Rust RFC (request for comments) process. These are the RFCs that were approved for implementation this week:

Final Comment Period

Every week the team announces the 'final comment period' for RFCs and key PRs which are reaching a decision. Express your opinions now.

RFCs
  • No RFCs entered Final Comment Period this week.
Tracking Issues & PRs
New and Updated RFCs

Upcoming Events

Rusty Events between 2022-06-01 - 2022-06-29 🦀

Virtual
North America
Europe
Oceania

If you are running a Rust event please add it to the calendar to get it mentioned here. Please remember to add a link to the event too. Email the Rust Community Team for access.

Rust Jobs

Element

Quickwit

Micropelt

Stockly

Audiotool

Zcash Foundation

Bolt Labs

Tweet us at @ThisWeekInRust to get your job offers listed here!

Quote of the Week

Rust is a perfect language for a dad like me, who every day puts kids to sleep, and tired after long day of work and chores, can sit down and possibly write some code for the hobby open source project, even when he's already just half awake. And it usually just works, tend to be robust and make the day feel extra productive.

– Dawid Ciężarkiewicz on /r/rust

Please submit quotes and vote for next week!

This Week in Rust is edited by: nellshamrell, llogiq, cdmistman, ericseppanen, extrawurst, andrewpollack, U007D, kolharsam, joelmarcey, mariannegoldin.

Email list hosting is sponsored by The Rust Foundation

Discuss on r/rust

The Talospace Project — Firefox 101 on POWER

Firefox 101 is out, mostly of interest to developers but with a few liveability improvements. I suspect breakage of our LTO-PGO patch will be a regular occurrence and Fx101 indeed broke the diff again, but after updating rust, cargo and cbindgen the optimized build works fine with this revised PGO-LTO patch and the same .mozconfigs from Firefox 95.

I can't figure out where the bugs are in our POWER9 Ion JavaScript JIT implementation, though I still strongly suspect it's because we use sign-extended 32-bit ints. This is enough to pass the test suite but still breaks with the web. MIPS does too and we are strongly based on the MIPS port, but one wonders if MIPS suffers from the same issues, and I'm quite sure it's not tested anywhere near as well. As 32-bit arithmetic instructions aren't orthogonal in Power ISA this change would require some significant rearchitecting and I've just come off a really bad week at work, so I'll just try to pull up the current JIT to 102 so that those of you building the new ESR can continue to use Baseline Interpreter and Baseline Compiler, and then it will be time for more major surgery on the source code. You can help (please!).

Mozilla Thunderbird — Thunderbird + RSS: How To Bring Your Favorite Content To The Inbox

The official RSS logo

I first discovered RSS feeds in 2004 when I fell in love with podcasting. That’s when I learned I could utilize RSS to bring my favorite web content to me, on my schedule. Whether it was weekly music podcasts, tech blogs, newspaper articles, or a local weather forecast, RSS became a way to more easily digest and disseminate the growing onslaught of content on the web. Back then, I used Google Reader (RIP). But now I use Thunderbird to manage and read all my news feeds, and I love it!

In this post I’ll explain what RSS is, why it’s useful, and how to get all set up with some feeds inside of Thunderbird.

What Is RSS?

Skip ahead if you’re old-school and already know this. But if you’re hearing about RSS for the first time, here’s a brief description since it’s not exactly a household name.

RSS stands for “Really Simple Syndication.” It’s a web technology that regularly monitors web pages for frequently updated content, and then delivers that content outside of the web page in a universal, computer-readable format. This is done through an auto-generated XML file that feed readers (and software like Thunderbird) transform back into a tidy, human-readable format.

A raw RSS feed is an XML file containing important information about, and enclosures for, web content. <figcaption>Here’s what a raw RSS feed looks like</figcaption>

Adding that resulting URL (for example https://blog.thunderbird.net/rss) to RSS-compatible software like Thunderbird gives you an always-updating, free subscription to that content.

And RSS supports more than text: it’s also built for images, audio and video “enclosures.” (As a Thunderbird user, just think of them as attachments!)

Want to learn more about RSS feeds? Read this great explainer.

Why You’ll Love RSS

Here are a few compelling reasons for using RSS feeds to consume your favorite web content:

  • You don’t have to track down the news. The news comes to you!
  • Stay on top of updates from your favorite sites without needing to subscribe to newsletters or remembering to manually check in. (Especially useful for sites that don’t update regularly.)
  • Organize your favorite content into categories, tags, folders and sub-folders just like your email.
  • Bypass algorithms, intrusive ads and privacy-invading trackers.
  • RSS feeds are free
  • All podcasts (except Spotify exclusives) use RSS feeds. So does YouTube!
  • It’s easy to move your RSS feed subscriptions to other desktop and mobile apps.
  • Shameless plug: You can read this Thunderbird blog in Thunderbird!

Why You’ll Love The Thunderbird + RSS Combo

It all clicked for me when I realized how intuitive it was treating news content just like email. If you already use Thunderbird as your main email client, you know how powerful it is. Put simply, I organize all the web content I subscribe to in Thunderbird the same way I organize my email.

My own Feeds (that’s what we call RSS subscriptions) are divided into subfolders like Music, Tech, Comics, Gaming, and World News.

Just like email, I can favorite/star articles I love or want to revisit later. I can utilize tags and Quick Filters and tabs. I can search them, save them, print them, and delete them. Plus, sharing something is as easy as forwarding an email.

And because of Thunderbird’s flexibility, you’re not just limited to text excerpts. You can read full web pages, meaning consuming the content as originally published. If you’re privacy-conscious, you can absolutely block intrusive trackers and ads with this uBlock Origin extension for Thunderbird.

Need a bonus reason? You can even watch your YouTube subscriptions inside Thunderbird via RSS! (Keep reading to learn how to easily discover a channel’s RSS feed.)

How To Set Up Thunderbird For RSS

Add a new "Feeds" Account<figcaption>First, add a New Feed Account that will contain all your RSS subscriptions</figcaption>

The first step is adding a Feed Account to Thunderbird. This process is similar to adding email accounts, but faster. You won’t need a username or password; this is simply a place to collect all your RSS feeds.

From the File or ≡ Menu, select New > Feed Account. Now, give it a name. Any name you like!

Here's what I named my Feeds account<figcaption>Here’s what I named my Feeds account</figcaption>

Having a unique “account” and name for your RSS feeds is helpful if you decide to export your subscriptions to another version of Thunderbird, or another app altogether.

And that’s it! Now you just need to find and add some feeds. Let’s tackle that next, and then we’ll get into some organizational tips to streamline your experience.

How To Find RSS Feeds

An RSS icon on the Thunderbird blog<figcaption>An RSS icon on the Thunderbird blog</figcaption>

In the image above, see the circled image that looks like a WiFi icon? That’s an RSS logo, and it’s how you’ll know a website offers a feed. Sometimes it’ll be orange (left), and it may be located on the sidebar or bottom footer of a page.

While they may not be advertised, the majority of websites do offer RSS feeds. Sometimes you just have to search a little harder to find them, occasionally needing to view a page’s source code to find any reference to it.

Fortunately, there are useful browser extensions that eliminate all the searching and guesswork. If a page offers an RSS feed, these extensions will auto-magically find the URL for you:

How To Add RSS Feeds To Thunderbird

Once you have that URL, here’s how to add it to Thunderbird.

First, just click on your Feed Account folder (in my case it’s “Jason’s News Hub” as highlighted in the image above). Then click “Manage feed subscriptions.”

It feels appropriate to use the Thunderbird blog as our first feed. Find the RSS icon at the top of this page, and copy the link. (Or just copy this: https://blog.thunderbird.net/feed/.)

Adding a new RSS feed to Thunderbird

Now paste that link inside the “Feed URL” box, and click the “Add” button. If it’s a valid link, you should see the Title populate, and the see it nested under your Feeds account.

Here’s what my personal Feeds account currently looks like. As you can see, I’ve loosely categorized my feed with subfolders:

<figcaption>I have some catching up to do!</figcaption>

If you use Thunderbird, you certainly know what to do next. Just click and enjoy your favorite content delivered to your inbox. Most web content these days is responsive, so articles and videos should adapt when adjusting your Thunderbird layout or window size.

If you want to read the content in its own tab, just double click the title as you would an email. It looks pretty slick, especially with the Dark Reader Add-on enabled to remove the original white background of this blog. Here’s an example using a recent article highlighting 7 great new features coming to Thunderbird 102:

Want More RSS Guides?

Using RSS with Thunderbird has dramatically changed how I consume content. This may sound borderline melodramatic, but it’s removed some anxiety surrounding my habit of constantly scouring the web for information or entertainment. Now, I invest some time upfront to find the sources I love, and then sit back and get all that content delivered to me.

Specifically, it’s rekindled my love of discovering new music. Thunderbird makes this particularly productive for me since I can manage, organize and manipulate all those new music recommendations with tags, filters, search, subfolders and more.

If you want to dive deeper into RSS and read more guides on managing your own RSS feed collection, leave a comment here and let me know! Have questions about using RSS? You can ask them in the comments, or reach out to us on Mastodon, Twitter, or Facebook.

Thanks for reading, and thanks for using Thunderbird!

The post Thunderbird + RSS: How To Bring Your Favorite Content To The Inbox appeared first on The Thunderbird Blog.

Mozilla Reps Community — New (…and less new) Council members – H2 2021 – H1 2022

Hello all,

 

As you know, we all elected three new members to our Reps Council. As usual, we are writing a post about them and the wonderful things they did for Mozilla.

 

However, this time, I have to admit something:

A few months ago, you elected two other new members to the Reps Council. They finished their onboarding and started actively contributing to the program. But….we did not post about them!

 

So in this post, and terribly late, we want to talk about all these five new (and…less new) council members.

 

Let’s get to know them!

 

Yuliana Reyna Jiménez, is a Mozilla Rep from Mexico. She is a programmer. She is part of the Mozilla Hispano community, and she is currently very active in the localization of MDN to Spanish. As part of the Reps Council, she is working to revitalize the Mozilla Social Champions.

 

Ruwan Ranganath Senarathne is a Rep from Sri Lanka. By day, he is a DevOps and Cloud Engineer. As part of Mozilla Sri Lanka, he is very active in establishing new Mozilla groups. As part of the Reps Council, he is also working on a project to create and activate new communities.

 

Robert Reyes is a Mozilla Rep from the Philippines. He is actively involved in leading the Philippines community and recently helped create our first Mozilla Community in Oman!

 

Paul Wright is an active SUMO contributor and MozFest volunteer coordinator from the UK.

 

Siddhartha Rao is a Rep from the US. He is part of the Onboarding Team and one of our newest mentors!

 

Thanks to all the new members for joining the Council, and thank you very much to all the outgoing council members for your contributions!

 

Keep rockin’ the free and open web everyone!

Support.Mozilla.Org — What’s up with SUMO – May

Hi everybody,

Q2 is a busy quarter with so many exciting projects on the line. The onboarding project implementation is ongoing, mobile support project also goes smoothly so far (we even start to scale to support Apple AppStore), but we also managed to audit our localization process (with the help of our amazing contributors!). Let’s dive more into it without further ado.

Welcome note and shout-outs

  • Welcome to the social support program to Magno Reis and Samuel. They both are long-time contributors on the forum who are spreading their wings to Social Support.
  • Welcome to the world of the SUMO forum to Dropa, YongHan, jeyson1099, simhk, and zianshi17.
  • Welcome to the KB world to kaie, alineee, and rodrigo.bucheli.
  • Welcome to the KB localization to Agehrg4 (ru), YongHan (zh-tw), ibrahimakgedik3 (tr), gabriele.siukstaite (t), apokvietyte (lt), Anokhi (ms), erinxwmeow (ms), and dvyarajan7 (ms). Welcome to the SUMO family!
  • Thanks to the localization contributors who helped me understand their workflow and pain points on the localization process. So many insightful feedback and things that we may not understand without your input. I can’t thank everybody enough for your input!
  • Huge shout outs to Kaio Duarte Costa for stepping up as Social Support moderator. He’s been an amazing contributor to the program since 2020, and I believe that he’ll be a great role model for the community. Thank you and congratulations!

If you know anyone that we should feature here, please contact Kiki and we’ll make sure to add them in our next edition.

Community news

  • I highly recommend checking out KB call from May. We talked about many interesting topics, from KB review queue, to a group exercise on writing content for localization.
  • It’s been 2 months since we on board Dayana as a Community Support Advocate (read the intro blog post here) and we can’t wait to share more about our learnings and accomplishment!
  • Please read this forum thread and this bug report for those of you who experiences trouble with uploading images to Mozilla Support.

Catch up

  • Watch the monthly community call if you haven’t. Learn more about what’s new in April! Reminder: Don’t hesitate to join the call in person if you can. We try our best to provide a safe space for everyone to contribute. You’re more than welcome to lurk in the call if you don’t feel comfortable turning on your video or speaking up. If you feel shy to ask questions during the meeting, feel free to add your questions on the contributor forum in advance, or put them in our Matrix channel, so we can answer them during the meeting.
  • There’s also KB call, this one was the recording for the month of May. Find out more about KB call from this wikipage.
  • If you’re an NDA’ed contributor, you can watch the recording of the Customer Experience weekly scrum meeting from AirMozilla to catch up with the latest product updates.
  • Check out SUMO Engineering Board to see what the platform team is currently doing.

Community stats

KB

KB pageviews (*)

* KB pageviews number is a total of KB pageviews for /en-US/ only

Month Page views Vs previous month
Apr 2022 7,407,129 -1.26%

Top 5 KB contributors in the last 90 days: 

KB Localization

Top 10 locales based on total page views

Locale Mar 2022 pageviews (*) Localization progress (per Apr, 11)(**)
de 7.99% 98%
zh-CN 7.42% 100%
fr 6.27% 87%
es 6.07% 30%
pt-BR 4.94% 54%
ru 4.60% 82%
ja 3.80% 48%
It 2.36% 100%
pl 2.06% 87%
ca 1.67% 0%

* Locale pageviews is an overall pageviews from the given locale (KB and other pages)

** Localization progress is the percentage of localized article from all KB articles per locale

Top 5 localization contributors in the last 90 days: 

Forum Support

Forum stats

-TBD-

 

Top 5 forum contributors in the last 90 days: 

Social Support

Channel Total incoming message Conv interacted Resolution rate
Apr 2022 504 316 75.00%

Top 5 Social Support contributors in the past 2 months: 

  1. Bithiah Koshy
  2. Christophe Villeneuve
  3. Magno Reis
  4. Md Monirul Alom
  5. Felipe Koji

Play Store Support

Channel Apr 2022
Total priority review Total priority review replied Total reviews replied
Firefox for Android 1226 234 291
Firefox Focus for Android 109 0 4
Firefox Klar Android 1 0 0

Top 5 Play Store contributors in the past 2 months: 

  • Paul Wright
  • Tim Maks
  • Selim Şumlu
  • Bithiah Koshy

Product updates

Firefox desktop

Firefox mobile

  • TBD

Other products / Experiments

  • TBD

Useful links:

Martin Thompson — Entropy and Privacy Analysis

Aggregation is a powerful tool when it comes to providing privacy for users. But analysis that relies on aggregate statistics for privacy loss hides some of the worst effects of designs.

Background

A lot of my time recently has been spent looking at various proposals for improving online advertising. A lot of this work is centred on the Private Advertising Technology Community Group in the W3C where the goal is to find designs that improve advertising while maintaining strong technical protections for privacy.

Part of deciding whether a design does in fact provide strong privacy protections requires understanding firstly what that means. That is a large topic on which the conversation is continuing. In this post, my goal is to look at some aspects of how we might critically evaluate the privacy characteristics of proposals.

Limitations of Differential Privacy

A number of designs have been proposed in this space with supporting analysis that is based on differential privacy. Providing differential privacy involves adding noise to measurements using a tunable parameter (usually called $\varepsilon$) that hides individual contributions under a random distribution.

I’m a big fan of differential privacy, but while differential privacy provides a good basis for understanding the impact of a proposal, it is recognized that there is a need to continuously release information in order to maintain basic utility in a long-running system.

Continuous release of data potentially leads to the protections offered by differential privacy noise being less effective over time. It is prudent therefore to understand the operation of the system without the protection afforded by noise. This is particularly relevant where the noise uses a large $\varepsilon$ value or is applied to unaggregated outputs, where it can be easier to cancel the effect of noise by looking at multiple output values.

Information exposure is often expressed using information theoretic statistics like entropy. This note explores how entropy — or any single statistic — is a poor basis for privacy analysis and suggests options for more rigorous analysis.

Information Theory and Privacy

Some analysis of Web privacy features often looks at the number of bits of information that a system releases to an adversary. Analyses of this type use the distribution of probabilities of all events as a way of estimating the amount of information that might be provided by a specific event.

In information theory, each event provides information or surprisal, defined by a relationship with the probability of the event:

$$I(x)=-\log_2(P(x))$$

The reason we might use information is that if a feature releases too much information, then people might be individually identified. They might no longer be anonymous. Their activities might be linked to them specifically. The information can be used to form a profile based on their actions or further joined to their identity or identities.

Generally, we consider it a problem when information enables identification of individuals. We might express concern if:

$$2^{I(x)} \ge \text{size of population}$$

Because surprisal is about specific events, it can be a little unwieldy. Surprisal is not useful for reaching a holistic understanding of the system. A statistic that summarizes all potential outcomes is more useful in gaining insight into how the system operates as a whole. A common statistic used in this context is entropy, which provides a mean or expected surprisal across a sampled population:

$$H(X)=\sum_{x\in X}P(x)I(x)=-\sum_{x\in X}P(x)\log_2(P(x))=-\frac{1}{N}\sum_{i=1}^N\log_2(P(x_i))$$

Entropy has a number of applications. For instance, it can be used to determine an optimal encoding of the information from many events, using entropy coding (such as Huffman or Arithmetic coding).

Using Entropy in Privacy Analysis

The use of specific statistics in privacy analysis is useful to the extent that they provide an understanding of the overall shape of the system. However, simple statistics tend to lose information about exceptional circumstances.

Entropy has real trouble with rare events. Low probability events have high surprisal, but as entropy scales their contribution by their probability, they contribute less to the total entropy than higher probability events.

In general, revealing more information is undesirable from a privacy perspective. Toward that end, it might seem obvious that minimizing entropy is desirable. However, this can be shown to be counterproductive for individual privacy, even if a single statistic is improved.

An example might help prime intuition. A cohort of 100 people is arbitrarily allocated into two groups. If people are evenly distributed into groups of 50, revealing the group that a person has been allocated provides just a single bit of information, that is, surprisal is 1 bit. The total entropy of the system is 1 bit.

An asymmetric allocation can produce a different result. If 99 people are allocated to one group and a single person to the other, revealing that someone is in the first group provides almost no information at 0.0145 bits. On the contrary, revealing the allocation for the lone person in the second group — which uniquely identifies that person — produces a much larger surprisal of 6.64 bits. Though this is clearly a privacy problem for that person, their privacy loss is not reflected in the total entropy of the system, which at 0.0808 bits is close to zero.

Entropy tells us that the average information revealed for all users is very small. That conclusion about the aggregate is reflected in the entropy statistic, but it hides the disproportionately large impact on the single user who loses the most.

The more asymmetric the information contributed by individuals, the lower the entropy of the overall system.

Limiting analysis to simple statistics, and entropy in particular, can hide privacy problems. Somewhat counterintuitively, the adverse consequences of a design are felt more by a minority of participants for systems with lower entropy.

This is not a revelatory insight. It is well known that a single metric is often a poor means of understanding data.

Entropy can provides a misleading intuitive understanding of privacy as it relates to the experience of individual users.

Recommendations

Information entropy remains useful as a means of understanding the overall utility of the information that a system provides. Understanding key statistics as part of a design is valuable. However, for entropy measures in particular, this is only useful from a perspective that seeks to reduce overall utility; entropy provides almost no information about the experience of individuals.

Understand the Surprisal Distribution

Examining only the mean surprisal offers very little insight into a system. Statistical analysis rarely considers a mean value in isolation. Most statistical treatment takes the shape of the underlying distribution into account.

For privacy analysis, understanding the distribution of surprisal values is useful. Even just looking at percentiles might offer greater insight into the nature of the privacy loss for those who are most adversely affected.

Shortcomings of entropy are shared by related statistics, like Kullback–Leibler divergence or mutual information, which estimate information gain relative to a known distribution. Considering percentiles and other statistics can improve understanding.

Knowing the distribution of surprisal admits the combination of privacy loss metrics. As privacy is affected by multiple concurrent efforts to change the way people use the Web, the interaction of features can be hard to understand. Richer expressions of the effect of changes might allow for joint analysis to be performed. Though it requires assumptions about the extent to which different surprisal distributions might be correlated, analyses of surprisal that assume either complete independence or perfect correlation could provide insights into the potential extent of privacy loss from combining features.

For example, it might be useful to consider the interaction of a proposal with extant browser fingerprinting. The users who reveal the most information using the proposal might not be the same users who reveal the most fingerprinting information. Analysis could show that there are no problems or it might help guide further research that would provide solutions.

More relevant to privacy might be understanding the proportion of individuals that are potentially identifiable using a system. A common privacy goal is to maintain a minimum size of anonymity set. It might be possible to apply knowledge of a surprisal distribution to estimating the size of a population where the anonymity set becomes too small for some users. This information might then guide the creation of safeguards.

Consider the Worst Case

A worst-case analysis is worth considering from the perspective of understanding how the system treats the privacy of all those who might be affected. That is, consider the implications for users on the tail of any distribution. Small user populations will effectively guarantee that any result is drawn from the tail of a larger distribution.

Concentrating on cases where information might be attributed to individuals might miss privacy problems that might arise from people being identifiable in small groups. Understand how likely smaller groups might be affected.

The potential for targeting of individuals or small groups might justify disqualification of — or at least adjustments to — a proposal. The Web is for everyone, not just most people.

Mozilla Open Policy & Advocacy Blog — Mozilla Meetups: The Building Blocks of a Trusted Internet

Join us on June 9 at 3 PM ET for a virtual conversation on how the digital policy landscape not only shapes, but is also shaped by, the way we move around online and what steps our policymakers need to take to ensure a healthy internet.

The post Mozilla Meetups: The Building Blocks of a Trusted Internet appeared first on Open Policy & Advocacy.

Data@Mozilla — Crash Reporting Data Sprint

Two weeks ago the Socorro Eng/Ops team, in charge of Socorro and Tecken, had its first remote 1-day Data Sprint to onboard folks from ops and engineering.

The Sprint was divided into three main parts, according to the objectives we initially had:

  • Onboard new folks in the team
  • Establish a rough roadmap for the next 3-6 months.
  • Find a more efficient way to work together,

The sprint was formatted as a conversation followed by a presentation guided by Will Kahn-Greene, who leads the efforts in maintaining and evolving the Socorro/Tecken platforms. In the end we went through the roadmap document to decide what our immediate future would look like.

 

Finding a more efficient way to work together

We wanted to track our work queue more efficiently and decided that a GitHub project would be a great candidate for the task. It is simple to set up and maintain and has different views that we can lightly customize.

That said, because our main issue tracker is Bugzilla, one slightly annoying thing we still have to do while creating an issue on our GitHub project is place the full url to the bug in the issue title:

If we could place links as part of the title, then we could do something like:

Which is much nicer, but GitHub doesn’t support that.

Here’s the link to our work queue: https://github.com/orgs/mozilla-services/projects/16/views/1

 

Onboarding new people to Socorro/Tecken

This was a really interesting part of the day in which we went through different aspects of the crash reporting ecosystem and crash ingestion pipeline.

Story time

The story of Mozilla’s whole Crash Reporting system dates back to 2007, when the Socorro project was created. Since then, Crash Reporting has been an essential part of our products. It is present in all stages, from development to release, and is comprised of an entire ecosystem of libraries and systems maintained by different teams across Mozilla.

Socorro is one of the longer-running projects we have at Mozilla. Along with Antenna and Crash Stats it comprises the Crash Ingestion pipeline, which is maintained by the socorro-eng team. The team is also in charge of the Symbol and Symbolication Servers a.k.a. Tecken.

Along with that story we also learned interesting facts about Crash Reporting, such as:

    • Crash Reports are not the same as Crash Pings: Both things are emitted by the Crash Reporter when Firefox crashes, but Reports go to Socorro and Pings go to the telemetry pipeline
    • Not all Crash Reports are accepted: The collector throttles crash reports according to a set of rules that can be found here
    • Crash Reports are pretty big compared to telemetry pings: They’re 600Kb aggregate but stack overflow crashes can be bigger than 25MB
    • Crash Reports are reprocessed regularly: Whenever something that is involved in generating crash signatures or crash stacks is changed or fixed we reprocess the Crash Reports to regenerate their signatures and stacks

What’s what

There are lots of names involved in the Crash Reporting. We went over what most of them mean:

Symbols: A symbol is an entry in a .sym file that maps from a memory location (a byte number) to something that’s going on in the original code. Since binaries don’t contain information about the code such as code lines, function names and stack navigation, symbols are used to enrich minidumps emitted by binaries with such info. This process is called symbolication. More on symbol files: https://chromium.googlesource.com/breakpad/breakpad/+/HEAD/docs/symbol_files.md

Crash Report: When an application process crashes, the Crash Reporter will submit a Crash Report with metadata annotations (BuildId, ProductName, Version, etc) and minidumps which contain info on the crashed processes.

Crash Signature: Generated to every Crash Report by an algorithm unique to Socorro with the objective of grouping similar crashes.

Minidump: A file created and managed by the Breakpad library. It holds info on a crashed process such as CPU, register contents, heap, loaded modules, threads etc.

Breakpad: A set of tools to work with minidump files. It defines the sym file format and includes components to extract information from processes as well as package, submit  and process them. More on Breakpad: https://chromium.googlesource.com/breakpad/breakpad/+/master/docs/getting_started_with_breakpad.md#the-minidump-file-format

A peek at how it works

Will also explained how things work under the hood and we had a look on the diagrams that show what comprises Tecken and Socorro:

 

Tecken/Eliot

Tecken architecture diagram

Tecken (https://symbols.mozilla.org/) is a Django web application that uses S3 for storage and RDS for bookkeeping.

Eliot (https://symbolication.services.mozilla.com/) is a webapp that downloads sym files from Tecken for symbolication.

 

Socorro

Socorro Architecture Diagram

Socorro has a Crash Report Collector, a Processor and a web application (https://crash-stats.mozilla.org/) for searching and analyzing crash data. Notice the Crash Ingestion pipeline processes Crash Reports and exports a safe form of the processed crash to Telemetry.

More on Crash Reports data

The Crash Reporter is an interesting piece of an application since it needs to do its work while the world around it is collapsing. That means a number of unusual things can happen to the data it collects to build a Report. That being said, there’s a good chance the data it collects is ok, and even when it isn’t, it can still be interesting.

A real concern toward Crash Report data is how toxic it can get: While some pieces of the data are things like the ProductName, BuildID, Version and so on, other pieces are highly identifiable such as URL, UserComments and Breadcrumbs.

Add that to the fact that minidumps contain copies of memory from the crashed processes, which can store usernames, passwords, credit card numbers and so on, and you end up with a very toxic dataset!

 

Establishing a roadmap for the next 3-6 months

Another interesting exercise that made that Sprint feel even more productive was going over the Tecken/Socorro roadmap and reprioritizing things. While Will was explaining the reasons why we should do certain things, I took that chance to also ask questions and get better context on the different decisions we made, our struggles of past and present and where we aim to be.

 

Conclusion

It was super productive to have a full day on which we could focus completely on all things Socorro/Tecken. That series of activities allowed us to improve the way we work, transfer knowledge and prioritize things for the not-so-distant future.

Big shout out to Will Kahn-Greene for organizing and driving this event, and also for the patience to explain things carefully and precisely.

Firefox Nightly — These Weeks in Firefox: Issue 116

Highlights

  • ESMification is a project to move away from our custom JS module system (JSMs) to the standard ES Module system most are familiar with when writing JavaScript.

  • A prototype of loading ESM modules has landed in bug 1432901. We are looking for feedback. Please get in touch with yulia (@yulia on matrix, yulia@mozilla.com) if you would like to discuss any of the above.

  • Picture-in-Picture subtitles support has been added for SonyLiv and Dailymotion

  • Picture-in-Picture now supports forward/backward seeking with the left/right arrow keys on Netflix

  • The address bar used to not convert negative temperatures. Now it does! If you type in -30C to F, it converts it to -22 F. This feature is only available with browser.urlbar.unitConversion.enabled set to true.

Resolved bugs (excluding employees)

Script to find new contributors from bug list

Volunteers that fixed more than one bug

  • Itiel

  • manisha.singh2019

  • serge-sans-paille

  • Shane Hughes [:aminomancer]

New contributors (🌟 = first patch)

General triage

Project Updates

Add-ons / Web Extensions
Addon Manager & about:addons

  • Thanks to Eemeli some remaining localized strings used in about:addons have been migrated to Fluent  – Bug 1507471

WebExtensions Framework and APIs

  • As part of the deprecation of the “network.cookie.lifetimePolicy” Firefox about:config preference (Bug 1681493), the privacy.websites.cookieConfig WebExtension API JSONSchema has been updated to mark the “nonPersistentCookies” option as deprecated – Bug 1754924

  • As part of the ongoing ManifestVersion 3 work:

    • Fixed missing “action” WebExtensions API namespace on Android builds – Bug 1768839

    • Introduced new telemetry probes related to the extension startupCache usage – Bug 1767336

    • Enforced host permissions checks on content scripts registered using the scripting APIs – Bug 1760451

    • Enabled “scripting” WebExtensions API for ManifestVersion 2 extensions

      • Part of changes introduced for ManifestVersion 2 extensions to help extension developers with transitioning their addons to ManifestVersion 3.

    • Reset the event pages idle timeout when the event page does still have “webRequest.filterReponseData” filters attached to intercepted web requests – Bug 1759300

Developer Tools

 

  • Added filter for list of stylesheets in Browser Toolbox (bug) (Style Editor panel). The user can also use Ctrl/Cmd + p shortcut to focus the input field (bug)

Image of the style sheet filter in the Browser Toolbox Style Editor

Considering a change of style? The new style sheet filter can help you find what you’re looking for.

 

  • Show frames under their related content process ID (bug)

Image of frames being labeled according to their content process IDs in a dropdown menu within the Browser Toolbox

Reframe your debugging routine by viewing relevant content process IDs (pids) for individual frame targets

  • Don’t add expressions evaluated in the Console panel (command line) to history if the browser is running in private mode (bug)

 

WebDriver BiDi M3 Completed
Form Autofill
Lint, Docs and Workflow
New Tab Page
Password Manager
Picture-in-Picture

 

Search and Navigation

  • [daisuke] fixed Bug 1457149 on Win64 where Ctrl-L Delete / CTRL-L Backspace does not  clear the address bar when done extremely fast.

  • James [:jteow] fixed a  Bug 1560676  where autofill was broken for URLs such as localhost:8888/foo.

  • [daisuke] and Drew [:adw] have been working on autofill so that it fills urls not only origins. Bug 1597791

  • [daisuke] fixed Bug 1633924 where adaptive results were case sensitive. This was changed so it’s case insensitive. If you typed FIRE or fire the url https://www.reddit.com/r/firefox/ will be shown in results

  • Mandy [:mcheang] Fixed Bug 1673868 where DDG Ads on Search Result Pages was reported inconsistently.

  • Mandy [:mcheang] Did some work for Bug 1759207 to show an infobar for when an application default search engine is removed from configuration.

  • [jamie] Fixed a Bug 1761154  for accessibility screen readers. The reader read the first result twice.

  • [daisuke] fixed Bug 1761154  where the address bar does not convert negative temperatures. Now it does. If you type in -30C to F, it converts it to -22 F. This feature is only available with browser.urlbar.unitConversion.enabled set to true.

  • Drew [:adw] on Bug 1765595 added a Nimbus variable to load Fx Suggest suggestions with a different type on Remote settings.

  • Mark [:standard8] removed osfile.jsm usage from search because of new linting rules for Bug 1769047 .

Screenshots

  • [niklas] I landed the initial patch for the draggable overlay. Set `screenshots.browser.component.enabled` to true to test out the new screenshots component!

 

ESMification status

  • ESMification is a project to move away from our custom JS module system (JSMs) to the standard ES Module system most are familiar with when writing JavaScript.

  • A prototype of loading ESM modules has landed in bug 1432901, it is now possible to load modules synchronously for system code (in progress example for a content/system component).

  • The team is working on a lazy module API similar to defineLazyModuleGetters & co.

  • There are a couple of questions around the API and naming that are currently being explored.

  • We are looking at a migration in H2.

  • We are looking for feedback. Please get in touch with yulia (@yulia on matrix, yulia@mozilla.com) if you would like to discuss any of the above.

Tiger Oakes — How to use Segoe UI Variable in a website

Loading this font locally is a bit tricky.

Mozilla Thunderbird — Thunderbird By The Numbers: Our 2021 Financial Report

Transparency and open source go hand-in-hand. But just because Thunderbird’s development work, roadmap, and financials are public, doesn’t always mean they’re well publicized.

That’s where my role as Marketing Manager comes into focus. To shine a spotlight on the numbers, the features, the facts, and the future. I want to keep you informed without you needing to hunt down every scrap of information!

With that in mind, let’s talk about money. Specifically, Thunderbird’s income for 2021, and how it positively affects our team, our product, and our roadmap.

Thunderbird Income in 2021

You may have heard rumors of Thunderbird’s demise, or assumed that the project’s financial outlook was bleak. Thankfully, that’s not remotely close to reality.

In fact, 2021 was a very successful year for Thunderbird. Our income, which is sourced almost entirely by user donations, totaled $2,796,996. That represents a 21% increase over donations in 2020, and it’s more than twice the amount we received in 2018.

Thunderbird year-to-year donations: 2017 through 2021<figcaption>Thunderbird year-to-year donations: 2017 through 2021</figcaption>

Do we have other sources of income? Yes, but that non-donation income is negligible, and represents less than a fraction of a fraction of one percent. It comes from our partnerships with Gandi and Mailfence for users to get new email addresses. That said, we are exploring other potential revenue opportunities beyond user donations, but only those that align with our mission and values. (When we make concrete decisions regarding those opportunities, you’ll read about them here).

Thunderbird Spending in 2021

In total we spent $1,984,510 last year. The majority of that (78.1%) was allocated to Thunderbird personnel. These are the talented full-time employees and contractors paid to work on Thunderbird. And with an increase in generous donations, we’ve been able to grow our core staff.

We currently employ a talented team of 20 people in the following roles:

  • Technical Manager
  • Product and Business Development Manager
  • Community Manager
  • QA Engineer
  • Add-ons Coordinator
  • Lead UX Architect
  • Security Engineer
  • Senior Developers x3
  • Developers x4
  • Infra Team Lead
  • Build Engineer
  • Thunderbird Release Engineer / Web Infra Engineer
  • Director of Operations
  • Designer
  • Marketing Manager

But we’re not done expanding the team! For those interested, we are still hiring!

A pie chart showing Thunderbird spending in 2021. <figcaption>Total Thunderbird spending in 2021</figcaption>

The pie chart above breaks down the rest of Thunderbird’s spending in 2021.

“Professional Services” include things like HR (Human Resources), tax services, and agreements with other Mozilla entities (for example, access to build infrastructure). The remaining items help us to run the business, such as various services and technology that help us communicate and manage operations.

2022 and 2023: Not Surviving, THRIVING!

7 Great New Features Coming To Thunderbird 102

As 2021 came to a close, we had a total of $3,616,032 in the bank. This means we can afford to pursue a variety of bold initiatives that will radically improve Thunderbird. We don’t want to just meet your expectations of what a modern, best-in-class communication tool can be. We want to exceed them.

And you’ve graciously given us those resources!

Moving forward, you’ll see fantastic new features and quality-of-life improvements in Thunderbird 102 this June.

Also happening in June: we’ll also be unveiling our plans to bring Thunderbird to Android, providing a much-needed open source alternative for mobile users. (June is going to be awesome!)

And in 2023, you can look forward to a modernized Thunderbird experience with a completely overhauled UX and UI.

One more thing before we sign off: having cash reserves doesn’t make us complacent. We are careful stewards of the donations we receive from you. We don’t just use it to enhance features; we invest it strategically and wisely towards ensuring long-term stability and viability of Thunderbird. That’s what you deserve!

Your ongoing contributions not only enable Thunderbird to survive, but to thrive. And we can’t thank you enough for that.

Thunderbird is the leading open-source, cross-platform email and calendaring client, free for business and personal use. We want it to stay secure and become even better. A donation will allow us to hire developers, pay for infrastructure, expand our userbase, and continue to improve.

Click here to make a donation. 

The post Thunderbird By The Numbers: Our 2021 Financial Report appeared first on The Thunderbird Blog.

Wladimir Palant — Hijacking webcams with Screencastify

Everyone has received the mails trying to extort money by claiming to have hacked a person’s webcam and recorded a video of them watching porn. These are a bluff of course, but the popular Screencastify browser extension actually provides all the infrastructure necessary for someone to pull this off. A website that a user visited could trick the extension into starting a webcam recording among other things, without any indications other than the webcam’s LED lighting up if present. The website could then steal the video from the user’s Google Drive account that it was uploaded to, along with anything else that account might hold.

Screencastify is a browser extension that aids you in creating a video recording of your entire screen or a single window, optionally along with your webcam stream where you explain what you are doing right now. Chrome Web Store shows “10,000,000+ users” for it which is the highest number it will display – same is shown for extensions with more than 100 million users. The extension is being marketed for educational purposes and gained significant traction in the current pandemic.

As of now, it appears that Screencastify only managed to address the Cross-site Scripting vulnerability which gave arbitrary websites access to the extension’s functionality, as opposed to “merely” Screencastify themselves and a dozen other vendors they work with. As this certainly won’t be their last Cross-site Scripting vulnerability, I sincerely recommend staying clear of this browser extension.

Update (2022-05-25): Version 2.70.0.4517 of the extension released today restricted the attack surface considerably. Now it’s only five subdomains, run by Screencastify and one other vendor. Update (2022-05-31): Screencastify reduced the attack surface further by no longer granting websites access to the user’s Google Drive. And as of version 2.70.1.4520 all five subdomains with privileged access to the extension are run by Screencastify themselves. As far as I can tell, with these changes the issue can no longer be exploited.

Website integration

Screencastify extension integrates with its website. That’s necessary for video editing functionality which isn’t part of the extension. It’s also being used for challenges: someone creates a challenge, and other people visit the corresponding link, solving it and submitting their video. To enable this integration the extension’s manifest contains the following:

"externally_connectable": {
  "matches": [
    "https://*.screencastify.com/*"
  ]
},

So any page on the screencastify.com domain can send messages to the extension. What kind of messages? Lots of them actually. For example:

case 'bg:getSignInToken':
  // sends token for success and false for failure
  asyncAction = this._castifyAuth.getAuthToken({ interactive: true });
  break;

Let’s see what happens if we open Developer Tools somewhere on screencastify.com and run the following:

chrome.runtime.sendMessage("mmeijimgabbpbgpdklnllpncmdofkcpn", {
  type: "bg:getSignInToken"
}, response => console.log(response));

This indeed produces an authentication token, one that starts with ya29.. And a quick search shows that this is the format of Google’s OAuth access token. It indeed belongs to the account that the Screencastify extension has been configured with (extension isn’t usable without), granting full read/write privileges for Google Drive.

Side note: There is a reason why Screencastify has to request full access to Google Drive, and it is a design flaw on Google’s end. An app can request access to an app-specific folder which would be sufficient here. But unlike Dropbox for example, Google Drive hides app-specific folders from the user. And users not being able to see their own uploaded videos is quite a showstopper. So Screencastify has to request full access, create a user-visible folder and upload videos there.

Another piece of functionality allows websites to establish a connection to the extension. Once it has the connection port, the website can send e.g. the following message through it:

port.postMessage({type: "start", data: {captureSource: "desktop"}});

This will bring up the following dialog window:

Chrome window titled: Choose what you want to share. Text inside the window says: Screencastify - Screen Video Recorder wants to share the contents of your screen. User can choose between sharing the entire screen and a specific window.

That’s Chrome’s desktopCapture API in action. If this dialog is accepted, recording will start. So the user needs to cooperate.

There are other possible values for captureSource, for example "tab". It will attempt to use tabCapture API and fails because this API can only be used in response to a direct user action (no idea why Chrome is stricter here than with screen recordings).

And there is "webcam". This uses the same WebRTC API that is available to web pages as well. The difference when using it from a browser extension: the user only needs to grant the permission to access the webcam once. And Screencastify requests that permission as part of its onboarding. So starting a webcam recording session usually happens without requiring any user interaction and without any visual indicators. Later the recording can be stopped:

port.postMessage({type: "stop", data: false});

And with the default settings, the recorded video will be automatically uploaded to Google Drive. As we remember, the website can get the access token necessary to download it from there.

Who controls screencastify.com?

Let’s sum up: the extension grants screencastify.com enough privileges to record a video via user’s webcam and get the result. No user interaction is required, and there are only minimal visual indicators of what’s going on. It’s even possible to cover your tracks: remove the video from Google Drive and use another message to close the extension tab opened after the recording. That’s considerable power. Who could potentially abuse it?

Obviously, there is Screencastify itself, running app.screencastify.com and a bunch of related subdomains. So the company itself or a rogue employee could plant malicious code here. But that’s not the end of it. The entities controlling subdomains of screencastify.com (not counting hosting companies) are at the very least:

  • Webflow, running www.screencastify.com
  • Teachable, running course.screencastify.com
  • Atlassian, running status.screencastify.com
  • Netlify, running quote.screencastify.com
  • Marketo, running go.screencastify.com
  • ZenDesk, running learn.screencastify.com

That’s quite a few companies users are expected to trust.

And who else?

So it’s a domain running quite a few web applications from different companies, all with very different approaches to security. One thing they have in common however: none of the subdomains seem to have meaningful protection in terms of Content Security Policy. With such a large attack surface, exploitable Cross-site Scripting (XSS) vulnerabilities are to be expected. And these would give anyone the power to attack Screencastify users.

In order to make a point I decided to locate an XSS vulnerability myself. The Screencastify web app is written with the Angular framework which is not up my alley. Angular isn’t usually prone to XSS vulnerabilities, and the Screencastify app doesn’t do anything out of the ordinary. Still, I managed to locate one issue.

The problem was located in the error page displayed if you already submitted a video to a challenge and were trying to submit another one. This error page is located under a fixed address, so it can be opened directly rather than triggering the error condition:

Screencastify message with the title: 'Whoops! You already turned in a video.' The text says: 'It looks like you already turned in a video to this assignment on Classroom and cannot submit another video.' Below is a button titled: 'View on Classroom'

The interesting part here is the “View on Classroom” button meant to send you to Google Classroom. What does clicking this button do?

window.open(this.courseworkLink);

Where does this.courseworkLink come from? It’s a query string parameter. Is there some link validation in between? Nope. So, if the query string parameter is something like javascript:alert(document.domain), will clicking this button run JavaScript code in the context of the screencastify.com domain? It sure will!

The attackers would still need to trick the user into actually clicking this button of course. But without any framing protection, this page is susceptible to Clickjacking attacks. So my proof-of-concept page loaded the vulnerable page in an invisible frame and positioned it under the mouse cursor in such a way that any click would go to the “View on Classroom” button. Once the user clicked the page could message Screencastify, retrieve the Google access token from it and ask Google for user’s identity. Reading out Google Drive contents or starting a recording session would have been possible as well.

The (lack of a) fix

I reported the issues to Screencastify on February 14, 2022. I received a response on the very same day, and a day later came the confirmation about the XSS vulnerability being resolved. The mail also mentioned that adding a Content Security Policy would be their long-term plan, but as of now this hasn’t happened on either app.screencastify.com or www.screencastify.com beyond framing protection.

Looking at the most current Screencastify 2.69.0.4425 release, the externally_connectable manifest entry still allows all of screencastify.com to connect to the extension. In fact, there is now even a second entry: with app.pendo.io one more vendor was added to the mix of those who could already potentially exploit this extension.

Was the API accessible this way restricted at least? Doesn’t look like it. This API will still readily produce the Google OAuth token that can be used to access your Google Drive. And the onConnectExternal handler allowing websites to start video recording is still there as well. Not much appears to have changed here and I could verify that it is still possible to start a webcam recording without any visual clues.

So, the question whether to keep using Screencastify at this point boils down to whether you trust Screencastify, Pendo, Webflow, Teachable, Atlassian, Netlify, Marketo and ZenDesk with access to your webcam and your Google Drive data. And whether you trust all of these parties to keep their web properties free of XSS vulnerabilities. If not, you should uninstall Screencastify ASAP.

Update (2022-05-25): Screencastify released version 2.70.0.4517 of their extension today. This release finally restricts the externally_connectable manifest entry, now it says:

"externally_connectable": {
  "matches": [
    "https://captions.screencastify.com/*",
    "https://edit.screencastify.com/*",
    "https://questions.screencastify.com/*",
    "https://watch.screencastify.com/*",
    "https://www.screencastify.com/*"
  ]
},

While this still includes www.screencastify.com which is run by Webflow, this is much better than before and limits the attack surface considerably.

Update (2022-05-31): A further look into version 2.70.0.4517 shows that bg:getSignInToken endpoint has been removed, this further defuses the issue here. Also, version 2.70.1.4520 released on May 27 changes the list of the sites allowed to connect once again, replacing www.screencastify.com by app.screencastify.com. So now all domains granted this access are indeed run by Screencastify and not by third parties.

Mozilla Security Blog — Upgrading Mozilla’s Root Store Policy to Version 2.8

In accordance with the Mozilla Manifesto, which emphasizes the open development of policy that protects users’ privacy and security, we have worked with the Mozilla community over the past several months to improve the Mozilla Root Store Policy (MRSP) so that we can now announce version 2.8, effective June 1, 2022. These policy changes aim to improve the transparency of Certificate Authority (CA) operations and the certificates that they issue. A detailed comparison of the policy changes may be found here, and the significant policy changes that appear in this version are:

  • MRSP section 2.4: any matter documented in an audit as a qualification, a modified opinion, or a major non-conformity is also considered an incident and must have a corresponding Incident Report filed in Mozilla’s Bugzilla system;
  • MRSP section 3.2: ETSI auditors must be members of the Accredited Conformity Assessment Bodies’ Council and WebTrust auditors must be enrolled by CPA Canada as WebTrust practitioners;
  • MRSP section 3.3: CAs must maintain links to older versions of their Certificate Policies and Certification Practice Statements until the entire root CA certificate hierarchy operated in accordance with such documents is no longer trusted by the Mozilla root store;
  • MRSP section 4.1: before October 1, 2022, intermediate CA certificates capable of issuing TLS certificates are required to provide the Common CA Database (CCADB) with either the CRL Distribution Point for the full CRL issued by the CA certificate or a JSON array of partitioned CRLs that are equivalent to the full CRL for certificates issued by the CA certificate;
  • MRSP section 5.1.3: as of July 1, 2022, CAs cannot use the SHA-1 algorithm to issue S/MIME certificates, and effective July 1, 2023, CAs cannot use SHA-1 to sign any CRLs, OCSP responses, OCSP responder certificates, or CA certificates;
  • MRSP section 5.3.2: CA certificates capable of issuing working server or email certificates must be reported in the CCADB by July 1, 2022, even if they are technically constrained;
  • MRSP section 5.4: while logging of Certificate Transparency precertificates is not required by Mozilla, it is considered by Mozilla as a binding intent to issue a certificate, and thus, the misissuance of a precertificate is equivalent to the misissuance of a certificate, and CAs must be able to revoke precertificates, even if corresponding final certificates do not actually exist;
  • MRSP section 6.1.1:  specific RFC 5280 Revocation Reason Codes must be used under certain circumstances (see blog post Revocation Reason Codes for TLS Server Certificates)
  • MRSP section 8.4: new unconstrained third-party CAs must be approved through Mozilla’s review process that involves a public discussion.

These changes will provide Mozilla with more complete information about CA practices and certificate status. Several of these changes will require that CAs revise their practices, so we have also sent CAs a CA Communication and Survey to alert them about these changes and to inquire about their ability to comply with the new requirements by the effective dates.

In summary, these updates to the MRSP will improve the quality of information about CA operations and the certificates that they issue, which will increase security in the ecosystem by further enabling Firefox to keep your information private and secure.

The post Upgrading Mozilla’s Root Store Policy to Version 2.8 appeared first on Mozilla Security Blog.

Firefox Nightly — These Weeks in Firefox: Issue 115

Highlights

  • Firefox 100 goes out today! A milestone release, but also business as usual!
    • Congratulations to contributors who helped get Firefox 100 out the door!
  • Initial support for subtitles and captions in the Picture-in-Picture video player goes out today in Firefox 100. Woohoo!
  • Starting from Firefox >= 101, when a WebExtension calls the browser.downloads.download API method, the download panel will be automatically opened if and only if the extension was handling user input when that API method has been called – Bug 1761828 / Bug 1762033 / Bug 1759231
  • This fixes the “Firefox :: Downloads Panel” issue reported from more than a few users due to the downloads panel opening and quickly closing automatically when one of the extensions installed uses the downloads API to periodically backup the extension settings (see Bug 1759231 and its duplicates)
    • Thanks to aminomancer for the work on the “Firefox :: Downloads Panel” part of this fix!
  • Raphaël improved the DevTools Inspector class autocomplete so that the selected class in the autocomplete is applied as you browse the autocomplete list. Super handy on website using CSS utility rules systems like Bootstrap (bug)

    • An animated GIF showing how the autocomplete field for adding a new class to a node in the Developer Tools Inspector will automatically apply that class as the autocomplete list is browsed.

      Autocomplete? Autoclass!

  • We’ve implemented enough Webdriver BiDi commands to begin the process of shipping it! (bug, intent-to-ship) Congrats to Henrik, James, Julian and Sasha 🎉

Friends of the Firefox team

Resolved bugs (excluding employees)

Script to find new contributors from bug list

Volunteers that fixed more than one bug

  • F3n67u
  • kernp25
  • manisha.singh2019
  • Oriol Brufau [:Oriol]
  • Shane Hughes [:aminomancer]

New contributors (🌟 = first patch)

Project Updates

Add-ons / Web Extensions

Addon Manager & about:addons
  • Fixed a small visual regression related to the about:addons add-on card, due to the sitepermission row not hidden for non-sitepermission addon types  – Bug 1766167
WebExtensions Framework
  • As part of the ongoing ManifestVersion 3 work:
    • Manifest V3 “host_permissions” to be visible and user-controllable from “about:addons” add-on cards – Bug 1745820
    • Deprecated cross-origin requests from ManifestVersion 3 extension content scripts – Bug 1578405
    • Added support for the new “injectImmediately” option to browser.scripting.executeScript – Bug 1762366
WebExtension APIs

Developer Tools

Toolbox
  • F3n67u simplified a few files which were violating eslint complexity rule (bug, bug, bug)
  • Sebo made progress on custom formatters support in DevTools by handling JsonML data in DevTools (bug)
  • Arai fixed console.table when it is called with unexpected arguments (bug)
  • Alex fixed setting breakpoints in HTML inline scripts in iframes (bug)
  • mach run takes a –jsdebugger argument that will automatically open the Browser Toolbox. The arg can be empty, or you can pass a path to a specific binary (e.g. ./mach run –jsdebugger /Applications/FirefoxDeveloperEdition.app/Contents/MacOS/firefox) that will be used for the Browser Toolbox, which can be handy if you’re trying to debug a broken local build. If you reload the main browser window, the Browser Toolbox will reload but keep using the binary you set in the first place. Thanks Alex! (bug)
  • Julian fixed a pretty nasty inspector bug impacting pages embedding Shopify chat UI (bug)
  • A few releases ago we added the ability to tweak any dimension in the inspector rule view by dragging the mouse over the value. Some people are not fan of this feature so Julian added an item in the Settings panel to disable it (bug)
WebDriver BiDi
  • Henrik made navigator.webdriver to be  true when Remote Agent is enabled (bug)
  • Julian implemented the browsingContext.create command, which enable opening tabs (bug)

Fluent

  • Thanks to Niklas who ported videocontrols.js to Fluent! This pulls out more DTDs (which is good), but should also help prevent Yellow Screen of Death frames from appearing on video elements.

Form Autofill

Lint, Docs and Workflow

Password Manager

Picture-in-Picture

Performance Tools (aka Firefox Profiler)

  • Improve Firefox Profiler user docs (#3998). Take a look at https://profiler.firefox.com/docs/ for our user guide. More improvements are on the way as well.
  • Add a “Copy page URL” option to the right click context menu for markers (#3981)
    • In the Marker Chart for a profile in the Firefox Profiler tool, a next context menu is available for markers associated with a page URL that allows for copying that URL to the clipboard.
  • Add “Hide matching tracks” option to the track menu (#3955) It appears after searching something in the filter input and allows you to hide multiple tracks quickly.

    • A new "Hide all matching tracks" item in the track filter panel in the Firefox Profiler allows for quickly hiding all tracks matching the query.

      This makes it easier to drill down to what really matters in a profile.

  • Draw all interval markers as rectangles in the marker chart (#3996)

    • An animated GIF of the Firefox Profiler Marker Chart showing that interval markers are now rendered as rectangles rather than ellipses. This makes their start and end times more visually precise.

      Now all markers that represent a duration are more accurate in the graph!

Search and Navigation

  • Mark has updated the expiring version for the browser.searchinit.engine_invalid_webextension telemetry, that measures disabled or potentially invalid WebExtension search engines – Bug 1763528
  • Drew has improved Firefox Suggest telemetry to measure dismissed or blocked Firefox Suggest entries – Bug 1764669 – and properly report reset impression counts – Bug 1765881

Storybook / Reusable components

The Rust Programming Language Blog — Announcing Rust 1.61.0

The Rust team is happy to announce a new version of Rust, 1.61.0. Rust is a programming language empowering everyone to build reliable and efficient software.

If you have a previous version of Rust installed via rustup, you can get 1.61.0 with:

rustup update stable

If you don't have it already, you can get rustup from the appropriate page on our website, and check out the detailed release notes for 1.61.0 on GitHub.

If you'd like to help us out by testing future releases, you might consider updating locally to use the beta channel (rustup default beta) or the nightly channel (rustup default nightly). Please report any bugs you might come across!

What's in 1.61.0 stable

Custom exit codes from main

In the beginning, Rust main functions could only return the unit type () (either implicitly or explicitly), always indicating success in the exit status, and if you wanted otherwise you had to call process::exit(code). Since Rust 1.26, main has been allowed to return a Result, where Ok translated to a C EXIT_SUCCESS and Err to EXIT_FAILURE (also debug-printing the error). Under the hood, these alternate return types were unified by an unstable Termination trait.

In this release, that Termination trait is finally stable, along with a more general ExitCode type that wraps platform-specific return types. That has SUCCESS and FAILURE constants, and also implements From<u8> for more arbitrary values. The Termination trait can also be implemented for your own types, allowing you to customize any kind of reporting before converting to an ExitCode.

For example, here's a type-safe way to write exit codes for a git bisect run script:

use std::process::{ExitCode, Termination};

#[repr(u8)]
pub enum GitBisectResult {
    Good = 0,
    Bad = 1,
    Skip = 125,
    Abort = 255,
}

impl Termination for GitBisectResult {
    fn report(self) -> ExitCode {
        // Maybe print a message here
        ExitCode::from(self as u8)
    }
}

fn main() -> GitBisectResult {
    std::panic::catch_unwind(|| {
        todo!("test the commit")
    }).unwrap_or(GitBisectResult::Abort)
}

More capabilities for const fn

Several incremental features have been stabilized in this release to enable more functionality in const functions:

  • Basic handling of fn pointers: You can now create, pass, and cast function pointers in a const fn. For example, this could be useful to build compile-time function tables for an interpreter. However, it is still not permitted to call fn pointers.

  • Trait bounds: You can now write trait bounds on generic parameters to const fn, such as T: Copy, where previously only Sized was allowed.

  • dyn Trait types: Similarly, const fn can now deal with trait objects, dyn Trait.

  • impl Trait types: Arguments and return values for const fn can now be opaque impl Trait types.

Note that the trait features do not yet support calling methods from those traits in a const fn.

See the Constant Evaluation section of the reference book to learn more about the current capabilities of const contexts, and future capabilities can be tracked in rust#57563.

Static handles for locked stdio

The three standard I/O streams -- Stdin, Stdout, and Stderr -- each have a lock(&self) to allow more control over synchronizing read and writes. However, they returned lock guards with a lifetime borrowed from &self, so they were limited to the scope of the original handle. This was determined to be an unnecessary limitation, since the underlying locks were actually in static storage, so now the guards are returned with a 'static lifetime, disconnected from the handle.

For example, a common error came from trying to get a handle and lock it in one statement:

// error[E0716]: temporary value dropped while borrowed
let out = std::io::stdout().lock();
//        ^^^^^^^^^^^^^^^^^       - temporary value is freed at the end of this statement
//        |
//        creates a temporary which is freed while still in use

Now the lock guard is 'static, not borrowing from that temporary, so this works!

Stabilized APIs

The following methods and trait implementations are now stabilized:

The following previously stable functions are now const:

Other changes

There are other changes in the Rust 1.61.0 release. Check out what changed in Rust, Cargo, and Clippy.

In a future release we're planning to increase the baseline requirements for the Linux kernel to version 3.2, and for glibc to version 2.17. We'd love your feedback in rust#95026.

Contributors to 1.61.0

Many people came together to create Rust 1.61.0. We couldn't have done it without all of you. Thanks!

Mozilla Addons Blog — Manifest v3 in Firefox: Recap & Next Steps

It’s been about a year since our last update regarding Manifest v3. A lot has changed since then, not least of which has been the formation of a community group under the W3C to advance cross-browser WebExtensions (WECG).

In our previous update, we announced that we would be supporting MV3 and mentioned Service Workers as a replacement for background pages. Since then, it became apparent that numerous use cases would be at risk if this were to proceed as is, so we went back to the drawing board. We proposed Event Pages in the WECG, which has been welcomed by the community and supported by Apple in Safari.

Today, we’re kicking off our Developer Preview program to gather feedback on our implementation of MV3. To set the stage, we want to outline the choices we’ve made in adopting MV3 in Firefox, some of the improvements we’re most excited about, and then talk about the ways we’ve chosen to diverge from the model Chrome originally proposed.

Why are we adopting MV3?

When we decided to move to WebExtensions in 2015, it was a long term bet on cross-browser compatibility. We believed then, as we do now, that users would be best served by having useful extensions available for as many browsers as possible. By the end of 2017 we had completed that transition and moved completely to the WebExtensions model. Today, many cross-platform extensions require only minimal changes to work across major browsers. We consider this move to be a long-term success, and we remain committed to the model.

In 2018, Chrome announced Manifest v3, followed by Microsoft adopting Chromium as the base for the new Edge browser. This means that support for MV3, by virtue of the combined share of Chromium-based browsers, will be a de facto standard for browser extensions in the foreseeable future. We believe that working with other browser vendors in the context of the WECG is the best path toward a healthy ecosystem that balances the needs of its users and developers. For Mozilla, this is a long term bet on a standards-driven future for WebExtensions.

Why is MV3 important to improving WebExtensions?

Manifest V3 is the next iteration of WebExtensions, and offers the opportunity to introduce improvements that would otherwise not be possible due to concerns with backward compatibility. MV2 had architectural constraints that made some issues difficult to address; with MV3 we are able to make changes to address this.

One core part of the extension architecture is the background page, which lives forever by design. Due to memory or platform constraints (e.g. on Android), we can’t guarantee this state, and termination of the background page (along with the extension) is sometimes inevitable. In MV3, we’re introducing a new architecture: the background script must be designed to be restartable. To support this, we’ve reworked existing and introduced new APIs, enabling extensions to declare how the browser should behave without requiring the background script.

Another core part of extensions are content scripts, to directly interact with web pages. We are blocking unsafe coding practices and are offering more secure alternatives to improve the base security of extensions: string-based code execution has been removed from extension APIs. Moreover, to improve the isolation of data between different origins, cross-origin requests are no longer possible from content scripts, unless the destination website opts in via CORS.

User controls for site access

Extensions often need to access user data on websites. While that has enabled extensions to provide powerful features and address numerous user needs, we’ve also seen misuse that impacts user’s privacy.

Starting with MV3, we’ll be treating all site access requests from extensions as optional, and provide users with transparency and controls to make it easier to manage which extensions can access their data for each website.

At the same time, we’ll be encouraging extensions to use models that don’t require permanent access to all websites, by making it easier to grant access for extensions with a narrow scope, or just temporarily. We are continuing to evaluate how to best handle cases, such as privacy and security extensions, that need the ability to intercept or affect all websites in order to fully protect our users.

What are we doing differently in Firefox?

WebRequest

One of the most controversial changes of Chrome’s MV3 approach is the removal of blocking WebRequest, which provides a level of power and flexibility that is critical to enabling advanced privacy and content blocking features. Unfortunately, that power has also been used to harm users in a variety of ways. Chrome’s solution in MV3 was to define a more narrowly scoped API (declarativeNetRequest) as a replacement. However, this will limit the capabilities of certain types of privacy extensions without adequate replacement.

Mozilla will maintain support for blocking WebRequest in MV3. To maximize compatibility with other browsers, we will also ship support for declarativeNetRequest. We will continue to work with content blockers and other key consumers of this API to identify current and future alternatives where appropriate. Content blocking is one of the most important use cases for extensions, and we are committed to ensuring that Firefox users have access to the best privacy tools available.

Event Pages

Chrome’s version of MV3 introduced Background Service Worker as a replacement for the (persistent) Background Page. Mozilla is working on extension Service Workers in Firefox for compatibility reasons, but also because we like that they’re an event-driven environment with defined lifetimes, already part of the Web Platform with good cross-browser support.

We’ve found Service Workers can’t fully support various use cases we consider important, especially around DOM-related features and APIs. Additionally, the worker environment is not as familiar to regular web developers, and our developer community has expressed that completely rewriting extensions can be tedious for thousands of independent developers of existing extensions.

In Firefox, we have decided to support Event Pages in MV3, and our developer preview will not include Service Workers (we’re continuing to work on supporting these for a future release). This will help developers to more easily migrate existing persistent background pages to support MV3 while retaining access to all of the DOM related features available in MV2. We will also support Event Pages in MV2 in an upcoming release, which will additionally aid migration by allowing extensions to transition existing MV2 extensions over a series of releases.

Next Steps for Firefox

In launching our Developer Preview program for Manifest v3, our hope is that authors will test out our MV3 implementation to help us identify gaps or incompatibilities in our implementation. Work is continuing in parallel, and we expect to launch MV3 support for all users by the end of 2022. As we get closer to completion, we will follow up with more detail on timing and how we will support extensions through the transition.

For more information on the Manifest v3 Developer Preview, please check out the migration guide.  If you have questions or feedback on Manifest v3, we would love to hear from you on the Firefox Add-ons Discourse.

The post Manifest v3 in Firefox: Recap & Next Steps appeared first on Mozilla Add-ons Community Blog.

Tiger Oakes — Make advanced React animation easy using finite state machines

Sync state changes with animations while keeping code readable.

David Humphrey — HTTP Testing with Hurl in node.js

The JavaScript ecosystem has been benefiting lately from pieces of its dev tooling being (re)written in Rust.  Projects like swc, Parcel 2 and parcel-css, deno, dprint and others have brought us tremendous performance improvements with tasks like bundling, formatting, etc.  Recently, my favourite Rust-based, HTTP testing tool gained the ability to be run in node/npm projects, and I wanted to show you how it works.

Hurl is a command-line tool for running HTTP requests defined in simple text files (*.hurl).  I learned about it by chance on Twitter over a year ago, and have been using and teaching it to my programming students ever since.  The name comes from the fact that it builds on top of curl's HTTP code. The real benefit to Hurl is that it lets you write simple, declarative tests that read just like the HTTP requests and responses they model.  Oh, and it runs them ridiculously fast!

Here's an example test file that makes sure http://example.net/404.html returns a 404:

GET http://example.net/404.html

HTTP/1.0 404

You can get much fancier, by setting headers, cookies, auth, etc. on the request and assert things on the response, including using JSONPath, XPath, Regexes, and lots of other conveniences.  You can also capture data from the headers or body, and use these variables in subsequent chained requests. The docs are fantastic (including this tutorial), and go through all the various ways you can write your tests.

Here's a slightly more complex test, which uses a few of the techniques I've just mentioned:

# 1. Get the GitHub user info for @Orange-OpenSource
GET https://api.github.com/users/Orange-OpenSource

# 2. We expect to get back an HTTP/2 200 response. Also, assert
# various things about the Headers and JSON body. Finally
# capture the value of the `blog` property from the body into
# a variable, so we can use that in the next step.
HTTP/2 200
[Asserts]
header "access-control-allow-origin" == "*"
jsonpath "$.login" == "Orange-OpenSource"
jsonpath "$.public_repos" >= 286
jsonpath "$.folowers" isInteger
jsonpath "$.node_id" matches /^[A-Za-z0-9=]+$/
[Captures]
blog_url: jsonpath "$.blog" 

# 3. Get the blog URL we received earlier, GET it, and make
# sure it's an HTML page
GET {{blog_url}}

HTTP/2 200
[Asserts]
header "Content-Type" startsWith "text/html"

I've been using Hurl to write tests for node.js HTTP APIs, especially integration tests, and it's been a joy to use.  I still write unit tests in JavaScript-based testing frameworks, but one immediate benefit of adding Hurl is its speed, which helps shake out race conditions.  Many of my students are still learning asynchronous programming, and often forget to await Promise-based calls.  With JavaScript-based test runners, I've found that the test runs take long enough that the promises usually resolve in time (despite not being await'ed), and you often don't realize you have a bug.  However, when I have the students use Hurl, the tests run so fast that any async code path that is missing await becomes obvious: the tests pass in JS but start failing in Hurl.

I also found that Hurl is pretty easy to learn or teach.  My AWS Cloud students picked it up really quickly last term, and I think most node.js devs would have no trouble becoming productive with it in a short time.  Here's what one of my students wrote about getting started with Hurl in his blog:

"The learning curve is pretty simple (I managed to learn the basics in a couple of hours), less setup todo since it's just a plain file, the syntax is English friendly, besides the jsonPath that could take some times to adapt."

As I've been writing tests and teaching with Hurl over the past year, I've been pretty active filing issues. The devs are really friendly and open to suggestions, and the tool has gotten better and better with each new release.  Recently, I filed an issue to add support for running hurl via npm, and it was shipped a little over a week later!

Installing and Using Hurl with npm

Let me show you how to use Hurl in a node.js project.  Say you have a directory of *.hurl files, maybe inside ./test/integration.  First, install Hurl via npm:

$ npm install --save-dev @orangeopensource/hurl

This will download the appropriate Hurl binary for your OS/platform from the associated release, and create node_modules/.bin/hurl which you can call in your scripts within package.json.  For example:

"scripts": {
  "test:integration": "hurl --test --glob \"test/integration/**/*.hurl\""
}

Here I'm using the --test (i.e., run in test mode) and --glob (specify a pattern for input files) options, but there are many more that you can use.  NOTE: I'm not showing how to start a server before running these tests, since that's outside the scope of what Hurl does.  In my case, I typically run my integration tests against Docker containers, but you could do it lots of ways (e.g., use npm-run-all to start your server before running the tests).

In terms of Hurl's output, running the two tests I discussed above looks like this:

npm test

> hurl-npm-example@1.0.0 test
> hurl --test --glob *.hurl

expr=test1.hurl
test2.hurl: RUNNING [1/2]
error: Assert Failure
  --> test2.hurl:14:0
   |
14 | jsonpath "$.folowers" isInteger
   |   actual:   none
   |   expected: integer
   |

test2.hurl: FAILURE
test1.hurl: RUNNING [2/2]
error: Assert Http Version
  --> test1.hurl:3:6
   |
 3 | HTTP/1.0 404
   |      ^^^ actual value is <1.1>
   |

test1.hurl: FAILURE
--------------------------------------------------------------------------------
Executed:  2
Succeeded: 0 (0.0%)
Failed:    2 (100.0%)
Duration:  174ms

As you can see, both tests are failing.  The error message format is more Rust-like than most JS devs will be used to, but it's quite friendly.  In test2.hurl, I've got a typo in $.folowers, and in test1.hurl, the response is returning HTTP/1.1 vs. HTTP/1.0.  A few quick fixes and the tests are now passing:

$ npm test

> hurl-npm-example@1.0.0 test
> hurl --test --glob *.hurl

expr=test1.hurl
test2.hurl: RUNNING [1/2]
test2.hurl: SUCCESS
test1.hurl: RUNNING [2/2]
test1.hurl: SUCCESS
--------------------------------------------------------------------------------
Executed:  2
Succeeded: 2 (100.0%)
Failed:    0 (0.0%)
Duration:  169ms

Part of what's great about Hurl is that it isn't limited to a single language or runtime.  Despite the title of my post, Hurl isn't really a JS testing tool per se.  However, being able to "npm install" it and use it as part of your local or CI testing adds something new to your testing toolkit.  I still love, use, and teach tools like Jest, Playwright, and others, but I'm excited that JS devs now have an easy way to add Hurl to the mix.

Hopefully this will inspire you to try including Hurl in your node.js HTTP project testing.  I promise you that you'll write less test code and spend less time waiting to find out if everything works!

Will Kahn-Greene — Socorro/Tecken Overview: 2022, presentation

Socorro and Tecken make up the services part of our crash reporting system at Mozilla. We ran a small Data Sprint day to onboard a new ops person and a new engineer. I took my existing Socorro presentation and Tecken presentation 1, combined them, reduced them, and then fixed a bunch of issues. This is that presentation.

1

I never blogged the Tecken 2020 presentation.

Read more… (36 min remaining to read)

Mozilla Security Blog — Revocation Reason Codes for TLS Server Certificates

In our continued efforts to improve the security of the web PKI, we are taking a multi-pronged approach to tackling some long-existing problems with revocation of TLS server certificates. In addition to our ongoing CRLite work, we added new requirements to version 2.8 of Mozilla’s Root Store Policy that will enable Firefox to depend on revocation reason codes being used consistently, so they can be relied on when verifying the validity of certificates during TLS connections. We also added a new requirement that CA operators provide their full CRL URLs in the CCADB. This will enable Firefox to pre-load more complete certificate revocation data, eliminating dependency on the infrastructure of CAs during the certificate verification part of establishing TLS connections. The combination of these two new sets of requirements will further enable Firefox to enforce revocation checking of TLS server certificates, which makes TLS connections even more secure.

Previous Policy Updates

Significant improvements have already been made in the web PKI, including the following changes to Mozilla’s Root Store Policy and the CA/Browser Forum Baseline Requirements (BRs), which reduced risks associated with exposure of the private keys of TLS certificates by reducing the amount of time that the exposure can exist.

  • TLS server certificates issued on or after 1 September 2020 MUST NOT have a Validity Period greater than 398 days.
  • For TLS server certificates issued on or after October 1, 2021, each dNSName or IPAddress in the certificate MUST have been validated within the prior 398 days.

Under those provisions, the maximum validity period and maximum re-use of domain validation for TLS certificates roughly corresponds to the typical period of time for owning a domain name; i.e. one year. This reduces the risk of potential exposure of the private key of each TLS certificate that is revoked, replaced, or no longer needed by the original certificate subscriber.

New Requirements

In version 2.8 of Mozilla’s Root Store Policy we added requirements stating that:

  1. Specific RFC 5280 Revocation Reason Codes must be used under certain circumstances; and
  2. CA operators must provide their full CRL URLs in the Common CA Database (CCADB).

These new requirements will provide a complete accounting of all revoked TLS server certificates. This will enable Firefox to pre-load more complete certificate revocation data, eliminating the need for it to query CAs for revocation information when establishing TLS connections.

The new requirements about revocation reason codes account for the situations that can happen at any time during the certificate’s validity period, and address the following problems:

  • There were no policies specifying which revocation reason codes should be used and under which circumstances.
  • Some CAs were not using revocation reason codes at all for TLS server certificates.
  • Some CAs were using the same revocation reason code for every revocation.
  • There were no policies specifying the information that CAs should provide to their certificate subscribers about revocation reason codes.

Revocation Reason Codes

Section 6.1.1 of version 2.8 of Mozilla’s Root Store Policy states that when a TLS server certificate is revoked for one of the following reasons the corresponding entry in the CRL must include the revocation reason code:

  • keyCompromise (RFC 5280 Reason Code #1)
    • The certificate subscriber must choose the “keyCompromise” revocation reason code when they have reason to believe that the private key of their certificate has been compromised, e.g., an unauthorized person has had access to the private key of their certificate.
  • affiliationChanged (RFC 5280 Reason Code #3)
    • The certificate subscriber should choose the “affiliationChanged” revocation reason code when their organization’s name or other organizational information in the certificate has changed.
  • superseded (RFC 5280 Reason Code #4)
    • The certificate subscriber should choose the “superseded” revocation reason code when they request a new certificate to replace their existing certificate.
  • cessationOfOperation (RFC 5280 Reason Code #5)
    • The certificate subscriber should choose the “cessationOfOperation” revocation reason code when they no longer own all of the domain names in the certificate or when they will no longer be using the certificate because they are discontinuing their website.
  • privilegeWithdrawn (RFC 5280 Reason Code #9)
    • The CA will specify the “privilegeWithdrawn” revocation reason code when they obtain evidence that the certificate was misused or the certificate subscriber has violated one or more material obligations under the subscriber agreement or terms of use.

RFC 5280 Reason Codes that are not listed above shall not be specified in the CRL for TLS server certificates, for reasons explained in the wiki page.

Conclusion

These new requirements are important steps towards improving the security of the web PKI, and are part of our effort to resolve long-existing problems with revocation of TLS server certificates. The requirements about revocation reason codes will enable Firefox to depend on revocation reason codes being used consistently, so they can be relied on when verifying the validity of certificates during TLS connections. The requirement that CA operators provide their full CRL URLs in the CCADB will enable Firefox to pre-load more complete certificate revocation data, eliminating dependency on the infrastructure of CAs during the certificate verification part of establishing TLS connections. The combination of these two new sets of requirements will further enable Firefox to enforce revocation checking of TLS server certificates, which makes TLS connections even more secure.

The post Revocation Reason Codes for TLS Server Certificates appeared first on Mozilla Security Blog.

Karl Dubost — How to make « cidre » of Normandy

While I'm living in Japan, I'm from the Normandy region in France. That's me as a child helping my grand father in an orchard.

child filling bags with apples.

This is a region which is traditionally known for cows and dairy products (milk, butter, cheese like Camembert, Pont-L'evêque, Livarot, Neufchâtel, etc.) and also a region which led to the production of cidre (French only). The origin is not that clear but probably, people from Normandy have started to make « cidre » in the 12th century. Some competing stories have been developed about the origin. But people were growing apples for a long time already and probably were fermenting them. And a craft emerged.

The Web is also rich of its individual crafts, developed along the way. Some techniques have been lost, some are thriving. A long series of errors and trials has been essential in perfecting the art of making Websites.

Fast forward a couple of centuries, and here an image on my great grand father, René. He is collecting the apples in a big bag from his field to prepare « cidre ».

Man filling a bag of apples in an orchard.

The name of the apples is a long poetic list:

Blanc Mollet, Girard, Cimetière de Blangy, Argile nouvelle, Fréquin rouge, Gros matois rouge, Bon Henri, Médaille d'or, Petit amer, Binet rouge, Bouquet, Joly rouge, Longue, Bedan, Bouteille, Moulin à vent, Grise Dieppois, Gilet rouge, Doux Veret (petit), Rouge Bruyère, Reine des Pommes, Doux-Evêque précoce, Marin Onfroy, etc.

Each of them have their own qualities: sweetness, acidity, taste, … Once we have a good mix, we need to wash them carefully and then put them in the grinder.

My grand father, Jean, working at the grinder and we can see in the background the press in wood.

Man near the grinder with apples in a big barrel.

Grinder engraving.

Once the apples have been grinded, we need to let them with the juice exposed to the air for around 8 to 12 hours in a deep container covered by a cloth. The oxydation work will start. The must will get a better color, will be sweeter. The yeast will develop more rapidly. Containers must be as clean as possible.

Then will start the work with the press.

Press engraving.

The must is layered in 15 to 20 centimeters high layers, separated by layers of straws that will drain the juice.

Press detail engraving.

Once the juice has been drawn, it is put in big barrels where the fermentation process starts. After a while, the juice will be put in bottle. My grand-father was used to go in the cave and to turn the bottles according to the moon phases. He had 3 types of « cidre » in his cave: Very sweet, rough, and something very rough that was basically impossible to drink. The colors were on the bottles: red, grey and blue, a simple spot of paint.

These techniques are getting lost with the new generations and the industrializations. I wish I had spent more time with him for having a better understanding of the craft.

Different types of apples.

Now, I probably have a better understanding of the Web than the process of making « cidre ». It's probably why today is my first day working for Apple on the WebKit project to continue my journey in making the Web awesome for everyone: Web Compatibility, standards and interoperability.

Engravings coming from Le cidre by Labounoux and Touchard

Comments

If you have more questions, things I may have missed, different take on them. Feel free to comment…. Be mindful.

Otsukare!

Hacks.Mozilla.Org — Improved Process Isolation in Firefox 100

Introduction

Firefox uses a multi-process model for additional security and stability while browsing: Web Content (such as HTML/CSS and Javascript) is rendered in separate processes that are isolated from the rest of the operating system and managed by a privileged parent process. This way, the amount of control gained by an attacker that exploits a bug in a content process is limited.

Ever since we deployed this model, we have been working on improving the isolation of the content processes to further limit the attack surface. This is a challenging task since content processes need access to some operating system APIs to properly function: for example, they still need to be able to talk to the parent process. 

In this article, we would like to dive a bit further into the latest major milestone we have reached: Win32k Lockdown, which greatly reduces the capabilities of the content process when running on Windows. Together with two major earlier efforts (Fission and RLBox) that shipped before, this completes a sequence of large leaps forward that will significantly improve Firefox’s security.

Although Win32k Lockdown is a Windows-specific technique, it became possible because of a significant re-architecting of the Firefox security boundaries that Mozilla has been working on for around four years, which allowed similar security advances to be made on other operating systems.

The Goal: Win32k Lockdown

Firefox runs the processes that render web content with quite a few restrictions on what they are allowed to do when running on Windows. Unfortunately, by default they still have access to the entire Windows API, which opens up a large attack surface: the Windows API consists of many parts, for example, a core part dealing with threads, processes, and memory management, but also networking and socket libraries, printing and multimedia APIs, and so on.

Of particular interest for us is the win32k.sys API, which includes many graphical and widget related system calls that have a history of being exploitable. Going back further in Windows’ origins, this situation is likely the result of Microsoft moving many operations that were originally running in user mode into the kernel in order to improve performance around the Windows 95 and NT4 timeframe.

Having likely never been originally designed to run in this sensitive context, these APIs have been a traditional target for hackers to break out of application sandboxes and into the kernel.

In Windows 8, Microsoft introduced a new mitigation named PROCESS_MITIGATION_SYSTEM_CALL_DISABLE_POLICY that an application can use to disable access to win32k.sys system calls. That is a long name to keep repeating, so we’ll refer to it hereafter by our internal designation: “Win32k Lockdown“.

The Work Required

Flipping the Win32k Lockdown flag on the Web Content processes – the processes most vulnerable to potentially hostile web pages and JavaScript – means that those processes can no longer perform any graphical, window management, input processing, etc. operations themselves.

To accomplish these tasks, such operations must be remoted to a process that has the necessary permissions, typically the process that has access to the GPU and handles compositing and drawing (hereafter called the GPU Process), or the privileged parent process. 

Drawing web pages: WebRender

For painting the web pages’ contents, Firefox historically used various methods for interacting with the Windows APIs, ranging from using modern Direct3D based textures, to falling back to GDI surfaces, and eventually dropping into pure software mode.

These different options would have taken quite some work to remote, as most of the graphics API is off limits in Win32k Lockdown. The good news is that as of Firefox 92, our rendering stack has switched to WebRender, which moves all the actual drawing from the content processes to WebRender in the GPU Process.

Because with WebRender the content process no longer has a need to directly interact with the platform drawing APIs, this avoids any Win32k Lockdown related problems. WebRender itself has been designed partially to be more similar to game engines, and thus, be less susceptible to driver bugs.

For the remaining drivers that are just too broken to be of any use, it still has a fully software-based mode, which means we have no further fallbacks to consider.

Webpages drawing: Canvas 2D and WebGL 3D

The Canvas API provides web pages with the ability to draw 2D graphics. In the original Firefox implementation, these JavaScript APIs were executed in the Web Content processes and the calls to the Windows drawing APIs were made directly from the same processes.

In a Win32k Lockdown scenario, this is no longer possible, so all drawing commands are remoted by recording and playing them back in the GPU process over IPC.

Although the initial implementation had good performance, there were nevertheless reports from some sites that experienced performance regressions (the web sites that became faster generally didn’t complain!). A particular pain point are applications that call getImageData() repeatedly: having the Canvas remoted means that GPU textures must now be obtained from another process and sent over IPC.

We compensated for this in the scenario where getImageData is called at the start of a frame, by detecting this and preparing the right surfaces proactively to make the copying from the GPU faster.

Besides the Canvas API to draw 2D graphics, the web platform also exposes an API to do 3D drawing, called WebGL. WebGL is a state-heavy API, so properly and efficiently synchronizing child and parent (as well as parent and driver) takes great care.

WebGL originally handled all validation in Content, but with access to the GPU and the associated attack surface removed from there, we needed to craft a robust validating API between child and parent as well to get the full security benefit.

(Non-)Native Theming for Forms

HTML web pages have the ability to display form controls. While the overwhelming majority of websites provide a custom look and styling for those form controls, not all of them do, and if they do not you get an input GUI widget that is styled like (and originally was!) a native element of the operating system.

Historically, these were drawn by calling the appropriate OS widget APIs from within the content process, but those are not available under Win32k Lockdown.

This cannot easily be fixed by remoting the calls, as the widgets themselves come in an infinite amount of sizes, shapes, and styles can be interacted with, and need to be responsive to user input and dispatch messages. We settled on having Firefox draw the form controls itself, in a cross-platform style.

While changing the look of form controls has web compatibility implications, and some people prefer the more native look – on the few pages that don’t apply their own styles to controls – Firefox’s approach is consistent with that taken by other browsers, probably because of very similar considerations.

Scrollbars were a particular pain point: we didn’t want to draw the main scrollbar of the content window in a different manner as the rest of the UX, since nested scrollbars would show up with different styles which would look awkward. But, unlike the rather rare non-styled form widgets, the main scrollbar is visible on most web pages, and because it conceptually belongs to the browser UX we really wanted it to look native.

We, therefore, decided to draw all scrollbars to match the system theme, although it’s a bit of an open question though how things should look if even the vendor of the operating system can’t seem to decide what the “native” look is.

Final Hurdles

Line Breaking

With the above changes, we thought we had all the usual suspects that would access graphics and widget APIs in win32k.sys wrapped up, so we started running the full Firefox test suite with win32k syscalls disabled. This caused at least one unexpected failure: Firefox was crashing when trying to find line breaks for some languages with complex scripts.

While Firefox is able to correctly determine word endings in multibyte character streams for most languages by itself, the support for Thai, Lao, Tibetan and Khmer is known to be imperfect, and in these cases, Firefox can ask the operating system to handle the line breaking for it. But at least on Windows, the functions to do so are covered by the Win32k Lockdown switch. Oops!

There are efforts underway to incorporate ICU4X and base all i18n related functionality on that, meaning that Firefox will be able to handle all scripts perfectly without involving the OS, but this is a major effort and it was not clear if it would end up delaying the rollout of win32k lockdown.

We did some experimentation with trying to forward the line breaking over IPC. Initially, this had bad performance, but when we added caching performance was satisfactory or sometimes even improved, since OS calls could be avoided in many cases now.

DLL Loading & Third Party Interactions

Another complexity of disabling win32k.sys access is that so much Windows functionality assumes it is available by default, and specific effort must be taken to ensure the relevant DLLs do not get loaded on startup. Firefox itself for example won’t load the user32 DLL containing some win32k APIs, but injected third party DLLs sometimes do. This causes problems because COM initialization in particular uses win32k calls to get the Window Station and Desktop if the DLL is present. Those calls will fail with Win32k Lockdown enabled, silently breaking COM and features that depend on it such as our accessibility support. 

On Windows 10 Fall Creators Update and later we have a fix that blocks these calls and forces a fallback, which keeps everything working nicely. We measured that not loading the DLLs causes about a 15% performance gain when opening new tabs, adding a nice performance bonus on top of the security benefit.

Remaining Work

As hinted in the previous section, Win32k Lockdown will initially roll out on Windows 10 Fall Creators Update and later. On Windows 8, and unpatched Windows 10 (which unfortunately seems to be in use!), we are still testing a fix for the case where third party DLLs interfere, so support for those will come in a future release.

For Canvas 2D support, we’re still looking into improving the performance of applications that regressed when the processes were switched around. Simultaneously, there is experimentation underway to see if hardware acceleration for Canvas 2D can be implemented through WebGL, which would increase code sharing between the 2D and 3D implementations and take advantage of modern video drivers being better optimized for the 3D case.

Conclusion

Retrofitting a significant change in the separation of responsibilities in a large application like Firefox presents a large, multi-year engineering challenge, but it is absolutely required in order to advance browser security and to continue keeping our users safe. We’re pleased to have made it through and present you with the result in Firefox 100.

Other Platforms

If you’re a Mac user, you might wonder if there’s anything similar to Win32k Lockdown that can be done for macOS. You’d be right, and I have good news for you: we already quietly shipped the changes that block access to the WindowServer in Firefox 95, improving security and speeding process startup by about 30-70%. This too became possible because of the Remote WebGL and Non-Native Theming work described above.

For Linux users, we removed the connection from content processes to the X11 Server, which stops attackers from exploiting the unsecured X11 protocol. Although Linux distributions have been moving towards the more secure Wayland protocol as the default, we still see a lot of users that are using X11 or XWayland configurations, so this is definitely a nice-to-have, which shipped in Firefox 99.

We’re Hiring

If you found the technical background story above fascinating, I’d like to point out that our OS Integration & Hardening team is going to be hiring soon. We’re especially looking for experienced C++ programmers with some interest in Rust and in-depth knowledge of Windows programming.

If you fit this description and are interested in taking the next leap in Firefox security together with us, we’d encourage you to keep an eye on our careers page.

Thanks to Bob Owen, Chris Martin, and Stephen Pohl for their technical input to this article, and for all the heavy lifting they did together with Kelsey Gilbert and Jed Davis to make these security improvements ship.

The post Improved Process Isolation in Firefox 100 appeared first on Mozilla Hacks - the Web developer blog.

Firefox Add-on Reviews — Extension starter pack

You’ve probably heard about “ad blockers,” “tab managers,” “anti-trackers” or any number of browser customization tools commonly known as extensions. And maybe you’re intrigued to try one, but you’ve never installed an extension before and the whole notion just seems a bit obscure. 

Let’s demystify extensions. 

An extension is simply an app that runs on a browser like Firefox. From serious productivity and privacy enhancing features to really fun stuff like changing the way the web looks and feels, extensions give you the power to completely personalize your browsing experience. 

Addons.mozilla.org (AMO) is a discovery site that hosts thousands of independently developed Firefox extensions. It’s a vast and eclectic ecosystem of features, so we’ve hand-picked a small collection of great extensions to get you started…

I’ve always wanted an ad blocker!

uBlock Origin

Works beautifully “right out of the box.” Just add it to Firefox and uBlock Origin will automatically start blocking all types of advertising — display ads, banners, video pre-rolls, pop-ups — you name it. 

Of course, if you prefer deeper content blocking customization, uBlock Origin allows for fine control as well, like the ability to import your own custom block filters or access a data display that shows you how much of a web page was blocked by the extension. More than just an ad blocker, uBlock Origin also effectively thwarts some websites that may be infected with malware. 

For more insights about this excellent ad blocker, please see uBlock Origin — everything you need to know about the ad blocker, or to explore additional ad blocker options, you might want to check out What’s the best ad blocker for you?

I’m concerned about my digital privacy and being tracked around the web

Privacy Badger

The beauty of Privacy Badger is that once you install it on Firefox, it not only immediately begins blocking some of the sneakiest trackers on the web, it actually gets “smarter” the longer you use it. 

No complicated set-up required. Once installed, Privacy Badger automatically hunts down and blocks the most hidden types of trackers on the web. As you naturally navigate the internet, Privacy Badger will build on its knowledge base of invisible trackers and become increasingly effective at obscuring your online trail. 

Facebook Container

Mozilla’s very own Facebook Container was built to give Firefox users a no-fuss way to stop Facebook from tracking your moves outside of Facebook. 

As its title suggests, the extension “contains” your identity to just Facebook, without requiring you to  sign in/out from the website each time you use it (typically, the trade-off for the ease of always remaining signed in is that it gives Facebook a means of tracking you around the web). So the extension offers you the best of both worlds — maintain the convenience of auto sign-in and build privacy protection for yourself so Facebook can’t pry into the rest of your web activity. 

<figcaption>Social widgets like these give Facebook and other platforms a sneaky means of tracking your interests around the web.</figcaption>

I need an easier way to translate languages

Simple Translate

Do you do a lot of language translations on the web? If so, it’s a hassle always copying text and navigating away from the page you’re on just to translate a word or phrase. Simple Translate solves this problem by giving you the power to perform translations right there on the page. 

Just highlight the text you want translated and right-click to get instant translations in a handy pop-up display, so you never have to leave the page again. 

My grammar in speling is bad!

LanguageTool

Anywhere you write on the web, LanguageTool will be there to lend a guiding editorial hand. It helps fix typos, grammar problems, and even recognizes common word mix-ups like the there/their/they’re. 

Available in 25 languages, LanguageTool automatically works on any web-based publishing platform like Gmail, web docs, social media sites, etc. The clever extension will even spot words you’re possibly overusing and suggest alternatives to spruce up your prose. 

YouTube your way

Enhancer for YouTube

Despite offering dozens of creative customization features, Enhancer for YouTube is easy to grasp and gives you a variety of ways to radically alter YouTube functionality. 

Once the extension is installed you’ll find an additional set of controls just beneath YouTube’s video player (you can even select the extension features you want to appear in the control bar). 

Key features include… 

  • Customize video player size
  • Change YouTube’s look with a dark theme
  • Volume boost
  • Ad blocking (with ability to allow ads from channels you choose to support)
  • One-click screenshots
  • Change playback speed
  • High-def default video quality

I’m drowning in browser tabs! Send help! 

OneTab

You’ve got an overwhelming number of open web pages. You can’t close them. You need them. But you can’t organize them all right now either. You’re too busy. What to do?! 

If you have OneTab on Firefox you just click the toolbar button and suddenly all those open tabs become a clean list of text links listed on a single page. Ahhh serenity.

Not only will you create browser breathing room for yourself, but with all those previously open tabs now closed and converted to text links, you’ve also just freed up a bunch of CPU and memory, which should improve browser speed and performance. 

If you’ve never installed a browser extension before, we hope you found something here that piqued your interest to try. To continue exploring ways to personalize Firefox through the power of extensions, please see our collection of 100+ Recommended Extensions. 

Mozilla Thunderbird — 7 Great New Features Coming To Thunderbird 102

Welcome back to the Thunderbird blog! We’re really energized about our major 2022 release and cannot wait to put it in your hands. Thunderbird 102 includes several major new features for our global community of users, and we’re confident you’ll love them. So grab your favorite beverage, and let’s highlight seven features from Thunderbird 102 we’re most excited about.

Before we jump in, it’s worth mentioning that we’ve been rapidly expanding our team in order to power up your productivity and improve your favorite email client. From major milestones like a completely modernized UI/UX in next year’s Thunderbird 114 (codenamed “Supernova”) to smaller touches like new iconography, elegant new address book functionality, and an Import/Export wizard, all of it happens for you and because of you. Thunderbird not only survives but thrives thanks to your generous donations. Every amount, large or small, makes a difference. Please consider donating what you can, and know that we sincerely appreciate your support!

OK! Here's an overview of the new features in Thunderbird 102. Stay tuned to our blog for in-depth updates and deeper dives leading up to the late June release.

#1: The New Address Book In Thunderbird 102

We’ve teased a new address book in the past, and it’s finally coming in Thunderbird 102. Not only does the refreshed design make it easier to navigate and interact with your contacts, but it also boasts new features to help you better understand who you’re communicating with.

Complete address book entry in Thunderbird 102<figcaption>Address Book gets a new look and enhanced functionality in Thunderbird 102</figcaption>

The new Address Book has compatibility with vCard specs, the defacto standard for saving contacts. If your app (like Google Contacts) or device (iPhone, Android) can export existing contacts into vCard format, Thunderbird can import them. And as you can see from the above screenshot, each contact card acts as a launchpad for messaging, email, or event creation involving that contact.

We’re also adding several more fields to each contact entry, and they’re displayed in a much better, clearer way than before.

Your contacts are getting a serious upgrade in Thunderbird 102! There’s so much more to share on this front, so please watch this blog for a standalone deep-dive on the new Address Book in the near future.

#2: The Spaces Toolbar

One of the underlying themes of Thunderbird 102 is making the software easier to use, with smarter visual cues that can enhance your productivity. The new Spaces Toolbar is an easy, convenient way to move between all the different activities in the application. Such as managing your email, working with contacts via that awesome new address book, using the calendar and tasks functionality, chat, and even add-ons!

The Spaces Toolbar, on the left-hand side of Thunderbird<figcaption>The Spaces Toolbar, on the left-hand side of Thunderbird</figcaption>

If you want to save screen real estate, the Spaces Toolbar can be dismissed, and you can instead navigate the different activities Thunderbird offers with the new pinned Spaces tab. (Pictured to the left of the tabs at the top)

Pinned spaces tab showing the different activities, to the left of the tabs.<figcaption>Pinned Spaces Tab</figcaption>

#3: Link Preview Cards

Want to share a link with your friends or your colleagues, but do it with a bit more elegance? Our new Link Preview Cards do exactly that. When you paste a link into the compose window, we’ll ask you (via a tooltip you can turn off) if you’d like to display a rich preview of the link. It’s a great way for your recipient to see at a glance what they’re clicking out to, and a nice way for your emails to have a bit more polish if desired!

Embedded Link Previews in Thunderbird 102<figcaption>Embedded Link Previews in Thunderbird 102</figcaption>

#4: Account Setup Hub In Thunderbird 102

In past releases, we have improved first-time account setup. When setting up an email, autodiscovery of calendars and address books works really well! But managing accounts and setting up additional accounts beyond your initial setup has lagged behind. We are updating that experience in Thunderbird 102.

Want to use Thunderbird without an email account? We know you exist, and we’re making this much easier for you! After installing the software, from now on you’ll be taken to the below account hub instead of being forced to set up a new mail account. You’re free to configure Thunderbird in the order you choose, and only the elements you choose.

New Account Setup Hub in Thunderbird 102<figcaption>New Account Setup Hub in Thunderbird 102</figcaption>

#5: Import/Export Wizard

And that’s a perfect segue into the brand new Import and Export tool. Moving accounts and data in and out of Thunderbird should be a breeze! Until now, you’ve had to use add-ons for this, but we’re excited to share that this is now core functionality with Thunderbird 102.

A step-by-step wizard will provide a guided experience for importing all that data that’s important to you. Moving from Outlook, SeaMonkey, or another Thunderbird installation will be easier than ever.

A screenshot from the new Import/Export wizard<figcaption>A screenshot from the new Import/Export wizard</figcaption>

We’ve also taken extra precautions to ensure that no data is accidentally duplicated in your profile after an import. To that end, none of the actions you choose are executed until the very last step in the process. As with the new Address Book, watch for a deeper dive into the new Import/Export tool in a future blog post.

#6: Matrix Chat Support

We obviously love open source, which is one of the reasons why we’ve added support for the popular, decentralized chat protocol Matrix into Thunderbird 102. Those of you enjoying the Beta version know it’s been an option since version 91, but it will finally be usable out-of-the-box in this new release. We’re going to continuously develop updates to the Matrix experience, and we welcome your feedback.

#7: Message Header Redesign

Another UX/Visual update can be seen in the redesign of the all-important message header. The refreshed design better highlights important info, making it more responsive and easier for you to navigate.

Redesigned message header in Thunderbird 102<figcaption>Redesigned message header in Thunderbird 102</figcaption>

All of these improvements are gradual but confident steps toward the major release of Thunderbird 114 “Supernova” in 2023, which is set to deliver a completely modernized overhaul to the Thunderbird interface.

Thunderbird 102 Availability?

We think you’re going to love this release and can’t wait for you to try it!

Interested in experiencing Thunderbird 102 early? It should be available in our Beta channel by the end of May 2022. We encourage you to try it! We’ve entered “feature freeze” for version 102, and are focusing on polishing it up now. That means your Beta experience should be quite stable.

For everyone who’s enjoying the Stable version, you can expect it by the end of June 2022.

Thunderbird is the leading open-source, cross-platform email and calendaring client, free for business and personal use. We want it to stay secure and become even better. A donation will allow us to hire developers, pay for infrastructure, expand our userbase, and continue to improve.

Click here to make a donation. 

The post 7 Great New Features Coming To Thunderbird 102 appeared first on The Thunderbird Blog.

Karl Dubost — Mozilla, Bye!

This year, 2022, May the 4th was my last day at Mozilla.

Trunk from an oak tree and its canopy.

Alors, l'arbre et son rêveur, ensemble, s'ordonnent, grandissent. Jamais l'arbre, dans le monde du songe, ne s'établit comme un être achevé. — Poétique de l'espace, Gaston Bachelard

in English

Then, together, the tree and its dreamer, take their places, grow tall. Never, in the dream world, does a tree appear as a completed being. — The poetics of space, Gaston Bachelard

I started on July 2, 2013 on a 6 months contract at Mozilla to work on Firefox OS on Web Compatibility issues. I was living in Montréal, Canada at the time.

Lawrence Mandel (now at Shopify) trusted and hired me. His first words on our Web Compatibility work at Mozilla were aligned with my ideas and stance for the open Web.

We are here to make the Web more open, not only for making the Web usable on Firefox products. — Lawrence Mandel

After these 6 months, I moved to Japan and I'm still living there. I'm currently in Tokyo. On the span of 8 years and 10 months, I focused my energy on this mission inside the Web Compatibility.

A person should be able to use the Web with any devices and any browsers.

I was not alone. The success of a project never relies on a single individual, but a full team of people dedicated to make this mission a reality. At the very beginning, we were three coming from Opera Software, we all had an experience on Web compatibility issues: Mike Taylor, Hallvord R.M. Steen and me. Then Adam Stevenson joined. None of the initial team is still at Mozilla. I miss working with Eric Tsai too. Some people (open contributors) have also participated to the project like Abdul Rauf, Alexa Roman, Kate Manning, Guillaume Demesy, Reinhart Previano.

webcompat.com was setup on purpose without Mozilla branding to invite the participation of all browser implementers (Apple, Google, Microsoft, Opera, etc.) on solving issues resulting from website mistakes or interoperability issues. Mozilla put the main effort into it and in return webcompat.com helped Mozilla and Firefox Core team to fix a lot of issues.

The current Web Compatibility team is composed of Dennis Schubert (Germany), James Graham (UK), Ksenia Berezina (Canada), Oana Arbuzov (Romania), Raul Bucata (Romania) and Thomas Wisniewski (Canada). This team was distributed across three continents (two since I left), working around the clock to help solving Web compatibility issues. All the work done was in public, shared with others, written down and tracked in the open. This leveraged autonomy and responsibility from everyone in the team. Apart of a lack of a resource, my departure doesn't put in peril the work of the team. Even as I became the team manager 18 months ago, I was not a gatekeeper.

There is the Webcompat team… then there is the amazing group of Core Engineers who have the open Web deep in their heart. Many left Mozilla, but some of them are still there and they were instrumental in solving interoperability issues.

Emilio Cobos Álvarez, Daniel Holbert, Jonathan Kew, Masayuki Nakano, Makoto Kato, Brian Birtles, Boris Zbarsky, Hiroyuki Hikezoe, Botond Ballo, Olli Pettay, Henri Sivonen, Anne van Kesteren, Ting-Yu Lin, Cameron McCormack. These lists are dangerous, I keep forgetting people.

I could talk about all the things which have been solved around text input, CSS flexbox, JavaScript features, DOM and SVG, … but this starts to be long.

And finally the diagnosis ability of the Webcompat team would be nothing without the dedication of the devtools and performance teams. They helped us to work better, they develop amazing tools which are useful for the webcompat team and the web developers. They always care about what we do. Nicolas Chevobbe, Julien Wajsberg, Daisuke Akatsuka, Jan Odvarko (Honza), and many others …

But as Bachelard said above:

Never, in the dream world, does a tree appear as a completed being.

The new chapter is starting on May 16, 2022. More information on that later apart of the lunar eclipse.

Comments

If you have more questions, things I may have missed, different take on them. Feel free to comment…. Be mindful.

Otsukare!

The Rust Programming Language Blog — Security advisory: malicious crate rustdecimal

This is a cross-post of the official security advisory. The official advisory contains a signed version with our PGP key, as well.

The Rust Security Response WG and the crates.io team were notified on 2022-05-02 of the existence of the malicious crate rustdecimal, which contained malware. The crate name was intentionally similar to the name of the popular rust_decimal crate, hoping that potential victims would misspell its name (an attack called "typosquatting").

To protect the security of the ecosystem, the crates.io team permanently removed the crate from the registry as soon as it was made aware of the malware. An analysis of all the crates on crates.io was also performed, and no other crate with similar code patterns was found.

Keep in mind that the rust_decimal crate was not compromised, and it is still safe to use.

Analysis of the crate

The crate had less than 500 downloads since its first release on 2022-03-25, and no crates on the crates.io registry depended on it.

The crate contained identical source code and functionality as the legit rust_decimal crate, except for the Decimal::new function.

When the function was called, it checked whether the GITLAB_CI environment variable was set, and if so it downloaded a binary payload into /tmp/git-updater.bin and executed it. The binary payload supported both Linux and macOS, but not Windows.

An analysis of the binary payload was not possible, as the download URL didn't work anymore when the analysis was performed.

Recommendations

If your project or organization is running GitLab CI, we strongly recommend checking whether your project or one of its dependencies depended on the rustdecimal crate, starting from 2022-03-25. If you notice a dependency on that crate, you should consider your CI environment to be compromised.

In general, we recommend regularly auditing your dependencies, and only depending on crates whose author you trust. If you notice any suspicious behavior in a crate's source code please follow the Rust security policy and report it to the Rust Security Response WG.

Acknowledgements

We want to thank GitHub user @safinaskar for identifying the malicious crate in this GitHub issue.

The Talospace Project — Firefox 100 on POWER

You know, it's not been a great weekend. Between striking out on some classic hardware projects, leaving printed circuit board corpses in my living room like some alternative universe fusion of William Gibson and Jeffrey Dahmer, one of the yard sprinkler valves has decided it will constantly water the hedge (a couple hundred bucks to fix) and I managed to re-injure my calf muscle sprinting to try to get a phone call (it went pop, I yelped and they hung up anyway). But Firefox is now at version 100, so there's that. Besides the pretty version window when you start it up, it has captions on picture-in-picture and various performance improvements.

Fx100 also broke our PGO-LTO gcc patches again; mach now needs to be patched to ignore how gcc captures profiling information or it will refuse to start a PGO build. This is rolled into the ensemble PGO-LTO patch, which works with the same .mozconfigs from FIrefox 95.

Between everything that's been going on and other projects I've wanted to pay attention to I don't think we're going to make the Fx102 ESR window for the POWER9 JIT. I'll still offer patches for 102ESR; you'll just have to apply them like you do for Firefox 91ESR. Meanwhile, I'll keep trying to get the last major bugs out as I have time, inclination and energy, but although I know people want this really badly, we need more eyes on the code than just me.

Spidermonkey Development Blog — SpiderMonkey Newsletter (Firefox 100-101)

SpiderMonkey is the JavaScript engine used in Mozilla Firefox. This newsletter gives an overview of the JavaScript and WebAssembly work we’ve done as part of the Firefox 100 and 101 Nightly release cycles.

👷🏽‍♀️ New features

⚙️ Modernizing JS modules

We’re working on improving our implementation of modules. This includes supporting modules in Workers, adding support for Import Maps, and ESM-ification (replacing the JSM module system for Firefox internal JS code with standard ECMAScript modules).

  • We added support for caching module scripts in the bytecode cache.
  • We landed support for caching modules in the StartupCache.
  • We’ve landed many changes to improve the browser’s module loader code.
  • We’ve added a common base class to work towards supporting modules in Workers.
  • We’ve landed a cleanup for the Worker Script Loader.

🚀 JS Performance

  • We’ve added support for functions with simple try-finally blocks to the optimizing JIT, fixing a very old performance cliff. Support for more complicated cases will be added in a later release.
  • We’ve improved instanceof performance by removing the non-standard JSClass hook, proxy trap, and by optimizing polymorphic cases better in the JITs.
  • We changed how function.call and function.apply are optimized. This is more robust and fixes some performance cliffs.
  • We added support for optimizing builtin functions in CacheIR when called through function.call.
  • We used this to optimize the common slice.call(arguments) pattern in the JITs.
  • We optimized certain calls into C++ from optimized JIT code by removing (unnecessary) dynamic stack alignment.
  • We improved CacheIR support for property-init operations.
  • We reimplemented new.target as a real binding.
  • We added support for scalar replacement of array iterator objects.

🏎️ WebAssembly Performance

  • We moved trap instructions out-of-line to improve branch prediction.
  • We merged wasm::Instance and TlsData. This eliminates some memory loads.
  • We improved Baseline code performance by pinning the Instance/TLS register on most platforms.
  • We fixed some DevTools performance issues: opening the web console no longer results in using slower debugging code for Wasm modules and we fixed debugging support to not clone Wasm code.
  • We optimized a common instruction sequence with SIMD instructions.
  • We added AVX support for all binary SIMD operations.
  • We enabled support for AVX instructions for Wasm SIMD on Nightly.
  • We optimized table.get/set for anyref tables.
  • We optimized memory.copy/fill for Memory64.

📚 Miscellaneous

  • We fixed a number of compiler errors when compiling in C++20 mode.
  • We’ve updated ICU to version 71.
  • We landed a workaround for a glibc pthreads bug that was causing intermittent hangs in CI for JS shell tests on Linux.
  • We stopped using extended functions for most class methods, to reduce memory usage.

Mozilla Thunderbird — OpenPGP keys and SHA-1

As you may know, Thunderbird offers email encryption and digital email signatures using the OpenPGP technology and uses Ribose’s RNP library that provides the underlying functionality.

To strengthen the security of the OpenPGP implementation, a recent update of the RNP library had included changes to refuse the use of several unsafe algorithms, such as MD5 and SHA-1. The Thunderbird team had delivered RNP version 0.16.0 as part of the Thunderbird 91.8.0 update.

Unfortunately, this change resulted in some users no longer being able to use their OpenPGP keys. We learned that the affected users still depend on keys that were created or modified with OpenPGP software that used SHA-1 for the signatures that are part of OpenPGP keys.

After analyzing and discussing the issue, it was decided to continue to allow SHA-1 for this use of signatures, also known as binding signatures. This matches the behavior of other popular OpenPGP software like GnuPG. Thunderbird 91.9.0 includes this fix, and will be released today.

While some attacks on SHA-1 are possible, the currently known attacks are difficult to apply on OpenPGP binding signatures. In addition, RNP 0.16.0 includes an improvement that provides SHA-1 collision detection code, and it is assumed it makes it even more difficult for an attacker to abuse the fact that Thunderbird accepts SHA-1 in binding signatures.

More details on the background, on the affected and future versions, and considerations for other OpenPGP software, can be found in the following knowledge base article:

https://support.mozilla.org/en-US/kb/openpgp-unsafe-key-properties-ignored

 

Follow Thunderbird on Mastodon.

The post OpenPGP keys and SHA-1 appeared first on The Thunderbird Blog.

William Durand — Developing Firefox in Firefox with Gitpod

Gitpod provides Linux-based development environments on demand along with a web editor frontend (VS Code). There is apparently no limit on what you can do in a Gitpod workspace, e.g., I ran my own toy kernel in QEMU in the browser.

I like Gitpod because it…

  • avoids potential issues when setting up a new project, which is great for the maintainers (e.g., it is easier to reproduce an issue when you have access to the same environment) and the newcomers (e.g., no fatal error when trying to contribute for the first time)
  • allows anyone to have access to a (somewhat) powerful machine because not everyone can afford a MacBook Pro. I suppose one needs a pretty reliable Internet connection, though

Motivations

My main machine runs MacOS. I also have a Windows machine on my desk (useful to debug annoying Windows-only intermittent failures), which I access with Microsoft Remote Desktop.

It looks like Gitpod, runs like Gitpod, and quacks like Gitpod but it isn’t Gitpod!

Except my ever growing collection of single-board computers, I don’t have a Linux-based machine at home^W work, though. I haven’t used Virtual Machines on Apple Silicon yet and I’d rather keep an eye on Asahi Linux instead.

This is why I wanted to give Gitpod a try and see if it could become my Linux environment for working on Firefox. Clearly, this is a nice to have for me (I don’t necessarily need to build Firefox on Linux) but that might be useful to others.

Assuming a Gitpod-based setup would work for me, this could possibly become a strong alternative for other contributors as well. Then, mayyybe I could start a conversation about it internally in the (far) future.

Note that this isn’t a novel idea, Jan Keromnes was already working on a similar tool called “Janitor” for Mozilla needs in 2015.

Firefox development with Gitpod = ❤️

I recently put together a proof of concept and played with it since then. This GitHub repository contains the Gitpod configuration to checkout the Firefox sources as well as the tools required to build Firefox.

It takes about 7 minutes to be able to run ./mach run in a fresh workspace (artifact mode). It is not super fast, although I already improved the initial load time by using a custom Docker image. It is also worth mentioning that re-opening an existing workspace is much faster!

./mach run executed in a Gitpod workspace

Gitpod provides a Docker image with X11 and VNC, which I used as the base for my custom Docker image. This is useful to interact with Firefox as well as observing some of the tests running.

A “mochitest” running in a Gitpod workspace

I don’t know if this is the right approach, though. My understanding is that Gitpod works best when its configuration lives next to the sources. For Firefox, that would mean having the configuration in the official Mozilla Mercurial repositories but then we would need hgpod.io 😅

On the other hand, we develop Firefox using a stacked diff workflow. Therefore, we probably don’t need most of the VCS features and Gitpod does not want to be seen as an online IDE anyway. I personally rely on git with the excellent git-cinnabar helper, and moz-phab to submit patches to Phabricator. Except for the latter, which can easily be installed with ./mach install-moz-phab, these tools are already available in Gitpod workspaces created with my repository.

In terms of limitations, cloning mozilla-unified is the most time-consuming task at the moment. Gitpod has a feature called Prebuilds that could help but I am not sure how that would work when the actual project repository isn’t the one that contains the Gitpod configuration.

In case you’re wondering, I also started a full build (no artifact) and it took about 40 minutes to finish 🙁 I was hoping to have better performances out of the box even if it isn’t that bad. For comparison, it takes ~15min on my MacBook Pro with M1 Max (and 2 hours on my previous Apple machine).

There are other things that I’d like to poke around. For instance, I would love to have rr support in Gitpod. I gave it a quick try and it does not seem possible so far, maybe because of how Docker is configured.

gitpod /workspace/gitpod-firefox-dev/mozilla-unified (bookmarks/central) $ rr record -n /usr/bin/ls
[FATAL /tmp/rr/src/PerfCounters.cc:224:start_counter() errno: EPERM] Failed to initialize counter

After a few messages exchanged on Twitter, Jan Keromnes (yeah, same as above) from Gitpod filed a feature request to support rr 🤞

Next steps

As I mentioned previously, this is a proof of concept but it is already functional. I’ll personally continue to evaluate this Linux-based development environment. If it gets rr support, this might become my debugging environment of choice!

Now, if you are interested, you can go ahead and create a new workspace automatically. Otherwise please reach out to me and let’s discuss!

One more thing: Mozilla employees (and many other Open Source contributors) qualify for the Gitpod for Open Source program.

Cameron Kaiser — April patch set for TenFourFox

I've had my hands full with the POWER9 64-bit JIT (a descendant of TenFourFox's 32-bit JIT), but I had a better idea about the lazy image loader workaround in February's drop and got the rest of the maintenance patches down at the same time. These patches include the standard security fixes and updates to timezones, pinned certificates and HSTS, as well as another entry for the ATSUI font blacklist. In addition, a bug in the POWER9 JIT turns out to affect TenFourFox as well (going back to at least TenFourFox 38), which I ported a fix for. It should correct some residual issues with IonPower-NVLE on a few sites, though it may allow code to run that couldn't run before that may have its own issues, of course. A clobber is not required for this update. The commits are already hot and live on Github.

The next ESR, version 102, is due in June. I'll change the EV and certificate roots over around that time as usual, but we might also take the opportunity to pull up some of the vendored libraries like zlib, so it might be a somewhat bigger update than it would ordinarily be.

William Durand — Moziversary #4

Today is my fourth Moziversary 🎂 I have been working at Mozilla as a full-time employee for 4 years. I blogged two times before: in 2020 and 2021. What happened in 2019? I. Don’t. Know.

I was hired as a Senior Web Developer on addons.mozilla.org (AMO). I am now a Staff Software Engineer in the Firefox WebExtensions team. I officially joined this team in January. Since then, I became a peer of the Add-ons Manager and WebExtensions modules.

Farewell AMO!

As mentioned above, I moved to another team after many years in the AMO team. If I had to summarize my time in this team, I would probably say: “I did my part”.

Earlier this year, I transferred ownership of more than 10 projects that I either created or took over to my former (AMO) colleagues 😬 I was maintaining these projects in addition to the bulk of my work, which has been extremely diverse. As far as I can remember, I…

  • worked on countless user-facing features on AMO, quickly becoming the top committer on addons-frontend (for what it’s worth)
  • contributed many improvements to the AMO backend (Django/Python). For example, I drastically improved the reliability of the Git extraction system that we use for signed add-ons
  • developed a set of anti-malware scanning and code search tools that have been “a game changer to secure the add-ons ecosystem for Firefox”
  • introduced AWS Lambda to our infrastructure for some (micro) services
  • created prototypes, e.g., I wrote a CLI tool leveraging SpiderMonkey to dynamically analyze browser extension behaviors
  • almost integrated Stripe with AMO 🙃
  • shipped entire features spanning many components end-to-end like the AMO Statistics (AMO frontend + backend, BigQuery/ETL with Airflow, and some patches in Firefox)
  • created various dev/QA tools

Last but not least, I started and developed a collaboration with Mozilla’s Data Science team on a Machine Learning (security-oriented) project. This has been one of my best contributions to Mozilla to date.

What did I do over the last 12 months?

I built the “new” AMO blog. I played with Eleventy for the first time and wrote some PHP to tweak the WordPress backend for our needs. I also “hijacked” our addons-frontend project a bit to reuse some logic and components for the “enhanced add-on cards” used on the blog (screenshot below).

The AMO blog with an add-on card for the “OneTab” extension. The “Add to Firefox” button is dynamic like the install buttons on the main AMO website.

Next, I co-specified and implemented a Firefox feature to detect search hijacking at scale. After that, I started to work on some of the new Manifest V3 APIs in Firefox and eventually joined the WebExtensions team full-time.

I also…

This period has been complicated, though. The manager who hired me and helped me grow as an engineer left last year 😭 Some other folks I used to work with are no longer at Mozilla either. That got me thinking about my career and my recent choices. I firmly believe that moving to the WebExtensions team was the right call. Yet, most of the key people who could have helped me grow further are gone. Building trustful relationships takes time and so does having opportunities to demonstrate capabilities.

Sure, I still have plenty of things to learn in my current role but I hardly see what the next milestone will be in my career at the moment. That being said, I love what I am doing at Mozilla and my team is fabulous ❤️

Thank you to everyone in the Add-ons team as well as to all the folks I had the pleasure to work with so far!

Hacks.Mozilla.Org — Common Voice dataset tops 20,000 hours

The latest Common Voice dataset, released today, has achieved a major milestone: More than 20,000 hours of open-source speech data that anyone, anywhere can use. The dataset has nearly doubled in the past year.

Why should you care about Common Voice?

  • Do you have to change your accent to be understood by a virtual assistant? 
  • Are you worried that so many voice-operated devices are collecting your voice data for proprietary Big Tech datasets?
  • Are automatic subtitles unavailable for you in your language?

Automatic Speech Recognition plays an important role in the way we can access information, however, of the 7,000 languages spoken globally today only a handful are supported by most products.

Mozilla’s Common Voice seeks to change the language technology ecosystem by supporting communities to collect voice data for the creation of voice-enabled applications for their own languages. 

Common Voice Dataset Release 

This release wouldn’t be possible without our contributors — from voice donations to initiating their language in our project, to opening new opportunities for people to build voice technology tools that can support every language spoken across the world.

Access the dataset: https://commonvoice.mozilla.org/datasets

Access the metadata: https://github.com/common-voice/cv-dataset 

Highlights from the latest dataset:

  • The new release also features six new languages: Tigre, Taiwanese (Minnan), Meadow Mari, Bengali, Toki Pona and Cantonese.
  • Twenty-seven languages now have at least 100 hours of speech data. They include Bengali, Thai, Basque, and Frisian.
  • Nine languages now have at least 500 hours of speech data. They include Kinyarwanda (2,383 hours), Catalan (2,045 hours), and Swahili (719 hours).
  • Nine languages now all have at least 45% of their gender tags as female. They include Marathi, Dhivehi, and Luganda.
  • The Catalan community fueled major growth. The Catalan community’s Project AINA — a collaboration between Barcelona Supercomputing Center and the Catalan Government — mobilized Catalan speakers to contribute to Common Voice. 
  • Supporting community participation in decision making yet. The Common Voice language Rep Cohort has contributed feedback and learnings about optimal sentence collection, the inclusion of language variants, and more. 

 Create with the Dataset 

How will you create with the Common Voice Dataset?

Take some inspiration from technologists who are creating conversational chatbots, spoken language identifiers, research papers and virtual assistants with the Common Voice Dataset by watching this talk: 

https://mozilla.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=6492f3ae-3a0d-4363-99f6-adc00111b706 

Share with us how you are using the dataset on social media using #CommonVoice or sharing on our Community discourse. 

 

The post Common Voice dataset tops 20,000 hours appeared first on Mozilla Hacks - the Web developer blog.

Planet Mozilla

Collected here are the most recent blog posts from all over the Mozilla community. The content here is unfiltered and uncensored, and represents the views of individual community members. Individual posts are owned by their authors -- see original source for licensing information.

Subscribe to Planet

Feeds:

Subscription list:

Last update:

Other Planets

Subscriptions