Nightly Tester Tools Resurrection

Some of you may have noticed that the aptly named Nightly Tester Tools addon doesn’t work on nightly builds at the moment. A few months ago Dave Townsend mentioned that the popular Nightly Tester Tools extension was in need of some new owners and an update. A couple of people that work on automation have decided to lead an effort to revive Nightly Tester Tools and make it even better.

You can check out the list of current and proposed features on the wiki. We’re planning on removing a couple features, some of which have been superseded by Firefox features, including: crash report sidebar, session restore, and leak log analysis. Let us know if you use any of these features. In addition, we’re thinking about taking out the extension compatibility fixing. Extension compatibility is the main(/only) reason many people use Nightly Tester Tools, but now there’s the Addon Compatibility Reporter that keeps up to date with the extension compatibility changes (EDIT: many people have protested this and given some good reasoning, so it’s likely this will stay in).

A couple people already working on the effort are harth, jhammel, and aakashd on #qa on irc.mozilla.org. If you have any features you’d like to see in NTT you can of course comment on this blog post, or file a bug in the Nightly Tester Tools component.

If you’d like to contribute code, fork the Github repo. We’re sticking with the old code base for now but are keeping an eye on the Jetpack SDK for a switch at some point.

mozregression update

A good while ago I made a regression range finder for Firefox nightlies. It was a Python script intended to make it easier for people to manually find the regression ranges of bugs on the nightlies. The other day, someone actually used it! So I decided to revisit it and fix it up and make it easier to install. There’s more info about it here, but here’s a quick summary:

install with setuptools or pip if you know how to do that, otherwise checkout out the OS-specific installation instructions

run on the command line:

mozregression --good=2010-03-19 --bad=2010-09-12

Several nightlies will be downloaded and automatically run for you, asking you to check for the existence of the bug in each one, ending up with something like:

Last good nightly: 2010-09-08 First bad nightly: 2010-09-09

Pushlog: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=36f5cf6b2d42&tochange=8e0fce7d5b49

if you have questions, ask harth on #qa at irc.mozilla.org.

Jetpack: Showing Search Terms in Awesomebar

Note: this is not about searching from the awesomebar.


download addon·code on github

The awesomebar is one of my favorite things about Firefox. Especially compared to alternatives in other browsers – the awesomebar remains the fastest way for me to find where I want to go. Sometimes a site’s url and title aren’t enough to jog my memory however. At some point I wished Firefox would do full-text indexing for me. Then I realized that Google was already doing that, and there might be a way to hijack that power – by remembering what search terms you used to find the site and displaying that in the awesomebar. You can’t really get better than search terms, they put the value a site gives in your own words. Of course, you might find the site valuable for other things after viewing it, then you have no real choice but to manually tag it (or do you?).

I made a quick extension with Jetpack to do just that, implemented in the simplest possible way. It parses the referrer for ‘q=’, and appends that to the page’s user-set title (which is matched on in the awesomebar). It can be very helpful, but it was less helpful than I thought. It would be more helpful if the referrer persisted across link navigation, the referrer didn’t terminate after #, and more sites used ‘q’ to hold the query, but it was an interesting experiment, and I’d love to see more experiments in Firefox inferring tags for webpages.

Jetpack impressions

I enjoyed using the new Jetpack SDK, I’m a big fan. It took more than a couple minutes to get started but the docs were excellent and I love the CommonJS. I had to bust out the Cc but that’s just because the Places API isn’t quite ready yet. I only had to write 42 lines of code. I can confidently say I won’t be making a ‘regular’ addon again.

Black or White Text? WCAG vs. Neural Networks

The other day I ran into a problem that some developers of color tools probably run into, which is: given a color, do you display white or black text over it? An algorithm based on the WCAG luminosity contrast ratio measure works great on most colors, but can be spotty. After looking at the WCAG formula, it seemed this problem was a good candidate for neural networks.

A couple weeks ago I finally sat down to try out the idea, and made a page where you can train your own neural network when to use black vs. white text (bonus: Worker thread example). After training it on a dozen or so colors, it matches the WCAG algorithm in most cases and does better in others. It’s kind of crazy that you can just throw data at this network and it will learn it. Not every problem can be solved by a neural network, but it’s cool to see it work when the problem fits. Might try out some other classifiers next.

I ended up making a simple JavaScript neural network implementation, you can check out the code on github.

Getting the Color Scheme of a Website Using Canvas and Hierarchical Clustering

Update: brief update, I think there are better (faster) ways to do this. Check color quantization if you don’t care about preserving the exact color values.

For a Firefox addon I’m working on I wanted to grab the color scheme of whatever website the user is viewing. There are a few extensions that do something like this: Colorzilla and Palette Grabber get the colors used by the website’s DOM, and ColorSuckr gets the most common colors from an image. The problem with getting the colors from the DOM is that websites use images and gradients so sometimes you can’t get the overall color scheme just from looking at the CSS colors.

Luckily, you can capture the color of every pixel on a webpage from an extension using the HTML 5 standard canvas. You can draw any image onto a canvas and get an array of pixel values used by the image with the getImageData function. In Firefox, you can actually draw the entire webpage onto a canvas using the drawWindow function (drawWindow is a Firefox-only canvas extension, but you can at least use drawImage in other browsers).

So getting the number of occurrences of each color is as simple as drawing the page to a canvas, looping through each pixel value in the array returned by getImageData, and tallying each color’s frequency in a javascript hash. This is what you get when performing this analysis on the Twitter homepage:

So you can get the colors that occurred most on the page pretty easily, there is however, one big problem with this. On images and gradients, there are areas of very similar colors that might as well be the same color in the context of the overall theme. As I’ve found out, there are usually over 10,000 different colors on each webpage, so these colors need to be grouped together.

This kind of problem is called clustering, and it’s a problem that comes up a lot in image analysis, computational biology, and other computational disciplines. There are two common clustering algorithms, k-means and hierarchical clustering. K-means can be faster than hierarchical clustering, but the problem with k-means is that you have to know what k is before you even start — you have to know exactly how many clusters you want to end up with. That can’t be determined in this situation, so hierarchical clustering is the best bet.

The premise of hierarchical clustering is simple. Each color starts out in its own cluster. On each pass of the algorithm, the two clusters that are most similar (according to a metric you define yourself) are merged. You keep on doing this until there are no more clusters that are similar enough to merge.

I defined the distance of two colors to be the maximum difference between the two colors’ red, green, and blue components. Two colors were ‘similar enough’ if their distance was less than 12 (where each color component ranged from 0 to 255). When two clusters were merged, the new cluster’s representative color was a weighted average of the two clusters’ representative colors. The algorithm worked pretty well for this application, check out the results:

The algorithm takes a long time even if you just focus on the top few hundred colors (a few seconds), but that’s what Web Workers are for, after all. You can check out the code here. Do you know a faster clustering algorithm? How would you quantify the distance between two colors?

Update: after getting some great feedback, I refined the distance measure to be the 3d distance between the two colors in the L*a*b* color space (the CIE76 distance measure) , thanks to the developer of FireColour for the open source L*a*b* conversion code.