Thoughts on a Javascript “validator”

(X)HTML and CSS have their own validators, but we need one for Javascript as well. JSLint could be part of it, but what I’m imagining would flag the use of “native” objects/interfaces not defined by W3C or ECMA standards. E.g., document.layers or window.ActiveXObject.

The hard part in doing this is finding a full Javascript interpreter with W3C DOM interfaces, but without the proprietary features that browsers have to support to remain usable on the web. A couple ways come to mind:

1. Rhino

John Resig has just implemented “window” for the Javascript interpreter Rhino, which, supposedly, is already based strictly on the ECMAscript standards. The short term goal of his script was to give Rhino enough of a browser interface to pass jQuery’s test suite, but the code could be branched to implement all current W3C DOM recommendations (and none of the proprietary interfaces). A Java app would set up Rhino, run John’s script, then run your script and simply report any errors.

2. A stricter browser

Most modern browsers are pretty malleable as far as letting user code clobber native functions/objects; in fact it’s a real problem. But this is convenient if you want to, say, overwrite all the proprietary features with your own code! With a whitelist in hand, you could use for-in loops to sniff out non-standard functions/properties and either delete or overwrite them with functions of your choice.

The advantage of the browser solution is that you could still provide the non-standard features (so the script would continue to run) while logging them. Using closures you could, in theory, “wrap” each non-standard function so that each of its calls would also call your logging function before returning the expected value. A crude example:

// may fail spectacularly
var _parseInt = window.parseInt;
window.parseInt = function (str) {
    alert('parseInt was called.');
    return _parseInt(str);
};
parseInt('2');

In practice, you’ll be at the mercy of the browser to allow native functions to be stored in variables like this. Sometimes it works, sometimes not. As for properties, you may be able to similarly log their usage with Javascript’s relatively new getters and setters.

So any idea where to get such a whitelist?

Hacking a 3rd party script for bookmarklet fun

A few weeks ago I created a simple bookmarklet that loads del.icio.us’s PlayTagger script into the current page. This post covers how some problems with this script were worked through.

Too late

The first challenge was that PlayTagger was designed to initialize itself (let’s call this method “init“) on window.onload: If a user fired the bookmarklet after window.onload (99% of the time), playtagger.js would load but init would’ve missed its chance to be called. This means I had to call init manually, but since script elements load asynchronously, I had to wait until init actually existed in the global scope to call it. This was fairly easily accomplished by attaching my code to the new script element’s “load” event (and using some proprietary “readyState” junk for IE).

Too early

If the page takes a long time to load, it’s possible the user will fire the bookmarklet before window.onload. One of two things will occur:

If it’s fired before the DOM is even “ready”, the bookmarklet throws an error when it tries to append the script element. I could use one of the standard “DOMready” routines to run the bookmarklet code a little later, but this case is rare enough to be not worth the effort to support; by the time the user can see there are mp3s on the page, the DOM is usually ready.

Assuming the DOM is ready, playtagger.js gets loaded via a new script element, the bookmarklet fires init, but then, thanks to playtagger’s built-in event attachment, init is called a second time on window.onload, producing a second “play” button per mp3 link. Harmless, but not good enough.

Preventing the 2nd init call

It would be nice if you could sniff whether or not window.onload has fired, but this doesn’t seem to be possible. Maybe via IE junk. Any ideas for a standards based way to tell?

My only hope seemed to be to somehow disable init after manually calling it. The first try was to just redefine init to a null function after calling it:

init();
init = function () {};

I figured out that redefining init would not help here due to the way it’s attached to window.onload:

// simplified
var addLoadEvent = function(f) {
    var old = window.onload;
    window.onload = function() {
        if (old) { old(); }
        f();
    };
};
addLoadEvent(init);

What’s important to notice here is that init is passed to addLoadEvent as f and window.onload is redefined as a new function, capturing f in the closure. So now f holds init‘s original code (because functions are first-class in Javascript), and f, not the global init, is what is really executed at window.onload. As f is private (hidden by the closure), I can’t overwrite it.

Disabling init from the inside by “breaking” Javascript

The second thing I tried was to break init‘s code from the inside. The first thing init does is loop over the NodeList returned by document.getElementsByTagName('a'), so if I could get that function to return an empty array, that would kill init‘s functionality. Because Javascript is brilliantly flexible I can do just that:

// cache for safe keeping
document.gebtn_ = document.getElementsByTagName;
// "break" the native function
document.getElementsByTagName = function(tag) {
    if (tag != 'a') return document.gebtn_(a);

    // called with 'a' (probably from init)
    // "repair" this function for future use
    document.getElementsByTagName = document.gebtn_;
    // return init-busting empty array
    return [];
};

Simplest solution

While the code above works pretty well, I thought of a simpler, more elegant solution: just rewrite window.onload to what it was before playtagger.js was loaded.

And with that here is the final unpacked bookmarklet code:

javascript:(function () {
    if (window.Delicious && (Delicious.Mp3 || window.Mp3))
        return;
    var d = document
        ,s = d.createElement('script')
        ,wo = window.onload
        ,go = function () {
            Delicious.Mp3.go();
            window.onload = wo || null;
        }
    ;
    s.src = 'http://images.del.icio.us/static/js/playtagger.js';
    if (null === s.onreadystatechange) 
        s.onreadystatechange = function () {
            if (s.readyState == 'complete')
                go();
        };
    else 
        s.onload = go;
    d.body.appendChild(s);
})();

Kill these DOM0 shortcuts

A problem a decade in the making

You can refer to a form’s elements in your code by using the element’s name (from the NAME attribute)
— an ancient Javascript spec.

This means myForm.myElement is a shortcut for myForm.elements['myElement']. I’m sure this was seen as handy and harmless at the time, but the problem is that references to form elements, simply by using valid name attributes, can overwrite important DOM properties and methods on the form element. Make a text input with name=”submit” and you overwrite the form’s native submit method; no more form submission!

Workaround

Effectively, you have to avoid using names that already exist as properties or methods of the form DOM element, from modern specs back to the stone age. To get an idea of how many there are, I created an empty form element with no attributes, fired up the Javascript shell bookmarklet and used its handy props function:

Methods: addEventListener, addRepetitionBlock, addRepetitionBlockByIndex, appendChild, attachEvent, blur, checkValidity, cloneNode, contains, detachEvent, dispatchEvent, dispatchFormChange, dispatchFormInput, focus, getAttribute, getAttributeNS, getAttributeNode, getAttributeNodeNS, getElementsByTagName, getElementsByTagNameNS, getFeature, hasAttribute, hasAttributeNS, hasAttributes, hasChildNodes, insertAdjacentElement, insertAdjacentHTML, insertAdjacentText, insertBefore, isDefaultNamespace, isSupported, item, lookupNamespaceURI, lookupPrefix, moveRepetitionBlock, namedItem, normalize, removeAttribute, removeAttributeNS, removeAttributeNode, removeChild, removeEventListener, removeNode, removeRepetitionBlock, replaceChild, reset, resetFromData, scrollIntoView, selectNodes, selectSingleNode, setAttribute, setAttributeNS, setAttributeNode, setAttributeNodeNS, submit, toString

Fields: accept, acceptCharset, action, all, attributes, childNodes, children, className, clientHeight, clientLeft, clientTop, clientWidth, contentEditable, currentStyle, data, dir, document, elements, encoding, enctype, firstChild, id, innerHTML, innerText, isContentEditable, lang, lastChild, length, localName, method, name, namespaceURI, nextSibling, nodeName, nodeType, nodeValue, offsetHeight, offsetLeft, offsetParent, offsetTop, offsetWidth, onblur, onclick, ondblclick, onfocus, onkeydown, onkeypress, onkeyup, onload, onmousedown, onmousemove, onmouseout, onmouseover, onmouseup, onunload, outerHTML, outerText, ownerDocument, parentElement, parentNode, prefix, previousSibling, repeatMax, repeatMin, repeatStart, repetitionBlocks, repetitionIndex, repetitionTemplate, repetitionType, replace, scrollHeight, scrollLeft, scrollTop, scrollWidth, sourceIndex, style, tagName, target, templateElements, text, textContent, title, unselectable

These are just in Opera; to be really safe you need to add in any extras from all the other modern browsers.

But there’s more

This problem also exists on the document object: Give your form name="getElementById", then try using document.getElementById(). Even worse, IE/win registers all element ids onto the global (window) object.

The real solution

The W3C/WhatWG needs to step up and formally denounce these shortcuts in the HTML DOM specs, and, IMO, also generate warnings in the HTML validator where input names match those in W3C DOM specs, at the very least. Since there’s a sea of legacy code out there using these shortcuts, browser makers desperately need a defined way to handle these collisions.

Reload CSS Every… Bookmarklet

One annoying aspect of writing a stylesheet is having to reload the page in your browser to see your incremental changes. There are ways to workaround this, but I find them less than ideal so I created ReloadCSSEvery. It refreshes only the CSS of a loaded page every 2 seconds so you can use your CSS editor of choice and see semi-real-time changes in your browser when you save.
Continue reading  

widget--

17 Dec 2006: Latest Del.icio.us Posts now works again in the latest release 9.02. Huzzah.

My first Opera widget created back in July is now broken in Opera 9.02. The debugging and rebuilding process is so tedious that I probably won’t fix it very soon. A nice gesture on Opera’s part is the creation of an actual Widget API/specification for those in the trenches, but where can you find it? Nowhere on opera.com. I had to google “Opera Widget Object” to find it tucked away apparently as a left-over part of a presentation given at a conference in Finland. If you want developers to use your technology, spending an hour to write up even some preliminary docs would be nice.

After giving them a real chance, I still say widgets are a flop (at least on the desktop). Beyond the persistent storage, a widget is just a website and having it run outside the browser is a neat trick, but one that already failed. And as for porting widgets to mobile devices, I forsee a lot of problems in the area of performance and memory leaks since most widgets don’t go out of their way to preserve bandwidth and none share Javascript libraries. Leave a half-dozen web2.0 sites open in any browser all day and watch the memory usage. Will a mobile handle it?

Could Expression break the tedium of CSS design?

I’ve been dealing with CSS layout quirks and bugs for seven years and, frankly, by now the thrill of designing an elegant style sheet by hand has worn off. The process of choosing the right lengths and font-sizes, and setting all these properties by hand (even with autocomplete) is just unacceptably tedious and just bogs you down. The “CSS panels” various apps have just don’t save enough time; I can almost always edit the rules by hand faster than scrolling through lists of properties and clicking in value boxes. Yes, a 2nd monitor auto-refreshing the page would help, but I’m still guessing at values, saving, repeating endlessly.

I just want to drag and drop elements into place, maybe in a DOM tree view in a side panel, then zoom in and adjust layout properties like margin and padding by dragging handles or using the mousewheel, same with font-sizes in %/ems. Even if it stores these properties in inline style attributes, this would be a huge time-saver. Just also give me a wizard to take all inline styles and merge them into my external files intelligently, maybe let me pick the ids or choose when to use various contextual selectors.

It looks like Microsoft’s Expression designer has some of these features. Dreamweaver has a nice page zoom and yeah, you can stretch tables around like it’s 1996, but it doesn’t cut it. Right now DW8 is my primary editor, but as I do less front-end work I can forsee it losing ground to more capable PHP editors (if they could just match DW’s site management tools).

Why CSS layout is hard

The complaints are many and oft-repeated: “CSS has no grid system”, “the cascade is dumb”, “it’s broken”, etc. So why is CSS layout so hard? The obvious answer is that doing anything really well isn’t always easy. Most anyone can make a page in a modern WYSIWG editor (DW8/GoLiveCS2) in five minutes and have an accessible web page that nearly browser can read, and it can even look good–in that area we’ve come a long way–but it won’t necessarily look professional. There’s always a point in any field where your goals outmatch your abilities and I don’t see why CSS should be expected to be any different.

As for specifics, plain old CSS2 (1998!) can accomplish just about any layout you can dream up, but standing in your way more than anything else is the IE Factor. Even the upcoming version 7, which we’ll be supporting for a long time to come, will not support the table-* values for display that make multi-column layout easy as cake. Is it the CSS working group’s fault that a browser fails to implement standards for almost a decade? If existing standards aren’t supported, why should we believe alternatives would be/would’ve been? The sad part in the case of display is that IE also cannot change the display properties of table elements (the color chart is a table as it should be).

I’ve seen the point made that the cascade is great for text-styling, but terrible for layout. This person probably didn’t notice that the values of layout properties like display, position and float don’t actually cascade (aren’t inherited from the parent element). Oops.

Now, to give credit to the critics, there are a couple areas where I see CSS2 partucularly fails to deliver: 1) floated elements must precede elements that “wrap around” them in the markup. This is not usually a big issue since markup is easy to control, but at least one CSS3 draft addresses this. 2) the table-* display values mentioned before cannot duplicate table layouts that use rowspan/colspan. Again, the WG has put work into it, but don’t expect to see this anytime soon. Then again, non-IE browsers continuously forge ahead, supporting various bits of CSS3 like selectors, multiple backgrounds, rounded corners, opacity, speech, multi-columnal layout…

Opera folks: Make it easy to test in your browser

Opera wants to know what features web developers would most like to see. I posted this in response (but it never seemed to show up on the page):

“Testing” version: offer devs a download pre-configured for more easily testing site compatibility with Opera. Marketshare will only rise if more sites are tested in Opera, so this has to be dead-easy for devs used to testing in IE/FF.

Strict-mode JS Notices: tell devs when they’ve attempted to use a common interface that IE/FF supports but Opera doesn’t. Link to a knowledgebase article, which leads me to…

Documentation: Be public and upfront about lack of support for popular proprietary features that IE/FF support and that web devs are itching to use, and list practical alternatives and workarounds in knowledgebase articles. In fact, publish usable compatibility scripts if possible, not just secret browser.js fixes.

In the 7.5 days this would’ve been XMLHTTPRequest. Yes, Opera devs were working on it, but web devs were in the cold for knowledge of the current level of support (eg. you couldn’t POST until 7.6 so many sites half worked). If support is in the works, let us know how it’s coming along! Not just in an odd hint drop on the forums.

This also goes for Opera-specific tech; I shouldn’t have to figure out myself that UserJS doesn’t fire in panels (why not?) or search forums for the most basic widget methods. Devs need thorough docs. Take the PHP docs as a model.

JS Panel: I sorely miss not being able to watch JS errors in real-time in a panel, as I did with v8. In fact, it should be a top-notch panel developed by Opera that ships with the browser.

DOM Inspector: devs love them, plenty of bookmarklet versions exist out there. Grab the best, improve the hell out of it and ship it in a panel or publish it as an official Opera bookmarklet. Grab one of your widget developers for two days and you’ll have this.

widgets++

My first Opera widget is available. I designed it to take up as little desktop space as possible (a 22x22px icon) until you click it, which opens the interface and queries del.icio.us to show your latest bookmarks. By default, when you click a bookmark, the page opens in Opera and the widget “minimizes” back to the icon. I’ve followed Opera’s guidelines in using their standard “flip” button for preferences, and these persist after you close and reopen the widget.

For people who browse full screen, I’m still not sold on the widget concept, or at least Opera’s implementation, which is only slightly less limiting in functionality as a web page. The only big advantage widgets offer to, say, bookmarklets (since BM can open windows), is the persistent storage, which is thankfully larger than the cookie 4K limit. The testing and build process needs improvement; you have to delete the previous version in Opera, clear the cache, make a zip of the folder, serve it with a special content-type and reinstall in Opera. Easier testing can be done by including Javascript that emulates the widget API if it’s not available, but this still won’t allow cross-domain XHR requests, so for some widgets your workflow has to be tweak-rebuild-retest-repeat.

Opera 9 thoughts

Opera 8’s UI really had most everything it needed to be a great browser (I haven’t upgraded at work and can barely tell the difference). 9’s big delivery is in the area of web standards (opacity, SVG, DOM Style) and hot proprietary ones like rich text editing, Flash-Javascript communication and the Canvas element. There are additions to write home about on the UI side as well (site-specific settings, content blocking), but the under-the-hood stuff above is what will truly allow a new Opera user to use the latest web sites they’re already used to without getting a broken/dumbed down interface or being flat out blocked.

The new widgets feature is surely meant as an answer to (or at least a distraction from) Firefox Extensions, but I think this move will only make it more apparent that people really want to customize their browsing experience rather than collect desktop gizmos. Even with all the headaches associated with dealing with managing Extensions (and upgrades)–and I’ve dealt with them from the Mozilla 0.9 days through Pheonix and now Firefox)–the modifications they can perform are staggering and impossible to ignore. Although extensions give you the power to wreck/destabilize your browser and make upgrading a pain, users now expect that power.

A couple things I’d personally like to see in Opera is a richer panel implementation and maybe some advanced Javascript extensions. Mozilla/Netscape7’s “sidebar” could’ve used a better control UI, but it allowed multiple panels to be visible simultaneously. I think if Opera encouraged web developers to develop panels with the same gusto as they promote widgets, they might prove more useful just because they live inside the browser where the user lives most of the time anyway! Maybe I need to create a slick panel as a proof of concept. In the Javascript area I’m thinking some extra interfaces available to bookmarklets (better persistant storage than document.cookie, local storage of JS libraries) could help make up for the lack of extensions. Just some ideas…