Testing

Pass all the Breakpoints

 

When modifying a system without adequate tests, I found it helps to include debugger breakpoints in my manual testing checklist:

For each change I make, I add a breakpoint after the modified line, and won’t remove it until the interpreter has passed through it exercising the logic to my satisfaction. Before committing the changes I can pull up the list of breakpoints to make sure I’ve hit them all.

This is probably most valuable when I’ve made a lot of changes at once, or if temporary delusion is making me think I don’t need to test everything.

Caveat 1: I suppose, if there’s any parallel operations, you should also test everything with the breakpoints gone, lest the code relies on losing a race condition.

Caveat 2: In no way is this a substitute for automated tests!

What, when, and how is your IDE telling you?

 

A programmers.stackexchange question basically asks if there’s an ideal frequency to compile your code. Surely not, as it completely depends on what kind of work you’re doing and how much focus you need at any given moment; breaking to ask for feedback is not necessarily a good idea if you’re plan is solid and you’re in the zone.

But a related topic is more interesting to me: What’s the ideal form of automated information flow from IDE to programmer?

IDEs can now potentially compile/lint/run static analysis on your code with every keystroke. I’m reminded of that when writing new Java code in NetBeans. “OMG the method you started writing two seconds ago doesn’t have the correct return type!!!” You don’t say. And I’ve used an IDE that dumped a huge distracting compiler message in the middle of the code due to a simple syntax error that I could’ve easily dealt with a bit later. I vaguely remember one where the feedback interfered with my typing!

So on one side of the spectrum the IDE could be needling you, dragging you out of the zone, but you do want to know about real problems at some point. Maybe the ideal notification model is progressive, starting out very subtle then becoming more obvious as time passes or it becomes clear you’re moving away from that line.

Anyone seen any unique work in this area?

Stepping back to the notion of when to stop and see if your program works, I think the trifecta of dependency injection, sufficiently strong typing, and solid IDE static analysis has really made a huge difference in this for me. Assuming my design makes sense, I find the bugs tend to be caught in the IDE so that programs are more solid by the time I get to the slower—and more disruptive to flow—cycle of manual testing. YMMV!

Validate Private Page Bookmarklet

 

ValidatePrivatePage <– validates in current window

ValidatePrivatePage <– validates in new window (your pop-up blocker may complain)

If you need to validate the markup of a page that’s not public (e.g. on localhost), you can now use this bookmarklet to auto-submit the current page source to the validator (instead of viewing source, copying, opening the validator, pasting in, and pressing “check”).

Note: this gets the page source making an XMLHTTPRequest to the current URL, so it does not get interpreted by the browser; i.e. this is NOT based on innerHTML(). If the request made returns a different page (e.g. you were logged out in the meantime), that page’s source will be sent to the validator. Not much can be done about that. I once wrote a crusty PHP4 class/bookmarklet combo that helped do this, but thanks to the standardization of XMLHTTPRequest, this is easy in JS now. You should also thank W3C for allowing cross-domain POSTs to the validator :)

Perks of Life in the Kingdom of Nouns

 

Execution in the Kingdom of Nouns is one of Steve Yegge’s most entertaining posts about the verbosity and noun-centricity of Java (and other strongly typed languages without first-class functions). His post paints a picture of life outside JavaLand, where things are simpler and more straightforward:

There’s rarely any need in these kingdoms to create wrapper nouns to swaddle the verbs. They don’t have GarbageDisposalStrategy nouns, nor GarbageDisposalDestinationLocator nouns for finding your way to the garage, nor PostGarbageActionCallback nouns for putting you back on your couch. They just write the verbs to operate on the nouns lying around, and then have a master verb, take_out_garbage(), that springs the subtasks to action in just the right order.

With simplicity, there’s always a tradeoff. How does one test that take_out_garbage() actually works? You’ll need a full environment with real nouns lying around and all you can do is check the environment after completion. In the Kingdom of Nouns, you can pass in mock nouns and carefully watch what the function does with them. Maybe it’s important that the recycling gets separated, but if you only check the end environment, you won’t be able to tell it wasn’t.

What if trash day is moved to a different day? You might need a new take_out_garbage_wednesday(). As ridiculous as it seems, passing in a GarbageDestination makes the chore more flexible and testable. During testing I can pass in a destination of “right here”, saving me the hassle of creating an environment that includes the walkway out to the curb.

Whether you’re executing global/static functions or class methods, being able to pass dependencies in (where to find the trash, where it goes, when is it supposed to be there) not only makes your function more flexible, but more testable to boot.

This is not to suggest that Steve’s post was anti-dependency injection. I get the impression it was more a frustration of the expressive limitations of Java (when he wrote it—I think it has lambdas now). You can do dependency injection just fine without concrete classes, of course, but you wouldn’t want someone to pass in an array of GPS coordinates when you really need an object that you could call getStreetAddress() on. Type hinting an AddressProvider in the argument would let end users know exactly what to pass in and let the compiler/runtime know not to even bother executing the function, because we don’t have what we need. Also type hinted arguments allow good IDEs to prevent you from making mistakes, or to autocomplete existing variables in scope matching that type.