Author: Steve

You Probably Don’t Need ETag

 

(updated 3/4 to include the “Serving from clusters” case)

As I see more server scripts implementing conditional GET (a good thing), I also see the tendency to use a hash of the content for the ETag header value. While this doesn’t break anything, this often needlessly reduces performance of the system.

ETag is often misunderstood to function as a cache key. I.e., if two URLs give the same ETag, the browser could use the same cache entry for both. This is not the case. ETag is a cache key for a given URL. Think of the cache key as (URL + ETag). Both must match for the client to be able to create conditional GET requests.

What follows is that, if you have a unique URL and can send a Last-Modified header (e.g. based on mtime), you don’t need ETag at all. The older HTTP/1.0 Last-Modified/If-Modified-Since mechanism works just fine for implementing conditional GETs and will save you a bit of bandwidth. Opening and hashing content to create or validate an ETag is just a waste of resources and bandwidth.

When you actually need ETag

There are only a few situations where Last-Modified won’t suffice.

Multiple versions of a single URL

Let’s say a page outputs different content for logged in users, and you want to allow conditional GETs for each version. In this case, ETag needs to change with auth status, and, in fact, you should assume different users might share a browser, so you’d want to embed something user-specific in the ETag as well. E.g., ETag = mtime + userId.

In the case above, make sure to mark private pages with “private” in the Cache-Control header, so any user-specific content will not be kept in shared proxy caches.

No modification time available

If there’s no way to get (or guess) a Last-Modified time, you’ll have use ETag if you want to allow conditional GETs at all. You can generate it by hashing the content (or using any function that changes when the content changes).

Serving from clusters

If you serve files from multiple servers, it’s possible that file timestamps could differ, causing Last-Modified dates sent out to shift and needless 200 responses when a client hits a different server. Basically, if you can’t trust your mtime to stay synched (I don’t know how often this is an issue), it may be better to place a hash of the content in an ETag.

In any case using ETag, when handling a conditional GET request (which may contain multiple ETag values in the If-None-Match header), it’s not sufficient to return the 304 status code; you must include the particular ETag for the content you want used. Most software I’ve seen at least gets this right.

I got this wrong, too.

While writing this article I realized my own PHP conditional GET class used in Minify, has no way to disable unnecessary ETags (when the last modified time is known).

Safari Cache-Control:must-revalidate bug

 

Update Apr 8 2009: Apparently this bug existed in previous Safari versions (at least back to 3.1), i.e. including “must-revalidate” in Cache-Control means Expire and max-age will both be ignored by Safari. Here’s the Apple bug, if you happen to be an employee. I created an Apple account but still couldn’t find it.

Short version: Safari 4 beta incorrectly interprets the Cache-Control “must-revalidate” directive and re-requests the file each time despite freshness info sent via “max-age”.

Long version

When a server sends Cache-Control with “must-revalidate”, this tells clients/caches that, after it expires, the cached resource should not be used without first checking with the server (sending a conditional GET). From the spec (my emphasis):

When the must-revalidate directive is present in a response received by a cache, that cache MUST NOT use the entry after it becomes stale to respond to a subsequent request without first revalidating it with the origin server. [HTTP/1.1]

In other words, while the cache is fresh, there’s no need to revalidate.

E.g. I serve this Javascript file with the header “Cache-Control: public, max-age=1800, must-revalidate”. This tells clients: “consider this fresh for 30 minutes, then check back before using it again”. Unless you force a refresh, the browser won’t re-request the file for 30 minutes, after which it will send a conditional GET request. If the file hasn’t changed, the servers returns a “304 Not Modified” and the same Cache-Control header. This tells the browser to keep using the cache for another half hour and check back again. (Yes, half an hour is too short, I’ll change this later.)

Well, Safari 4 beta re-requests the file each time it needs it, seemingly ignoring the “max-age” directive. At least it’s sending a conditional GET, but it’s still a waste of bandwidth and time.

Plan Juarez

 

If you haven’t noticed, Juarez and many other Mexican cities are facing a violent crisis. The drug cartels are so well funded and armed that they can bribe officials and even threaten police chiefs into retirement. The drugs moved by the traffickers can, of course, cause harm, but Mexico’s problem isn’t a “drug problem” but rather a prohibition problem.

The relatively rich people of U.S. and Canada create a huge demand for drugs, both government regulated and illegal. By waging war on some drugs, we outsource much of our illegal drug production to South America. Mexico has traditionally given lip service to our prohibition, but not devoted much to the effort in practice, forcing the U.S. deal with the problem on our side of the border and in Colombia. This allowed the cartels to work in relative peace for many years, but this was still unacceptable to former president Vicente Fox. Fox battled the cartels and managed to unseat a few kingpins, which only led to more violence. When Calderón replaced Fox in 2006, he declared his own war on the cartels, calling in federal troops to replace the police and greatly escalating the war.

U.S. officials and drug warriors, of course, find the violence acceptable. “There will be more violence, more blood, and, yes, things will get worse before they get better. That’s the nature of the battle,” said U.S. Ambassador Tony Garza. (Would these people find it acceptable if thousands of U.S. citizens were dying and living in a police state to prohibit sales of goods to Canadians?)

A Solution for the People of Mexico

It is not your money, guns, or addictions that are feeding the violence in your country. I propose that you end the war on your fellow citizens.

  1. Create a license to legally produce and traffic “illegal” drugs strictly for export to the U.S.
  2. Allow licensees to freely operate, but with taxes and rules, while continuing to fight unlicensed organizations.
  3. Allow law-based disagreement resolution mechanisms and revoke licenses from organizations linked to violence.

In other words, you should make your narcotraficante industry look more like UPS and Budweiser than Al Capone’s Chicago Outfit.

Resistance to such a policy would come primarily from key players in the drug war. Your cartels will want to maintain control, but their profits will die as non-violent businessmen take up licenses and take their business. Our DEA may suggest closing your border or even replacing your government by force; we’ve done it before.

In the end you’ll have to decide what is in the best interest of your people. Is it worth their lives and the very stability of your government to prevent Michael Phelps from hitting a bong?

Sorry Jerks, Intimidation Won’t Change the Web

 

Shameful law firm Jones Day sued real estate news site BlockShopper claiming “trademark infringement” over the way the site used link text, but really because the firm didn’t like how the site reported the home purchases of their partners (which is public record). The case should’ve been thrown out, but the judge allowed it, and, in doing so, allowed the firm to bleed the site of $110K in legal fees. BlockShopper finally agreed to settle out of court and to format its links to the firm in a particular way.

…it signals to companies that they can force sites to revise their linking styles by alleging trademark infringement. And Judge Darrah’s decision not to dismiss the suit signals that Web publishers may have to spend significant sums to deal with this kind of litigation. [Slate]

Paul Levy noted the law firm is “a serial abuser of the trademark laws to suppress commentary that it does not like.”

Personally, I would’ve just used this URL to link to the bullies: http://google.com/search?btnI=1&q=Paul+W.+Schroeder+Professional+Biography E.g., “Paul W. Schroeder likes to strongarm web authors.” Or you could use a 3rd party redirect site like tr.im. In fact, there should be a redirector designed to shame site owners who attack linkers like this.

Update: We now have one: http://mrclay.org/warning/

E.g.: Paul W. Schroeder.

Zero Tolerance Nets Another Slimebag

 

What if we had zero tolerance for laws that have a lot of ugly side effects? Like busting an amazing teacher in front of her kids… for finding a couple Xanax pills… in a suspicion-less search of her car.

Herrick, 59, has taught at Roberts for 17 years. Parents describe her as “inspirational,” “talented” and “loving.”

She comes to work early so kids can finish projects they didn’t have time to complete in class. So many kids wanted to join her after-school art club that it had to be restricted to fifth-graders. More than one parent tells how she dropped by a student’s house, bearing art books that she thought might be of interest.

She’s a ferocious doubles tennis player, nationally ranked, so fanatical about her game that she hesitated a year before taking cholesterol meds that her doctor prescribed.

In 1995 and 1999, Herrick was Roberts’ teacher of the year. For 2005-06, she was teacher of the year for HISD’s entire Central District. And in 2009, she was busted.

In the middle of a class, police escorted her from her classroom. After she unlocked her car, police found a baggie with two Xanax pills.

Herrick said she has no idea how the pills got into her car, which other people in her family drive.

But no matter. She was hauled away from the school she loves in the back of a squad car and charged with possession of a controlled substance within 1,000 feet of a school. If convicted of that third-degree felony, she could serve two to 10 years in prison.

Even though she’s passed the drug test and a lie detector test, she’s out of the classroom for the time being. As Pete Guither writes, “It’s, you know, for the kids.”

What if she does have a Xanax problem? Having teachers like this lady, personal habits and all, is what builds kids that never need to try drugs. How about the experience of watching your favorite teacher taken to jail? or growing up a paramilitary police state?

Unstimulated

 

If this National Review article is accurate, the stimulus bill is worse than I was imagining. It’s not that I’m against every piece of it, but wrapping up countless unrelated projects under one bill and/or pushing it through Congress as an “emergency remedy” is a terrible way to create laws.

This is how we got the Patriot Act and the Commodity Futures Modernization Act of 2000.

Although the right is ablaze over this, I think I’m a little too cynical to believe that there’s a good/evil party boundary in this. I fear it’s in times like these that both parties will “reach across the aisle” and push through pet projects via a monumental bill to tell citizens they “did something about the recession”. It’s the minority party’s role to “oppose” and break out their own pens to fair it up.

If the Senate Reps oppose this with the same zeal, they’ll win some respect from me. If it passes, for years to come we’ll be discovering wonderful new treasures similar to the Enron loophole and unregulated credit default swaps, and plenty of junk both parties are opposed to. We’ve already found a revival of the Byrne grants frequently abused in our drug war.

I don’t see how any elected official can look us in the eye and say, “all this is what we need and we need it immediately, there’s no time to read the fine print.”

New general rule: the level of debate and time for press scrutiny of a bill should be proportional to its scope. Maybe there should even be some limitation of scope within a single bill.

Rational Debate Across the Pond: House of Lords Wants Harm Reduction

 

What if Congress debated the prohibition of drugs for over 2 hours, finding surprisingly that most members already favored harm reduction policies and, in some cases, regulation over criminalization?

This just happened in the U.K.’s House of Lords. On January 22nd, a debate was held (full transcript) to encourage the government to send a senior (rather than a junior) Minister to the U.N.’s upcoming conference on drug policy, and for that representative to push for harm reduction rather than blanket prohibition. (background).

The transcript shows a frank and open discussion of the negative effects of prohibition on many levels of society. At some point I’ll pull quotes, but basically members desired increased funds for treatment, expressed dismay that the U.N. treaties prevented countries from experimenting with alternative policies, recommended the U.N. officially recognize the difference between use and abuse of drugs, and agreed that the 40 year old policy has been a failure at reducing the use, and especially reducing the harms of drugs in the world. One mentioned that, even if the goal was to “sustain” the current level of drug abuse (as our Drug Czar frequently states), that that level of abuse and the societal price of prohibition is too high to continue to tolerate.

If the U.K. is not the only country that wants to get smarter rather than tougher on drugs, we may see some revolutionary reform discussion come out of the March meeting in Vienna.

Lying in your job description? You may be our Drug Czar.

 

The deeper you dig into the history of the Drug War the more craziness you uncover. You’d think an office in charge of drug policy on a national level would monitor science and policy outcomes and work to refine those policies over time, or in the very least not break laws spreading misinformation about drugs.

You’d be wrong! In 2003 Congressman Ron Paul found this out when he pointed out that it’s illegal for the ONDCP to propagandize untruths about marijuana, especially in attempt to lobby against legislation (typically for its medicinal use). The General Accounting Office’s response a year later? Sure it’s illegal, but technically not against state legislation! And since the drug czar’s job description includes doing “such actions as necessary” to oppose legalization attempts of any kind, why look into the misinformation claim at all.

If you’re curious to know what kind of lies come out of this office, I covered those of drug czar John Walters in detail on StopTheDrugWar.org. His writing is practically a guide for constructing logical fallacies.

Skate 2 First Impressions

 

The Bad

They tampered with the most important element of the game: the controls. The Flick It moves of Skate 1 all work identically, but the feel is definitely different. I spent countless hours perfecting a realistic style in S1, and even after 3 hours or so of S2 I still feel like I’m starting over.

The replay editor seems tremendously more complex and requires more buttons to get anything done. When you’re trying to film a line in S1 it’s nice to be able to jump in and review different angles immediately; the S2 editor requires you to enter a clip editor and go through another menu to select, adjust, and confirm every angle. Getting back to skating requires at least one or two more button presses than in S1.

The new default camera angle blocks your view less, but looks jerky and much less handheld. And why the “extreme” visual and sound effects whenever you make a big drop, get a ton of speed, or fall? This all makes the game look more cartoonish like the later THPS games, and gets tiring quick when you’re trying a line over and over.

This is a minor gripe, but the movable objects are squeeky clean and almost glowing, ruining an otherwise beautifully grimy environment. I don’t know why they have to stand out because there seems to be only a half-dozen different objects. Having to guess what isn’t bolted down would be a lot more fun.

The Good

It may be just the novelty, but new San Van seems enormous and like a real city that grew over many decades without city planning. Knowing the city in S1 pretty well, skating around is almost an eerie experience, like 50 years have passed and entire city blocks have been replaced. Sometimes you can see the seams where new and old coexist. Major streets dead end in new buildings, other areas have been left in blight with graffiti and handrails bent from skating. The sound of the city is nearly inescapable. If new San Van were the only new thing in S2 it would be worth it.

Returning to a session marker is usually instantaneous, and when there is a wait it’s never as long as in S1. Quick access to the trick guide is handy, as is being able to walk up steps (even if it is pretty glitchy). The graphics seem a little more detailed, but I’m playing on a 10″ SD set; I don’t really care. I like the songs so far.

Verdict @ 3 Hours

As I suspected, I should’ve waited for the price drop. S1 is so good that I’m still not anywhere close to tired of it, and its more minimal replay editor lets me quickly stockpile footage on the PC with minimal tinkering. I feared even a slight tampering of the controls would ruin S2 for me, and it kinda did, but I have a feeling the new replay editor will be even more of a drag over time.

New Think of the Children argument

 

Daniel Rodriguez said there’s no such thing as a healthy discussion about legalizing drugs because young people take their cues from adult conversation.

“There are things I believe should not be open for discussion, and this is one of them,” he said. [El Paso Newspaper Tree]

Thank goodness we now have more excuses to stifle discussion on a subject. It’s not even laws that might send messages to children (kids read statutes), but even discussion of them (kids watch C-Span).

El Paso (right across the border from this catastrophe) has been the center of the drug prohibition debate when their city council unanimously voted to add a few words to a resolution asking for, among other things, an “honest open national debate on ending the prohibition on narcotics.” But a request for debate in a city publication was evidently too radical, as state representatives swooped down to threaten the city’s future for federal funding.

Our real Message to Children about drugs?

Being an adult means drinking alcohol.

Youth exposure to alcohol advertising on television has risen by 38% since the Center on Alcohol Marketing and Youth began monitoring this exposure in 2001. From 2001 to 2007, the number of alcohol advertisements seen in a year by the average television-watching 12-to-20-year-old increased from 216 to 301. In 2007, approximately one out of every five alcohol advertisements was placed on programming that youth ages 12 to 20 were more likely per capita to see than adults of the legal drinking age. Almost all of these placements were on cable television, where distilled spirits companies in particular have dramatically increased their alcohol advertising in the past seven years.

Did you know alcohol advertising is one of those “self-regulated” industries? It was news to me. No bother, I’m sure with alcohol companies regulating what messages are sent to kids, we’re well on our way to a drug-free culture.