Configuring Sendmail for UF’s SMTP

Our Ubuntu web host, hosted with OSG, was not able to send mail (using PHP mail) outside of UF. An OSG tech said our From: header should be a valid address at UF (check) and that the logs at smtp.ufl.edu showed those messages never made it there.

The solution was to configure sendmail to use smtp.ufl.edu as the “smart” relay host (as it’s described in the config file):

$ sudo nano /etc/mail/sendmail.cf

Ctrl+w and search for smart. On the line below, add smtp.ufl.edu directly after DS with no space. The result should be:

# "Smart" relay host (may be null)
DSsmtp.ufl.edu

Ctrl-x and save the buffer.

Restart sendmail: $ sudo /etc/init.d/sendmail restart

As soon as I did this, the queued messages were sent out. I still don’t know why messages to ufl.edu succeeded while others sat in the queue.

Minify 2.1.3 released

This is how I spend my time off? I squashed a few more bugs and got Minify 2.1.3 finally out the door. It fixes a few HTTP bugs, including working around a Safari/webkit bug that was preventing efficient cache usage. There’s also code to make it far easier to diagnose users’ URI rewriting problems when they come up (caused a lot of past headaches).

I updated mrclay.org to use it, so you can check out Minify’s HTTP handling using Mark Nottingham’s new Resource Expert Droid:

RED is a robot that checks HTTP resources to see how they’ll behave, pointing out common problems and suggesting improvements. Although it is not a HTTP conformance tester, it can find a number of HTTP-related issues.

Miško Hevery Programming Talks

Miško Hevery gave several presentations at Google last year that are worth checking out, I think even if you’re familiar with Dependency Injections and unit testing. They cover the ways that global state can sneak into applications, how undeclared dependencies make classes harder to test and reuse, and how DI in general eases a lot of pain. He’s just a good teacher and the examples are clear and made me want to attack a lot of old code. Hig blog also covers a lot of the same topics.

Minify getting out there

Interest in Minify seems to be picking up:

Any ideas on how to make it better?

Where’s the code?

Google’s free open source project hosting has been awesome for Minify, so when I was looking around for Subversion hosting for my personal code, I figured why not host it there? So here’s a bunch of my PHP and Javascript code. Hopefully some of it will be useful to people. A few PHP highlights:

  • HashUtils implements password hashing with a random salt, preventing the use of rainbow table cracks. It can also sign and verify the signature of string content.
  • StringDebug makes debugging strings with whitespace/non-printable/UTF-8 characters much less painful.
  • CookieStorage saves/fetches tamper-proof and optionally encrypted strings in cookies.
  • TimeZone simplifies the handling of date/times between timezones using an API you already know: strtotime() and date().
  • Utf8String is an immutable UTF-8 string class that aims to simplify the API of the phputf8 library and make behind-the-scene optimizations like using native functions whenever possible. Mostly a proof-of-concept, but it works.

I’ll get the examples of these online at some point, but if you export /trunk/php, all the lowercase-named files are demos/tests. There’s also an “oldies” branch (not necessarily goodies).

Hopefully this makes the political jibba jabba more forgivable.

Getting phpQuery running under XAMPP for Windows

While trying to run the initial test scripts included with phpQuery 0.9.4 RC1, I got the following warning:

Warning: domdocument::domdocument() expects at least 1 parameter, 0 given in C:\xampp\htdocs\phpQuery-0.9.4-rc1\phpQuery\phpQuery.php on line 280

This is strange because DOMDocument’s constructor has only optional arguments.

As it turns out, XAMPP for Windows ships PHP with the old PHP4 “domxml” extension enabled by default (appears as extension=php_domxml.dll in \xampp\apache\bin\php.ini). This deprecated extension has a somewhat less-documented OO API, with domxml_open_mem() being accessible via new DOMDocument(). I.e. domxml hijacks PHP5’s DOMDocument constructor.

Comment extension=php_domxml.dll out with a semicolon, restart Apache and phpQuery seems to work as designed.

flock() blocking reads

(Sigh.) After applying recent upgrades to our Red Hat 5 server at work, suddenly PHP file locking blocks the script execution for exactly 30 seconds! Both a shared reading lock (LOCK_SH) and exclusive writing lock (LOCK_EX) do this. This was, shall we say, unpleasant to diagnose. Since we use Cache_Lite (which locks by default) to cache lots of stuff and sometimes in multiple layers, most of our pages suddenly took 30, 60, etc. seconds to load! Here were duration times from a test script:

  'write' => '0.00257301',
  'read' => '0.00039792',
  'write w/ exclusive lock' => '30.00274611', (file_put_contents w/ LOCK_EX)
  'fopen' => '0.00080800',
  'get shared lock' => '29.99504590', (flock w/ LOCK_SH)
  'read file' => '0.00034904',
  'close file' => '0.00005412',

Awesome. Now it is documented that flock() “will not work on NFS” (which we are on), but this has worked fine for over a year and continues to work on our unpatched server. The versions of Apache and PHP did not seem to change. Here’s how Red Hat listed them:

old: httpd-2.2.3-6.el5 new: httpd-2.2.3-11.el5_1.3
old: php-5.1.6-12.el5 new: php-5.1.6-20.el5_2.1

So… any ideas? I’ll have to turn off fileLocking in Cache_Lite to move forward, but I’d love to know what’s up. This especially troubles me because I recently added default cache locking to Minify. Locking is also the default in Zend_Cache_Backend_File, so it seems like the right thing to do.

Minifying Javascript and CSS on mrclay.org

Update: Please read the new version of this article. It covers Minify 2.1, which is much easier to use.

Minify v2 is coming along, but it’s time to start getting some real-world testing, so last night I started serving this site’s Javascript and CSS (at least the 6 files in my WordPress templates) via a recent Minify snapshot.

As you can see below, I was serving 67K over 6 requests and was using some packed Javascript, which has a client-side decompression overhead.

fiddler1.png

Using Minify, this is down to 2 requests, 28K (58% reduction), and I’m no longer using any packed Javascript:

fiddler2.png

Getting it working

  1. Exported Minify from svn (only the /lib tree is really needed).
  2. Placed the contents of /lib in my PHP include path.
  3. Determined where I wanted to store cache files (server-side caching is a must.)
  4. Gathered a list of the JS/CSS files I wanted to serve.
  5. Created “min.php” in the doc root:
    // load Minify
    require_once 'Minify.php';
    
    // setup caching
    Minify::useServerCache(realpath("{$_SERVER['DOCUMENT_ROOT']}/../tmp"));
    
    // controller options
    $options = array(
    	'groups' => array(
    		'js' => array(
    			'//wp-content/chili/jquery-1.2.3.min.js'
    			,'//wp-content/chili/chili-1.8b.js'
    			,'//wp-content/chili/recipes.js'
    			,'//js/email.js'
    		)
    		,'css' => array(
    			'//wp-content/chili/recipes.css'
    			,'//wp-content/themes/orangesky/style.css'
    		)
    	)
    );
    
    // serve it!
    Minify::serve('Groups', $options);

    (note: The double solidi at the beginning of the filenames are shortcuts for $_SERVER['DOCUMENT_ROOT'].)

  6. In HTML, replaced the 4 script elements with one:
    <script type="text/javascript" src="/min/js"></script>

    (note: Why not “min.php/js”? Since I use MultiViews, I can request min.php by just “min”.)

  7. and replaced the 2 stylesheet links with one:
    <link rel="stylesheet" href="/min/css" type="text/css" media="screen" />

At this point Minify was doing its job, but there was a big problem: My theme’s CSS uses relative URIs to reference images. Thankfully Minify’s CSS minifier can rewrite these, but I needed to specify that option just for style.css.

I did that by giving a Minify_Source object in place of the filename:

// load Minify_Source
require_once 'Minify/Source.php';

// new controller options
$options = array(
	'groups' => array(
		'js' => array(
			'//wp-content/chili/jquery-1.2.3.min.js'
			,'//wp-content/chili/chili-1.8b.js'
			,'//wp-content/chili/recipes.js'
			,'//js/email.js'
		)
		,'css' => array(
			'//wp-content/chili/recipes.css'

			// style.css has some relative URIs we'll need to fix since
			// it will be served from a different URL
			,new Minify_Source(array(
				'filepath' => '//wp-content/themes/orangesky/style.css'
				,'minifyOptions' => array(
					'prependRelativePath' => '../wp-content/themes/orangesky/'
				)
			))
		)
	)
);

Now, during the minification of style.css, Minify prepends all relative URIs with ../wp-content/themes/orangesky/, which fixes all the image links.

What’s next

This is fine for now, but there’s one more step we can do: send far off Expires headers with our JS/CSS. This is tricky because whenever a change is made to a source file, the URL used to call it must change in order to force the browser to download the new version. As of this morning, Minify has an elegant way to handle this, but I’ll tackle this in a later post.

Update: Please read the new version of this article. It covers Minify 2.1, which is much easier to use.

New Minify version in the works

Back in August I came across Ryan Grove‘s great Minify project, which is essentially a PHP client-optimizing file server for Javascript and CSS. I was in the design stage of a project with similar goals so I decided to try merging the two projects. Ryan thought the result had potential to be the base of the next major release, and since he hadn’t had much time lately to devote to Minify, he made me co-owner of the project.

Last week I finally got around to committing the code, and since then I’ve been fleshing it out, adding more features, docs, and tests, and updating the project pages a bit. The major difference is that the server is now built from separate utility classes, each lazy-loaded as needed. Some new features:

  • separate front controller makes it easy to change how HTTP requests are handled
  • file caching uses the very mature Cache_Lite
  • supports conditional GET and far-future Expires models of client caching
  • serve content from any source, not just files
  • new “minifiers” (content compressors) for CSS and HTML with unit tests

At this point to see the new Minify in action you’ll need a Subversion client, like the Windows shell extension TortoiseSVN or the Eclipse plugin Subclipse. You can checkout a read-only working copy at http://minify.googlecode.com/svn/trunk/ and the README has easy setup instructions.