PHP

A Zend Framework App in a Single File

 

Most PHP micro-frameworks I’ve reviewed have some major cons: incomplete namespacing of functions/classes/global vars; doing too much/little; being under-tested; and the worst: forcing a unique (and usually under-documented) code structure that will make it difficult to “graduate” an app into a more full-featured framework. It also seems silly to rely on “micro-optimized” code if performance isn’t your number one goal and you have well-tested libraries sitting around.

So over the weekend, as a proof of concept (and mostly to get better acquainted with ZF’s MVC implementation), I built some minimal classes allowing one to implement a “quick and dirty” Zend Framework MVC app in a single script, with the resulting abomination being relatively easy to convert to a full app. The README shows how such a script might look. Continue reading  

Minify update

 

Minify 2.1.4 is approaching release and will have several long-awaited features and hopefully easier configuration.

Looking towards version 2.2/3(?), I recently committed the beginnings of a complete refactoring of the Minify API. The goal is to have a more flexible and extensible design that can include plugins like LESS and maybe handle @imports on the server-side—I’m still mulling over how radically a plugin should be able to alter the overall processing of the request. A key to doing this is splitting plugins into at least two stages and allowing the first to influence the cache id, and possibly add more sources/processors. By asserting that the request’s cache id is stable after the first stage, you can serve requests from the cache most of the time without running the second stage (JSMin, calls to Closure Compiler API, etc.).

Another goal was to eliminate static/global state from all components and have each component independently testable, meaning refactoring out access to global state like request/$_SERVER vars, file access from controllers, etc. A bunch of new classes and interfaces (read: files) could mean a performance hit, but we will also add an internal URL cache so that most requests will not load the system at all. At most once every N seconds the full stack will be loaded in order to make sure the cache is up-to-date with file mtimes, etc., but most requests will hit a very minimal front controller that simply includes a pre-generated PHP script (sending the necessary headers and output) depending on query string and Accept-Encoding. In theory this should be absurdly fast (for PHP output anyway), but of course sticking a reverse proxy in front is always a better idea if you can.

There’s a lot of work to be done. I thought of the URL cache a little late so some of my earlier design decisions were based on performance, and they need to be revisited.

Elgg Core Proposal: New Table “entity_list_items”

 

[This proposal has been superseded with ElggCollection.]

As of Elgg 1.7.4 there’s no way to specify a specific list or ordering of entities for use in an elgg_get_entities query. E.g. one might want to implement:

  • A “featured items” list for a group or user widget. On a group page this could be implemented in “left” and “right” lists for better display control
  • “Sticky” entities in discussion lists, or any other object lists
  • A “favorites” list for each user

Continue reading  

PHP UTF-8 string class

 

A couple years ago I put together a class for UTF-8 strings, kind of a smart wrapper around Harry Fuecks’s phputf8 string functions. All Utf8String objects—if I’m not missing bugs!—are guaranteed to house valid UTF-8 strings. The factory, make(), enforces UTF-8 validity, stripping non-UTF-8 bytes (default) or replacing them with Utf8String::$replacement (? by default).

$str = Utf8String::make('āll īs & ōk');

All the major string functions are methods with similar signatures (dropping the input string argument). Since native string assignment is always a copy, I felt all Utf8String objects should be immutable to avoid accidental referencing. After the following line, $str actually points to a fresh Utf8String object:

$str = $str->ucwords(); // Āll Īs & Ōk

Of course this means they’re chainable:

$str = $str->strtolower()->toAscii(); // all is & ok

Casting to string does what you’d think:

echo $str; // out come UTF-8 bytes.

Checking for UTF-8/ASCII-ness can be slow, so methods that create new objects propagate this info into the constructor so the new objects don’t have to check again. The constructor is private to avoid misuse of those arguments. I also threw in some convenience methods:

// input() returns false if $_POST['msg'] isn't present
if ($msg = Utf8String::input($_POST, 'msg')) {
    echo $msg->_; // escape with htmlspecialchars
}

In theory a framework could use a class like this to force safer string handling throughout:

function handleStrings(Utf8String $input) { /**/ }

It’s a proof-of-concept anyway (it’s missing preg_* methods and a lot of other stuff), but if the API could be ironed out, someone could make it into a proper C extension.

MVC: M != the Database

 

Great article on the misunderstood scope of the “Model” in the Model-View-Controller architecture. The takehome: Models are commonly thought of as wrappers for database access/stored objects, but application state and business logic need to go in them, too. Otherwise you get bloated controller and/or views that clumsily try to take care of these concerns.

Earlier today I was building a multi-step wizard app—no DB storage—and found my controllers getting messy with session and state handling code. The solution was a Wizard class that encapsulated the steps through the forms and the maintenance/validation of the collected data, but until this article I might not have thought of it as a model.

Zend Framework kinda, sorta has a hack for making a wizard using “subforms” and what makes this an awkward construction is that a view helper is not the best place to maintain the state info that a wizard requires. A “Zend_Wizard” class would need to encapsulate several forms; requirements and behaviors for moving forward and backward through the forms; and methods to inc/dec the step state, fetching the active form injected with session data. Only a single controller would be needed with 2 next/back methods, rather than methods for each step.

In the Elgg plugin ecosystem, there’s very little guidance/emphasis on creating models, so writers rarely encapsulate business logic at all. Instead it becomes spread out amongst controllers and—worse—views. Since views can easily be overridden depending on the order of plugins in a list, business logic can easily be mangled by logic from another plugin. That said, the view system does allow mods to mix behaviors in ways that would be difficult were views to be tied more closely to individual plugin models; a few exemplary mods—that store state and validation in a model rather than the controller/views—might help positively influence the culture.

Bookmarklet and PHP to prevent Shibboleth-related Firefox Lockouts

 

Reason this might be useful.

/*
 * Remove all _shibstate cookies if there are too many of them. This usually
 * occurs due to Firefox session restores. Unfortunately we don't know which is
 * the active state cookie, so we have to delete them all, but this is a lessor
 * crime than locking the user out with server errors.
 *
 * In an app a good time to call this is when a user is not logged in or has an
 * expired app session. This way we can cleanup their cookies before forwarding
 * them to the shib login process. Also after logout you'll want to call this
 * with parameter 0 to always remove them.
 *
 * @param int $allowableStateCookies if the number of _shibstate cookies
 * exceeds this, they will all be removed.
 */
function Shibboleth_preventFirefoxLockout($allowableStateCookies = 10)
{
    $stateKeys = array();
    foreach ($_COOKIE as $key => $val) {
        if (0 === strpos($key, '_shibstate')) {
            $stateKeys[] = $key;
        }
    }
    if (count($stateKeys) > $allowableStateCookies) {
        foreach ($stateKeys as $key) {
            setcookie($key, '', time() - 3600, '/');
        }
    }
}

Here’s a bookmarklet that essentially does the same thing: Fix Shibboleth Lockout

Google’s School for Hackers

 

Google is offering programmers their own personal sandbox application—called Jarlsburg—and hints of how to exploit the common vulnerabilities purposefully left in it. Although Google is basically walking folks through how to attack apps, publicizing this info is a necessary evil in order to build safer programmers. We have to start thinking of each line of code, cookie, HTTP request, and configuration option as another attack surface.

The table of contents lists the who’s who of vulnerabilities (though there are a lot more out there). Several of these attacks no one would’ve even dreamed of a few years ago, so the sad reality is that the web is chock full of vulnerable “legacy” apps just waiting to be exploited—unless we can fix them in time.

  • Cross-Site Scripting (XSS)
    • XSS Challenges
    • File Upload XSS
    • Reflected XSS
    • Stored XSS
    • Stored XSS via HTML Attribute
    • Stored XSS via AJAX
    • Reflected XSS via AJAX
    • More about XSS
  • Client-State Manipulation
    • Elevation of Privilege
    • Cookie Manipulation
  • Cross-Site Request Forgery (XSRF)
    • XSRF Challenge
    • More about preventing XSRF
  • Cross Site Script Inclusion (XSSI)
    • XSSI Challenge
  • Path Traversal
    • Information disclosure via path traversal
    • Data tampering via path traversal
  • Denial of Service
    • DoS – Quit the Server
    • DoS – Overloading the Server
    • More on Denial of Service
  • Code Execution
    • Code Execution Challenge
    • More on Remote Code Execution
  • Configuration Vulnerabilities
    • Information disclosure #1
    • Information disclosure #2
    • Information disclosure #3
  • AJAX vulnerabilities
    • DoS via AJAX
    • Phishing via AJAX
  • Other Vulnerabilities
    • Buffer Overflow and Integer Overflow
  • SQL Injection

Elgg, ElggChat, and Greener HTTP Polling

 

At my new job, we maintain a site powered by Elgg, the PHP-based social networking platform. I’m enjoying getting to know the system and the development community, but my biggest criticisms are related to plugins.

On the basis of “keeping the core light”, almost all functionality is outsourced to plugins, and you’ll need lots of them. Venturing beyond the “core” plugins—generally solid, but often providing just enough functionality to leave you wanting—is scary because generally you’re tying 3rd-party code into the event system running every request on the site. Nontrivial plugins have to provide a lot of their own infrastructure and this seems to make it more likely that you’ll run into conflict bugs with other plugins. With Elgg being a small-ish project, non-core plugins tend to end up not well-maintained, which makes the notion of upgrading to the latest Elgg version a bit scary when there have been API changes. Then there’s the matter of determining in what order your many plugins sit in the chain; order can mean subtle differences in processing and you just have to shift things around hoping to not break something while fixing something else. Those are my initial impressions anyway, and no doubt many other open source systems relying heavily on plugins have these problems. There’s a lot of great rope to hang yourself with.

Jeroen Dalsem’s ElggChat seems to be the slickest chat mod for Elgg. Its UI more or less mirrors Facebook’s chat, making it instantly usable. It’s a nice piece of work. Now for the bad news (as of version 0.4.5):

  • Every tab of every logged in user polls the server every 5 or 10 seconds. This isn’t a design flaw—all web chat clients must poll or use some form of comet (to which common PHP environments are not well-suited)—but other factors make ElggChat’s polling worse than it needs to be:
  • Each poll action looks up all the user’s friends and existing chat sessions and messages and returns all of that in every response. If the user had 20 friends, a table containing all 20 of them would be generated and returned every 5 seconds. The visible UI would also become unwieldy if not unusable.
  • The poll actions don’t use Elgg’s “action token” system (added in 1.6 to prevent CSRFs). This isn’t much of a security flaw, but in Elgg 1.6 it fills your httpd logs with “WARNING: Action elggchat/poll was called without an action token…” If you average 20 logged in users browsing the site, that’s 172,800 long, useless error log entries (a sea obscuring errors you want to see) per day. Double that if you’re polling at 5 seconds.
  • The recent Elgg 1.7 makes the action tokens mandatory so the mod won’t work at all if you’ve upgraded.
  • Dalsem hasn’t updated it for 80 days, I can’t find any public repo of the code (to see if he’s working on it), and he doesn’t  respond to commenters wondering about its future.

The thought of branching and fixing this myself is not attractive at the moment, for a few reasons (one of which being our site would arguably be better served by a system not depending on the Elgg backend, since we have content in other systems, too), but here are some thoughts on it.

Adding the action token is obviously the low hanging fruit. I believe I read Facebook loads the friends and status list only every 3 minutes, which seems reasonable. That would cut most of the poll actions down to simply maintaining chat sessions. Facebook’s solution to the friends online UI seems reasonable: show only those active, not offline users.

“Greener” Polling

Setting aside the ideal of comet connections, one of the worst aspects of polling is the added server load of firing up server-side code and your database for each of those extra (and mostly useless) requests. A much lighter mechanism would be to maintain a simple message queue via a single flat file, accessible via HTTP, for each client. The client would simply poll the file with a conditional XHR GET request and the httpd would handle this with minimal overhead, returning minimal 304 headers when appropriate.

In its simplest form, the poll file would just be an alerting mechanism: To “alert” a client you simply place a new timestamp in its poll file. On the next poll the client will see the timestamp change and immediately make another XHR request to fetch the new data from the server-side script.

Integrating this with ElggChat

In ElggChat, clicking a user creates a unique “chatsession” (I’m calling this “CID”) on the server, and each message sent is destined for a particular CID. This makes each tab in the UI like a miniature “room”, with the ability to host multiple users. You can always open a separate room to have a side conversation, even with the same user.

In the new model, before returning the CID to the sender, you’d update the poll files of both the sender and recipient, adding the CID to each. When the files are modified, you really need to keep only a subset of recent messages for each CID. Just enough to restore the chat context when the user browses to a new page. The advantage is, all the work of maintaining the chat sessions and queues is only done when posts are sent, never during the many poll requests.

Since these poll files would all be sitting in public directories, their filenames would need to contain an unguessable string associated with each user.