Modern application design is solved! OK, well we at least have a set of camps with their own principles, tools, and paradigms leading us toward the light. One might be “build some components, wire them up with references to each other, and let them talk to each other.” We love/hate static typing, but it allows tools to reason about really large programs.
All that is great, but I feel like plugin system design is in the wild west. To be clear, I mean dropping a new directory of code in place, performing some ritual, and the system behaving radically differently. Of course, we’re getting better at this as apps learn from each other, but I feel like our principles and tools aren’t particularly well matched to this goal of plugins being able to cooperate on a deep level to build up a system.
Allowing plugins to provide APIs introduces a big set of challenges. Just a few:
- How do we conditionally use API’s if they’re available?
- How can a plugin decorate/filter the API’s output?
- How can a plugin replace the API with its own?
- If two plugins want to replace the API, which “wins”?
- How can a plugin remove part of the behavior/API of another plugin?
- How much can plugins detect about each other?
Most robust plugin systems solve these with event systems and some mechanism of ordering plugins to solve disputes (all unique).
A big problem is that events are almost always run-time linking based on strings. Hence it’s very difficult for tools and humans to reason about which listeners will be called, in what order, and what data will flow to each listener and be returned to the dispatching party. Ideally IDEs could sense all this stuff, and help wire up new listeners and events. Instead, devs on a plugin-heavy system must do string searching on event names or at best use some reporting tool built into the event system.
Symfony’s typed event objects and Ruby’s symbols probably help here. Drupal has a strong convention both in code and docs that helps, and is popular enough for toolmakers to focus on it. Middleware systems as in Express (node) and Stack (PHP) offer a formal way to compose applications, which is pretty exciting, but I’m not sure they solve any of the above problems of plugins tightly collaborating on processes.
What’s the future look like here, and what’s the way toward it? Standardizing on a single event system seems unlikely, but what are the best and most powerful ones out there? What language features would make this easier? Can someone stop me from diving deep into Event-driven programming literature?
Have you done any research into observables? Familiar collections based paradigm, but “asynchronous”. Emitted items can be typed.
Main issue seems to be the ad-hoc nature of the events right?
$events->on(‘db’, ‘create’, function ($event) {})
Would change to
$db->onCreate->each(function(CreateEvent $event) {})
And that would be substantially better for type hinting and such, but it means you can’t necessarily trigger arbitrary events on existing structures.
it may help to think inside-out:
> which listeners will be called
every registered… up to the listener to decide
> in what order
it’s an anti-pattern to rely on order… as ordering can/will change
> and what data will flow to each listener and be returned to the dispatching party
listeners register for data they want to receive and return no data at all… a flow forward (in time)
Sorry for the late comment approvals.
> listeners … return no data
Why is this important? Or rather, why is it important listeners can’t modify the event data?
I could imagine getting rid of a lot of runtime events with a compile step: The core and plugins declare their intentions to take part in processes, and the compiler builds a set of functions (in a central location for easy study) which just call the “listeners” sequentially. It wouldn’t solve the issue of runtime event registration.
> > listeners … return no data
> Why is this important?
single responsibility: If data gets returned, you need to handle it in issuer. Maybe data gets invalid and you need to inform all listeners again? There is some (not-so-global) “state” in here. It’s a time loop.
> Or rather, why is it important listeners can’t modify the event data?
They can modify data but only give it further not back. Issue a new event. Maybe work of listener is time consuming. Maybe there are more listeners interested in this modified data.
Its’s #stateless #asynchronous and #decoupled. :)
(Don’t know about the “compile” part. I wrote down only some abstract thoughts.)