Like many older PHP projects, Elgg has lots of problems with tight coupling, procedural patterns, and untestability; and has a very web 1.9 model: spit out full page, add a little Ajax. The good news is that Elgg has a ton of great functionality and ideas embedded in that mess, we have a core team which often can find agreement about dev principles and goals, and we have a new schedule-based release process that ensures that hard work going into the product makes it to release more quickly.
Lately I feel like the Elgg core team is excitedly gearing up for a long hike, during which we’ll make tons of hard decisions and churn lots of code remolding Elgg to look more like a modern JavaScript + PHP API framework.
I’m not sure I want to make that hike.
My suspicion is there’s a shorter route around the mountain; some modern framework may be out there whose team has already put in the hard effort of building something close to what we’re looking forward to. I think the time it would take us to get there would be long and filled with tons of wheel-rebuilding—work that won’t be going into improving UX and which provides no cross-project knowledge gain for Elgg devs.
I’m also wondering if we would be wise to ignore our itching about back end code quality for a bit and focus all attention on the front end and on UI/UX problems. As a plugin developer, I certainly see back end design choices that cause problems, but they’re rarely blockers. I spend a lot more time improving the UX and dealing with our incomplete Ajax implementation. The jewel of the 1.9 release isn’t going to be the dependency container and PSR-0 compatible autoloading; it’ll be the responsive Aalborg theme.
For me, back end refactoring work is fun because it’s relatively easy. You’re changing the way the pieces snap together, not necessarily making them work better or solving new problems. It also keeps me in the comfort zone of working mostly with code and people I’m already familiar with. This is OK for a little while but doesn’t push me to grow.
This isn’t to imply that the core team is infected with Not-invented-here. We definitely want to replace as much home-grown code as possible with well-tested alternatives maintained elsewhere. It’s just a hunch I have that this will be a long process involving tons of decisions that have already been made somewhere else.
I’m still having a lot of fun developing for and in Elgg, but I’m itching to pick up something new, and to work in a system that’s already making good use of and establishing newer practices. Hitching Elgg to another project’s wagon seems adventurous.
I also have to vent that the decision to maintain support for PHP 5.2—a branch that ended long-term support 3.5 years ago—during 1.9.x seems disastrously wrong. 1.9 had a long development process during which a significant amount of high-quality, highly-tested, and actively maintained community code was off-the-table because it wasn’t 5.2 compatible. We had to port some things to 5.2 and fix the resulting bugs, and some unit tests are a mess without Closures; just a huge waste of time. Nor could we benefit from the work being done on Drupal or WordPress because both are GPL, as are a lot of other older PHP projects with 5.2-compatible code. PHP 5.2 is still expressive enough to solve most problems without namespaces, Closures, et al., but in 2014 devs don’t want to code with hands tied behind their backs to produce less readable code that will soon have to be refactored. /rant
There’s a lot to be gained by tethering Elgg to PHP 5.4+; I hope that when it hits version 2.0, this decision will be made. There’s so much you can do to simplify the code with newer constructs, and things that had to be accomplished via shims in the old days are well-supported today. Backporting to 5.2 has the ring of a decision made by bureaucracy.
My suggestion, for what it’s worth: try a new branch of Elgg. Rip off the display layer completely, and tether the logic to the front-end framework of your choice. Play a little. Experiment and innovate with what a social platform can be, and be opinionated about the philosophy of social software. Upgrading the engineering is important and much-needed, but can only take the platform so far on its own. It’s got to stand for something.
I’ve been completely delighted at being able to work on a fully OO, namespaced, interface-separate codebase on Known. The front-end there is Bootstrap, and I’m already itching to try an alternative UI using Famo.us, for example. That’ll be easy. It seems like this should be relatively with Elgg, too. (I don’t agree with single-page apps, but I also think those should be possible, too.)
One thing I’m surprised to not see mentioned: the database model. I’m going to file that under “that’s a good thing”.
Just picked this up in my feedly. Thanks for your thoughts. Would have been glad to have this discussion in the Elgg community, FYI.
I feel that I personally have been pushing especially hard on the backend refactoring stuff. There is a certain amount of personal responsibility there that I feel, given that I am now the second oldest contributor on the core team. Like if I don’t do something, Elgg will rot away because people will be scared to touch it.
Because of this, I would like to delete as much of the Elgg codebase as possible. Like if actual PHP code maintained grows, I will be slightly bummed (hopefully any new code will at least be better tested out of the box, so that’s something…). Test code can grow. As long as it doesn’t add flakiness (therefore maintenance overhead), it’s net benefit in my mind. But regarding raw implementation code written: that would be a bummer if that actually increased in the near future. It’s all about the libraries at this point, I think. Especially now that we’ve got the floodgates of PHP 5.3 and possibly 5.4+ open.
But you mention ignoring that itch and addressing more of the user experience pain. You know what, I think you’re absolutely right. Basically all the customizations I did to Elgg for my last big project were direct, UX-related, frontend stuff. I’m going to strive to wrap up the PRs I have open now, but moving on to stuff that directly affects the frontend, and only implementing as much backend stuff as we need to get to that frontend goodness, that’s going to be the sweet spot.
Incidentally, my PR for Views is something we’ll need for maintainable frontend goodness, so it isn’t all just going to waste!
I also would like Elgg to be a model of user experience. And I would like to write as little code as possible there. That is why Angular is the best bet, IMO. Backbone is definitely not going to cut it because everything I hear about it is boilerplate, boilerplate, boilerplate. Besides that I’m just very familiar with Angular and therefore very productive in it. Ember could possibly work, but I simply haven’t been exposed to the possibility of integrating it piecemeal into a legacy app like ours. I imagine it would take a lot of learning and time that I don’t have. I’m already using Angular plenty at work so that knowledge is highly transferable for me at this point.
I am definitely happy to use as much outside code as we reasonably can. One thing I really am wary of, though, is exposing more API than we can handle to the outside world. Either we need to just say “hey, this project is now part of the API of Elgg. If they make breaking changes, so will we. Be ready for that.” Or else we need to be in total control and not expose anything beyond exactly what we can support.
I feel comfortable taking the former approach with our frontend/JS stuff, because we really are just that far behind. I tried to improve things with the 1.8 JS lib, but it is just impossibly hard to keep up with the pace of JS advancement these days. So we went AMD for 1.9 to help scale the codebase, and I think that was a good decision still, given the timescale of ES6 modules. I still would like to just go Angular for 1.10+ and see how it goes. We get some experience, get a little messy, and then reevaluate or whatever at 2.0. I suspect that having written code in an Angular style, switching to any other JS framework (e.g. Polymer, Ember, React, etc.) would actually be trivially easy compared to switching from what we have now (the Web 1.9-style UI, as you call it).
To get specific, I see this being our strategy for moving forward with the UX improvements:
Adopt AngularJS 1.3 wholesale. Angular 1.x branch becomes part of our API. They are on semver now, so we should be good until their 2.0, which seems about as far away as our 2.0 at this point.
Convert one component at a time: Pick a component and convert that to PickYourFrameworkJS and convert all pages that use that component to use the JS implementation instead of the PHP + JS implementation we currently have now. Which components do we convert first?
One *page* at a time: The overarching goal of converting a whole page affects how we prioritize which components get converted first. So we pick the page, then we list all the components on that page, then we convert all the components on that page one at a time. Once that is done, we can also convert the main content of the page to use PickYourFrameworkJS.
Whole app eventually converted: Each additional page can transition to other converted pages without refresh, but they transfer to legacy pages using full refresh. This allows us to smoothly convert the whole app, page-by-page, over time without having to go all gung-ho for one big run and do it all at once, probably cutting lots of corners in the process.
So I guess first up in my mind is picking a page to convert and then map out what it would take to get there.
Some caveats:
This will balloon our JS some more. jQuery, jQuery-UI, RequireJS, AngularJS. Some could easily accuse us of just willy-nilly dropping in whatever JS framework we want. Oh well. We can trim it down in 2.0 as our chance to break BC. As long as we just use AMD and only load what we need, I’m not sure what else there is anything special to do about this. Moving the basic Web 2.0 UX forward is more important than being purists about how much JS is included, especially since maintaining much of a JS framework is not really in scope for us.
We probably will not be able to support all the features of Elgg views that currently exist. E.g. I don’t see how extending/prepending views can work in this world. Oh well. Overriding should be so easy that it’s not a big deal anyways.
Angular 1.3 means no more IE8. We don’t have the capacity to support everything. Hard choice. Let’s just make it and move on. We’ve seen the damage that holding ourselves back causes with the PHP 5.2 nonsense.