Allen Ellis: Why the Packt CMS Competition is Broken, and How to Fix It

By now, many of us are excited to see the finalists for this year's Packt Publishing Open Source CMS (Content Management System) Award competition, the annual contest in which dozens of companies compete to be highlighted as the year's finest Content Management System.

Entering its fourth year, this competition has grown nearly five-fold and is widely regarded as the most prestigious award available in the CMS industry. But as successful as the competition has become, it sadly suffers from inherent issues which prevent it from truly presenting today's gamut of CMS choices in a valuable way.

Here is the reason: to even be considered as a finalist in the competition, each CMS is judged on exactly one condition: community vote counts. Granted, this can be a valid measure of a CMS's success, however today's CMS environment is very polarized. We see large majorities of people using Drupal, Joomla, and Wordpress, often simply because those are the CMSs in the headlines.

As a result, these CMSs are naturally going to have far more votes in every competition - they are votes from people who already use these CMSs and who rarely branch out to explore alternatives. This self-fulfilling cycle ensures that these few CMSs continue to dominate our headlines, simply because they were the early winners and can rely on their present install-base for votes, rather than relying on the merits, features, and usability of their own systems in comparison with other CMSs.

Consider the reasons the 3 aforementioned CMS's became so well known, prior to the Packt competition:

• Joomla grew because there was a lot of news coverage over several months when the developers of Mambo split to create Joomla. Since then Joomla cemented their position by encouraging commercial development of addons and now has a lot of companies incentivised to promote it and ready to come up under general Google searches.
• Wordpress got a massive boost from the most vocal group of people you could hope for: bloggers, due to its blog-positive simplicity.
• Drupal has developed a massive ecosystem of web development agencies and regularly gets noticed as a side-effect of the business activity of these agencies- for example, there are countless conferences held each year.

Much of the above is good (it's good that Wordpress is user-friendly, and that Drupal and Joomla have large support networks), but it should also clearly show how CMS popularity was based on headlines, the size of business networks, and incentivization, rather than a full range of factors such as usability or features. The problem we see is that the Packt competition now sustains this further.

In a way, the problems Packt creates are the problems of a two-party system in politics, and the reason why many people now promote the concept of having second-choice votes on ballots.

Put another way: The Packt Competition isn't really about finding the best CMS. The mechanics of it ensure the winners are simply those with the largest communities.

Case in point: everyone has acknowledged a need for easy theme creation but hardly anyone is doing anything about it. But the CMS I work with (which I won't mention here as I want this to stay neutral and not an advert) has had a feature to generate all-new theme stylesheets and image colors for years, and we are only now starting to see other CMSs add this.

Marketing is never free, and we understand this: we've made marketing efforts and wanted to compete in this year's competition, but realized we can't afford the high resources needed to compete on a per-vote count with the more established players. Specifically (assuming none of our users offers to do this for us) we'd need to spend a couple of weeks campaigning very hard, kind of like running a political campaign, which would be very distracting, diverting our resources. The established players, on the other hand, have to do very little to be nominated highly.

There are a lot of very good CMS's that won't do well in the Packt competition, that in many use-cases are probably better choices than those that will win.

The Solution

So how can we recognize the power of all CMSs in a more valuable way, and nurture more competition within the market?

The key is to rely on the best and most fair indicator we have: human experience. Imagine if the competition were to judge each entry on factors that really matter - which can't ever be measured by a machine. This would remove 'majority rule' as the primary factor and replace it with votes based on real experience - producing far more valuable results.

We suggest the following: allow people to nominate any CMS as they currently do, but then present the top 30 nominees for public voting. Then, instead of just tallying unscreened and anonymous votes, expand the voting process by requiring each voter to have personally used at least five of the nominated CMSs. Allow them to vote for as many of those five as they like, by writing a few sentences to a paragraph describing why they believe that particular CMS is the "best" among its competition in that category or writing why they are not giving that particular CMS a vote.

This fresh approach solves the previous issue by bringing back a vital concept: accountability. Like a salesman who has never used anything except his own product, that old system discourages voters from doing any research, and to vote up "their" CMS based on what may be a very narrow viewpoint and doesn't take into account less-known CMSs, despite how well those would actually compete.

A panel of judges would compile the results and decide on the winners.

This way, the competition does not need to be a popularity contest for the masses - a few dozen voters are enough. This change embodies the belief that it's better to present 100 good votes than 10,000 bad ones.

The best part is that these changes are very do-able and, at least in my mind, would bring some much-needed competition back to this industry. In my opinion Packt can add a lot of value to the CMS community at large, truly presenting valuable new information.

If you agree, please take a moment to share your thoughts with the Packt Publishing Award team [cmsaward@packtpub.com] or by continuing this discussion in the comments below or on your own websites. I'm sure there are a lot of ideas out there, and that if we truly intend to present an alternative we should share ideas.

Hopefully we can all see that a newer and fairer method of choosing the winners of the Packt Publishing CMS Award competition would lead to a more diverse and competitive CMS market - with more obvious choices for those looking to create a CMS-driven website. It would also increase competition within the industry, thus improving features and usability across the board.

And lastly, that all lesser-known CMSs should be given the fair chance to shine against their competition (if they're worthy), and not just languish in obscurity.

Disclosure: this article was written by Allen Ellis, Creative Director for a CMS (ocPortal) which did not compete in this year's competition.