Forrester is one of the most well-known analyst companies that publishes ranking reports for content management systems (Gartner being the other one). The most recent Forrester Wave report about web content management systems (Q1/2017) was published at the beginning of this year, and there were some surprises, to say the least. I will review the report in this article and offer some background for how Forrester’s report should be interpreted.
North Patrol is a consulting firm specialized in the design of digital services and information systems. We shape ideas into a vision and service concept, find the best architectural and technological solutions, design a functional user experience, and compete to find the ideal partner for implementation work. We do not sell implementation projects, nor do we sell licenses; we are genuinely on the side of the customer.
5 June 2017
This article is a part of CMS selection article series by North Patrol.
North Patrol helps customers to make smart technology decisions and find the best implementation partners. Typically, we facilitate prestudy projects and evaluate vendors and proposals. Most of our clients are large companies headquartered in Finland.
Image: The main graph from Forrester's report.
First of all, Forrester's report is targeted to large companies that have extensive ecosystems of websites. Companies that only have one or two medium-sized websites should not be reading Forrester's reports. Typical Forrester's readers come from organisations that employ thousands of people and have operations in numerous countries and regions.
Secondly, Forrester doesn't care about regional differences in partner networks. There are numerous systems in the report that have minimal presence in Nordic countries. If a Nordic based company is expecting to find a local integrator, then Forrester's report is a quite difficult starting point.
In my opinion, these two aspects are the basic limitations of the report. The report is applicable to only big companies that have a minimum of hundreds of thousands of euros to spend and don't really care whether the system has local partners.
You could even argue that the primary target audience are companies that are prepared to invest millions of euros into their web renewal. Naturally, that is also the reason why vendors compete so much to get into these reports.
Partner networks and capabilities are an afterthought
Before we look at the scoring system, I'd like to say that personally I consider, that by far, the biggest handicap of these reports is the so-called global focus. They evaluate market presence based on vendors' data and don't really give weight to partner networks. For example, Sitecore and Hippo are scored side by side in the report, even though those systems have very different global market presence and reach. And even though Sitecore is a truly global player, Drupal and WordPress still beat Sitecore's developer base in any region of the world. Forrester's report does not have any mention about these little differences between the products.
The report is also full of CMS products that have a very limited reach beyond their home markets. For example, there are many products that are mostly significant players in the German-speaking areas of Europe. From a Nordics perspective, half of the report's vendors are completely irrelevant players.
The rough reality with these systems is that they are complicated products, where hiring a local integrator to do the implementation is a recipe for failure. The best way to increase the odds of success is to hire an experienced partner that can do close co-operation with content producers, marketing people and other key stakeholders - and that doesn't really work if the nearest experienced partner is located thousands of miles away.
Limitations and bias of the scoring system
The biggest peculiarity of the scoring system is that half of the scoring is based on evaluation of vendor's strategy. That strategy is also evaluated based on solely vendor meetings and their materials. Additionally, there are no stated criteria for a "strong strategy", or any evaluation of whether the vendor actually delivers on their strategy. Therefore, you could argue that Forrester is basically giving points for sales pitches.
This year Forrester seemed to give significant value to vendors' cloud hosting capabilities, and that naturally created major differences between the vendors, since some vendors have already launched their upgraded platforms, while others are still working on them. Naturally, analysts have to make some statements about the future, and they have to draw the line somewhere, but it can be argued whether the cloud offering is the most crucial thing with CMSs right now? Yes, it is trendy, but is it the biggest thing?
For example, Acquia scores very well in the report, even though Forrester itself admits that Acquia has significant gaps in their product offering and in their ability to deliver on their promises. Acquia is no doubt a major player right now, but there are significant question marks in how it can bridge the open source Drupal world and the company’s proprietary commercial offering. Yet Forrester focuses on giving credit to Acquia's robust cloud hosting capabilities.
Limited representation of players at the market
The included systems are somewhat of a strange bunch. Many smaller and growing systems are not included (e.g. Umbraco, Kentico). Yet there are plenty of "old elephants", like IBM, Oracle and OpenText which are not known for signing new deals in recent years. Then there are a few really strange ones. For example, Jahia can't really be considered to be a web content management system with any sane criteria. It is true that Jahia has started to market its product for external scenarios also, but the truth is that the product is still a fairly traditional portal (and nothing wrong with that!) that is mostly used as an intranet platform.
It is hard to see what is the question to which Forrester's report answers. The report is not a good starting point for customers who are looking for the most capable and modern CMS products. Either the report is representative of the current installation base. For example, Automattic (the company behind WordPress) is categorized as a visionary. Yet Automattic probably has by far the biggest install base, biggest developer base, biggest community and biggest add-on ecosystem. It is very hard to understand the market realities by reading the report - if anything, reading it makes it more complicated, especially if one happens to know these products and their partner networks.
Especially "big elephants" (IBM, Oracle, OpenText) score much better than most experienced analysts or experts would expect them to score. Also, there are numerous niche players ranking very high, even though they have very limited market presence. For example, e-Spirit is a very regional player from Germany that might well have great technology, but even after years and years of hard work, they have not been able to get anything comparable to the market presence of leading systems, like Sitecore, Drupal, Adobe or Episerver.
Therefore, my advice for buyers is to pretty much ignore the latest Forrester's report. It might look like an easy starting point for choosing a new CMS, but the reality is that it gives very little guidance in how the market works in real life, or how a reasoned and sustainable CMS selection should be done.
