Selecting a CMS: How to Show Your Work

Content Management Systems (CMS) have hit the mainstream, and clients expect them as a standard deliverable for most web development projects. As a trusted partner, Mad*Pow is continually charged with choosing the best CMS for our clients. But how can we ensure that we've made the best choice with so many vendors on the market? And - just as important - how do we communicate to our clients that we've done our due diligence and selected the best CMS to set them up for success, now and in the future?

Show your work with a CMS Selection Matrix

First, let's take a step back to our formative high school days, specifically, algebra class. Remember those classic "solve for x" algebra equations?

2x + 4 = 10

If you happened to be one of those students that did all of the work in their head and simply wrote "x = 3", then you've most likely experienced the not-so-wonderful concept of receiving partial credit. Even though you gave the correct answer, you didn't do the due diligence of showing all of your hard work. Showing how and why you came up with the answer is as important as the answer itself.

The same holds true when selecting a CMS for Mad*Pow's clients. When we make a recommendation, the final deliverable includes the analysis executed to gain that valuable insight into the how and why, which in turn enables our clients to trust that we've listened to all of their needs and set them up for optimal success. The best way to show our work is with a CMS Selection Matrix.

At its core, a CMS Selection Matrix is fairly basic. It is a spreadsheet of weighted criteria that are used to evaluate and score a short list of CMS's. The CMS with the highest score wins! We've shown our work and everybody's happy, right? Right?

If only it were that simple

Scoring a CMS takes a little more than plugging numbers into a spreadsheet and tallying them up. Decisions need to be well-informed, and Mad*Pow has extensive knowledge of the advantages and pitfalls of many of the top tier CMS vendors. We stay up to date with the latest industry research and build vendor relationships that we can tap into for introductions and in-person demos.

With all that said, we can't claim to know the clients' pain points and requirements without first doing a little digging. We need to collaborate with our clients, having open and honest conversations with key stakeholders to understand their level of comfort and expertise with CMS solutions. Ensuring every voice is heard and every concern is brought to the forefront allows us to make a well-informed decision.

So who needs to be involved? With every project, we make sure to interview:

  • The Project Sponsor - The project sponsor will bring a unique perspective to any discussion around selecting a CMS. More than likely, they have been a part of the internal conversations from the start, long before Mad*Pow was engaged as a partner. They bring a wealth of knowledge and history along with them that can shape the conversation using past viewpoints gathered from internal team members.
  • The Business Users - Generally, these are the client's marketing and communications team. These are the people who will be using the product day in and day out, and have most likely been suffering with an archaic CMS for the past five years. Their wants and needs are top of mind, and they bring a lot of great usability, functionality, and workflow ideas to the table when defining requirements.
  • The Technical Users - In most cases, this is the client's IT team, who will keep the system up and running long after it goes live. They will be the driving force behind the chosen technology stack, security and performance requirements, and other hardware and software concerns. Long term success of the CMS implementation relies on their involvement. It may be necessary to break down internal client communication barriers to get them involved as early as possible.

The collective brain power has been gathered. Now what?

Now that all of the important players are gathered in one room, it's time to create selection criteria that are relevant to the success of the project. Focused conversation begins by breaking down the criteria into high-level categories, including:

  • Content Management - This is the core of what the CMS is meant to do, so we spend a lot of time creating criteria for this category. Is the content well organized with a structured taxonomy? Are all required graphic and media types supported? Does the CMS offer personalization features and does it support accessibility and web standards?
  • The CMS Vendor - The overall health and strategic strength of the CMS vendor is an important part of the equation. Determining if they have a strategic vision, are positioning themselves well in the market, and are professional and easy to communicate with will ensure that they will be around to support our clients in the next 3-5 years.
  • Hardware & Infrastructure - The client's IT team will drive most of the conversation around hardware and infrastructure. Security and performance requirements, hardware storage limitations, and the proper technology stack and hardware required to run the system are all important factors when selecting a CMS.
  • Third Party Product Integration - Seamless integration with existing client systems and data stores can enhance the basic functionality of the website and ensure we're selecting an extensible and modular platform.

    An example of evaluation criteria

    An example of evaluation critieria

  • User Interface - Successful adoption relies on a usable CMS that provides an efficient, task-based user interface that matches authors' mental models. Minimal training should be required and intuitive inline help and instructions should be available when needed. The interface should be robust and error-proof, hiding underlying implementation details while supporting both frequent and infrequent users.
  • Workflow & User Management - Most clients trust their CMS to a team of individuals, and not everyone on the team will have the same role. Definition of user groups and permissions to support complex review cycles may be required, along with content locking, branching, and merging.
  • Search - Content is king, and Search Engine Optimization (SEO) best practices such as SEO-friendly URLs, page redirect support, and page meta information ensures that content is fully visible to search engines. Lack of these features may hurt website search engine rankings as well as have a negative effect on the user experience.
  • Translation Management - The world is getting smaller and multilingual sites are common in today's web. The ease of creating and maintaining translated content will have a huge impact on those clients striving to reach the global community.
  • Metrics & Reports - Measuring the effectiveness of our client's web presence is important to understanding where their implementations are working and where they need improvement. Analytics packages, A/B testing, marketing campaign management, and detailed built-in reports and logging that are core to the CMS installation will provide the insight required for proper measurement.
  • Support - Having product support post-launch is critical. Does the license include product support or is it an add-on feature? Is customer support friendly, fast, and easy to interface with? Does the vendor maintain forums and publicly available product defect reports?

Not all criteria are considered equal

Once the individual criteria items have been identified, we conduct a session to review each item and assign a weight. Some criteria are more important than others and should have more of an influence on the final selection. Typically, we weigh each criteria on a scale of 1-10, with 1 being low importance and 10 being critically important. In the example below, the client is focusing on an internal application targeted specifically for the desktop. As a result, custom development and integration with a SharePoint system are weighed more heavily than device detection and a content delivery network.

Evaluation criteria with weighting applied

Evaluation criteria with weighting applied

It's not an exact science and it may take a few rounds of review, but the exercise calls out the "must have" from the "nice to have" features, helping our clients better understand their own priorities.

Begin the evaluation

Now that we have a thorough list of weighted evaluation criteria, we need to decide which CMS vendors to evaluate. In most cases, we limit ourselves to four or five and expand from there if needed. If a CMS is already being used by the client, we include it in the list for comparison purposes. They may have already made the decision to move away from that vendor, but discussing the evaluation criteria in the context of what they know tends to bring forth pain points that may otherwise go unmentioned. The target technology stack will also influence the initial short list; however, including an option outside of that stack may allow for deeper feature comparison.

Once we've settled on our short list, we try to schedule an in-person or online product demo from each vendor. The vendor's willingness to offer a demo will provide some insight into what they're like to work with and whether they're committed to our client for the long haul. We also use this opportunity to ask targeted questions, digging as deep as we need to get a sound understanding of their product offering from front to back.

Finally, we request an evaluation copy of the CMS. Firsthand experience on how easy or difficult it is to install the product, create pages, and add or publish content using the built-in workflows provides a surprising clarity of its strengths - and flaws.

Score the CMS's

Once we have a thorough understanding of each CMS option, we rate them against our criteria on a scale of 0-5, with 0 indicating no support and 5 indicating full support. We then multiply our evaluation by the criteria weight, resulting in a weighted score. Subtotaling each category, as well as all categories together for a grand score, begins to tell us a story on how each CMS stacks up against one another, specific to our client's needs.

Weighted scores generated for two fictitious CMS brands

Weighted scores generated for two fictitious CMS brands

A part of the whole

A solid CMS Selection Matrix is just one part of the process of recommending the right CMS for our clients. Total Cost of Ownership (TCO) analysis, including initial hardware costs, licensing costs, and personnel required to support content generation and upkeep plays an important role as well. The result of this process is a final selection that is well informed, and a client who understands our thinking and trusts our recommendations due to the fact that we've shown our hard work every step of the way.


By: Rich Storch

Want to offer your customers a better experience?

Let's work together: Contact Us Today

Contributed by:
Rich Storch Principal Application Developer   Contact Rich

Newsletter Sign Up

Mad*Pow HQ

27 Congress Street
Portsmouth, NH 03801

Office: 603.436.7177
Fax: 603.386.6608


179 Lincoln Street
Boston, MA 02111

Office: 617.426.7177


Complete Sale Inquiry Form


Sales: solutions
Press: communications

Get in Touch