How I Made It: the Anatomy of the WebOps Certification

In my previous post in this blog series about the Pantheon WebOps Certification Program, I provided an overview of the program, its origins and some of its aspirations. In this post, I will highlight how the exam came into being, including the selection process of the exam topics, and the creative minds supporting this process.I will also share some of the ideas we have for additional certifications in the future.

Conducting a Needs Analysis

The first step in creating a product certification program is to identify that there is a need for such a program. This may seem obvious, but without conducting a needs analysis, you risk devoting time and resources to a pursuit that will ultimately translate into a wasted effort. A thorough analysis also provides an initial guide for what your North Star finished product would look like.

At Pantheon, we aim to make the open web a first-class ecosystem that delivers results by providing the platform, tools, automation and guardrails to help our customers deliver on the promise of open source web technology. To use an analogy, we maintain the tracks on which our customers drive their trains. The Pantheon platform is a technical product with certain opinions on best practices baked into the product. The better our customers and partners (and our employees too!) understand the capabilities of our product and the logic behind our best practices, the more value they tend to derive from the platform.

Creating a Topic Outline

The next step after conducting a needs analysis was to create a topic outline of the content that would be covered in the exam. This early step was a critical piece of the puzzle because it required distilling the entire Pantheon knowledge base into a much narrower set of topics that made sense for a certified Pantheon developer to know. 

There is a definite art to this process, and it requires input from many knowledgeable stakeholders, from Subject Matter Experts (SMEs) in our Product and Engineering and Sales Engineering teams, to our customers and agency partners. 

To narrow down the full range of information into a logical topic outline for a certification program, I made a note of several things: 

• Common pitfalls and misconceptions of customers and prospects about the platform technology, which lead to dissatisfied customers, lost sales opportunities and headaches for our support and professional services teams.
• The fundamental tasks that I, as a developer and sales engineer running demos, performed most often as required actions to successfully work on the platform.
• Common questions that I received from prospects and partners during sales conversations and account management activities.
• New and exciting products and features that serve as value adds to our fundamental offering (Autopilot, Advanced Global CDN, Image Optimization to name a few).
• Gaps in our messaging that needed to be defined, specifically around the rigor we used to describe WebOps and its value proposition. 

Creating Learning Objectives

Once we identified the need for a certification program, the next step was to build out the learning objectives. 

Using a planning approach, called the Backward Design framework, that begins with the end goal in mind, I developed a comprehensive list of what learners should know and be able to do by the end of the certification program. This method starts by first deciding on the desired outcomes, then determining how we'll know if those outcomes are achieved, and finally, planning the teaching and learning activities to reach those goals. From there, I created a comprehensive list of learning objectives, using the SMART framework (Specific, Measurable, Achievable, Results-oriented, Time-bound) to assess the quality of Learning Objectives.

Learning Objectives tie the content and the exam questions together. The study guide has learning objectives that are found at the beginning of each chapter. The content in the study guide contains the information supporting each Learning Objective. Mapping each question in the assessment to one or more learning objectives ensures that the appropriate information is being assessed, nothing more and nothing less.

Once I had completed a comprehensive list of learning objectives, I created a weighted list of topics by grouping the learning objectives into logical clusters. In practice, this involved building out a large spreadsheet of learning objectives, then going through each learning objective and tagging it with one or more topics. From there, I built out a weighted list of topics based on how often they appeared in the learning objectives.

Here is a list of topics that appear in the Pantheon WebOps Certification exam, in descending order of frequency:

1. Site Dashboard
2. (tie) Domains, WebOps Workflow
3. (tie) Autopilot, Workspaces, Terminus, Custom Upstreams
4. Mutidev, Version Control, Git, Automation
5. Security
6. (tie) HTTPS, Role-based Access Control
7. (tie) SFTP, Database, Integrated Composer, WAF, Runtime Matrix, File System, Quicksilver, Authentication
8. (tie) Configuration, Redirects, WebOps, Caching & Performance

This ranking makes intuitive sense. Pantheon fundamentally is a platform for running websites. Of course, the details of our Site Dashboard need the most focus. Connecting those sites to the rest of the world through a custom domain and then pushing changes through our workflow are a close second.

In the next blog post in this series, I will cover in detail some of the top topics that appear in the Pantheon WebOps Certification Exam.