With the Dev Portal Awards 2021 under way, in this episode of the Cherryleaf Podcast, we look at the characteristics of an award-winning developer portal.
What is your developer portal’s KPI score?
Transcript
Hello again, welcome to the Cherryleaf podcast. In this episode we’re going to look at the question how to create an award winning developer portal? Because it’s that time of year for the developer Portal Awards.
And back in episode 90, I talked about being one of the judges for the developer Portal Awards for 2020 and I’m a judge again for 2021.
We’re going to be looking at around about 50 different organisations who stepped forward to submit an application to be seen as the best developer portal for this year.
And so I thought it would be good to talk about the process of the award, the criteria by which we are going to judge, a little bit about who’s going to be judging the awards, the different categories, and so on, to help you if you are developing a developer portal. Yo think about some of the yardsticks, some of the measurements that you might use to assess whether what you’re creating is good, whether it’s applying best practice, and perhaps help you identify the areas where you can focus on to improve your developer portal.
So let’s talk a little bit about the awards themselves. They’re organised by Pronovix. And they have a website called devportalawards.org where you can, for next year, apply to be one of the competitors. You’ll see information on the different categories. You’ll also find information on the award ceremony that’s coming up in November and December. You can sign up for free. You can watch the awards being made alive at this free virtual event.
So the website says that the Dev Portal Awards are an opportunity to celebrate the work that goes into developer portals that are not only targeted at developers but also aligned to business needs and are operable by its maintainers.
And Pronovix has categorised 15 different best in class awards into three main categories.
And those are around the developer experience: What do the readers of the content get from the portal?
The second is around operability.
And the third is around business alignments. The business benefits from having a developer portal.
To help assess who’s going to be the winner in all of those 15 different categories, there is a jury or a collection of judges to look at the different categories.
And they have been split into three groups to look at 1/3 of the categories each.
And then from those winners of those different 15 categories, there’s going to be the selection of two winners. And also there’s going to be an award for a winner that’s coming from what they call the public community vote.
The winners will be announced at two virtual free gala events, and those are going to be on the 10th of November and 15th of December.
And you can register to attend that gala. As I mentioned, at devportalawards.org.
So let’s talk about the 15 categories that are available. And you can judge these as 15 different ways of measuring your own developer portal.
As I mentioned, Pronovix has split these into three main categories: developer, experience, operability, and business alignment.
So we’ll look at the different awards within those three main groups.
So there are two awards related to business alignment, one of which is the best API business model.
And this category is for portals that have found an innovative way to present their business model.
How they can manage to increase the perceived value of the APIs and the developer portal?
And the website describes it in this way and I’ll read this from what the website said:
Well explained API business models can help to convince future customers. Not only do they reflect how organisations communicate around specific topics. How easy is it for users to find out what they actually need to know when working with the developer portal? Business models can comprise pricing, outlines, share uptime, status and maintenance news, and they can take care of legal documentation, accessibility and change indications.
And the second category in this business alignment main section is best internal developer portal. Developer portals that have been created to fulfil the need for collaboration, discoverability and interface sharing internally within an organisation.
Second main category is operational intelligence. And there are four awards within this area. So one is best developer portal for alternatives to REST APIs. Another is best use of monetization, and the other is API gateway integration.And the 4th is editorial experience.
So all these around the operation of having a developer.
Alternatives to rest APIs. Most APIs these days are rest APIs. This is looking at more recent trends towards low code and no code solutions for integrating with different interfaces.
So looking at what developments there have been in that area.
The best use of monetization is portals that document and use API monetization in an innovative and effective way.
And API gateway integration relates to portals that use one or more of their own or a third party gateway. To provide additional levels of access control and metrics to their API calls.
And the 4th one is editorial experience. How easy is it for editors to create and maintain content?
But the bulk of the awards, and probably the ones that are of most interest.
It’s around the main category of developer experience.
So how do you measure and what awards are available in the field of developer experience?
So the awards that are available under developer experience.
Let’s go through those.
The main one, unsurprisingly, relates to API reference documentation. Developer portals that improve on API reference documentation. And those that integrate that reference documentation with try it out features and sandbox functionality. And portals that really provide high levels of trust with their APIs.
So that’s one of the main categories.
The second one is best findability of products in a developer portal and as developer portals get more complex as organisations add more and more APIs.
It can be more and more important to make that information findable. The right API findable.
So this is a category where it’s looking at developer portals that have easy to discover tools or structures that allow users developers to find the right API.
The third category is the best accessible developer portal, so ones that look at accessibility in the content make their content accessible to as many people as possible by considering people that have different levels of disability or severity of impairment.
Fourth category is best developer dashboard. Portals that take specific care of the logged in developer user journey and provide a dashboard that makes the developer experience better by providing informational metrics and the status of applications to submit products and the like.
So how else can you measure the developer experience?
So we’ve looked so far API reference we’ve mentioned findability. We’ve mentioned accessibility. We’ve mentioned developer dashboard. There are a few more that we can also use, and where there are awards for API portals, developer portals, that have these different features and capabilities.
So the next one is best international localised developer portal.
Now it’s true that most API content is written in English, but some are localised and this is an award for portals that implement content strategy mechanisms to accommodate an international audience cultural diversity.
For example, taking on board different languages or legal issues.
Another category is best design portals that have found that balance of usability, content and aesthetics and present all the aspects of the API in a well structured, understandable way.
And the final ree awards in this category are one is best new developer experience innovation.
So which are the portals that are leading the way by doing things differently to make the developer experience better, either for a specific niche subgroup or for all types of developers.
And the next word is best onboarding.
Which developer portals have good, clear and transparent steps to enable the developers that users to register to get access, and then to understand what the APIs do, how they work, and how they, as developers, can get going to start integrating and using the information and where can they go for extra information, extra resources.
And the final category or award within the developer Experience section is best community spotlight and outreach portals that have interesting creative solutions for showing developers how their work is appreciated.
Portals with great community sections where developers can share their knowledge and build upon that.
So there has been an increase in the number of awards compared to last year.
So from this list of these 15 we will pick 15 award winners and the nominees for the best in class awards will then go through to the competition for the best overall developer portal award. And those are split to being the best overall enterprise developer portal.
So this is for the ones that are large organisations that have many API’s.
An award for best overall SME developer portal. So for organisations that have perhaps only one API.
And finally, an overall world for the best developer portal community prize.
And if you’re thinking wow, that’s a lot to take in.
Well spare thought for us, poor judges. So in total there were originally 52 entries.
And a couple of people dropped out. There’s now 49 entries and there are 11 judges that have been split into three groups to assess all those different categories.
So the group I’m in the judges are myself and Tom Johnson. You may know from the tech writing community he works at Google. He has the idratherbewriting.com blog and podcast. Bob Watson PhD, who works at Amazon. And Lucas Rosenstock, who is based in Germany, and he is an API consultant.
So the category that our group is going to be looking at his best overall developer Portal award of course, the best onboarding the best developer dashboard. The best API reference and the best editorial experience.
So how are we going to judge?
Those different categories? Well, let me share that with you.
Now the judging process is still going on at the moment. What we have got to is a stage of selecting a shortlist and then we will rate the candidates on the shortlist on a scale of 1 to 5, have another meeting to see which sites have scored the best, talk about those and see if we can identify a winner, talk about our thoughts of what we liked.
And most of the sites are very good. We’ve got to select the best from a collection of very good sites.
Talking about how we’re going to do this, we came up and agreed a criteria for doing this, and it’s a criteria you might be familiar with yourself. Because basically it’s the same criteria that is used in content strategy. And you might have heard of something called the content strategy honeycomb, and it’s also based on the quality checklist that IBM published many, many years ago for assessing the quality of technical documentation. Now usually there’s about eight or nine different criteria that you can use.
We have agreed that we’ll consider six main criteria because things like legal accuracy we’re not in a position to judge externally.
So we are looking at it with a view of
Is the content usable?
Is the content findable?
Is the content complete?
Is it practical?
Is it attractive and is it cohesive?
So what do we mean by those? Well, let’s explain that a little bit more in terms of attractiveness. It’s how is the information presented. It’s a use of white space. Is the colour contrast good? Have they used sentence case or if they got lots of capitals, how easy is it to understand them?
And linked with that, are there sandboxes API explorers, so Postman collections, ways of actually making test calls to the API and getting a response back.
And with that information design aspects, is it findable, can you navigate around to different areas or not certain something? Can you jump to the conceptual information or the task based information.
And then on completeness how extensive? How detailed is a reference? Does it cover the edge cases? Does it cover all of the parameters?And related to completeness, are there code samples? How many code samples which languages? Are there sample requests?
And cohesive, how well does it connect between the reference content and the task waste content? The conceptual information? Does it jump you off to a different website or all within the same website?
And the ones that, from my perspective, are standing out are the ones where it looks like they’ve done more than just information design.
So a lot of the sites are structured well with content by different information types organised in good, clear, logical ways. There’s navigation for you to move around.
But there are others, the ones that, for me, stand out the ones where it looks like they’ve done some usability testing, I’ll talk a little bit more about that when I talk about the onboarding aspect,
And they’ve also set the reference content in the context of jobs to be done. So the tasks that somebody wants to achieve and another aspect is they have used clear, simple English that’s easy to understand.
The ones that weren’t so good, the main issues were around usability sites that used everything in capitals or had white text on a black background by default, without an obvious way of converting it to being black text on a white background.
Sites where the content was crowded together. There wasn’t a great use of white space and where the fonts were basically too small and then for some others not so much on the reference content.
But elsewhere they were wordy.
Another category that we looked at was on onboarding, So what does a great on boarding developer portal look like? So we have some guidance on this from Pronovix on the websites. It said onboarding is about how you can attract and convert new users through your developer portal and make sure your onboarding developers can easily onboard and integrate.
This category is for developer portals that clearly show what their APIs are about, how they work, how developers can start integrating, and where they can find resources.
I’ve mentioned before we are in the process of developing an onboarding course, so I also considered the definition from Chrystal Higgins of Google, who argues that onboarding is guided interaction.
In her words, onboarding is support in the context of a user interaction with a product instead of a one size fits all piece of instruction at the beginning of the onboarding experience.
So the questions Pronovix suggested we consider were,
How can users test the integrations?
Can they evaluate them before or after logging in?
How easy is it for users to register?
How can they start using the API that they’ve selected and what documentation types are available to help them find out how everything work?
I also considered the developer journey, the user journey and how simple and straightforward is it to get to a result time to the first “Hello world”, as it were.
And again it was quite interesting how some sites really looked like. There had been some usability testing and optimising to make the user journey as seamless as possible. They developed a workflow they had created a clear starting point.
And some of the sites also used progressive disclosure so that they showed that there were six steps and you were at step one and then when you did that particular task you were at Step 2 and so on.
So it helped you get a sense of how quickly you would get through the onboarding process or registration process to getting to a result, to using it.
There were sites where you would receive a series of emails. And again there was quite a difference in the quality or the level of detail and usefulness of the onboarding emails. Some would come through in just a few minutes with other sites. They may not appear for maybe 24 hours, even longer than that.
And with the onboarding you saw portals where they set the API in the context of jobs to be done. Tangible real things.
Now the ones that stood out in this area were developer portals that had some connection, I would argue, with the Internet of Things: car manufacturers and other physical hardware that was being driven by an API. There seemed to be with those a greater consideration of the user journey and making it clear how you could use the API to do so.
I thought really good signs within the sites that stood out were they again used to clear English again, they were easy to navigate and another aspect they used graphics to show the process.
And that could be flow charts or illustrations where you could see what needed to be done and where you were within the context of the task in the onboarding and also outside of the on boarding. Now it may be that those companies that are not necessarily aimed at developers or have a wider audience.
These ones that I mentioned of Internet of Things, maybe because they have to deal with less technical audience as well as a technical audience. That they’ve spent this time on making the onboarding as streamlined and simple as possible.
The ones that were less good and there were some where the onboarding process was quite painful.
The main issue was around registration. There were registration processes that required captures, specific password requirements and the length of the password and certain characters that had to be two factor authentication.
Then questions as you signed up about your use for the system, which didn’t seem to be particularly related to the use of the product, but more for marketing purposes.
And for one site it took 15 minutes to get through that process. Other sites myself and others just couldn’t log in for one site for myself. Being based in the UK, I put in a UK post code and the site said the UK post code was not valid even though you could select the country as the UK it wouldn’t accept the UK postcode.
There were others that took you straight into a walkthrough a series of screens you could not escape from those.
There was another where it took you to a dashboard – essentially an empty screen at empty state.
At the other extreme, we had sites where they would give you countless choices as soon as you landed through the onboarding, onto a dashboard or into the documentation.
So those factors also affected those sites that had good and great developer dashboards. Essentially for those the same factors came up in assessing which ones of those were good:
How many choices was there a natural single starting point?
Was it just an empty screen?
Was the content clear?
Was it attractive?
Was there good use of white space?
So if we go back to the question at the beginning.
What makes an award winning developer portal?
I would argue the key factor and what will probably show in the winner or winners, I think we’ll find out when I discuss this with the other judges, are the sites where they have done usability testing and have considered what the user journey is the developer journey. And where there are bottlenecks where things are unclear, they have removed those and made it as streamlined and as simple as possible.
And that is really the extra over and above good excellent information design that many of the other sites had. Where it was easier to navigate.There was good use of white space. The sentences were clear and understand. It was a cohesive site. Where there was a connexion with the reference content, the conceptual content, and the task based content.
So with going through the process at the moment it will be interesting to see who ends up as the winners in this.
There’s these three different judging groups, so we’ll talk about the winners that come through to select the two overall winners for the SMEs and for the corporate enterprise level ones.
And as I said, the awards ceremony will be in November and in December, let me tell you the dates again: 10th of November and 15th of December.
So I hope you have the opportunity to attend those.
It’s been again, as with last year, interesting because we write and improve developer portals for our clients, to look at these excellent sites and then compare it and reconsider how we approach doing portals for clients as well,
So thank you for listening. Do give us a rating if you have an opportunity on Apple Podcasts because that helps other people find the site.
If you want to speak to us, it’s info at cherryleaf.com. I’ll talk to us I should say. And if you’re interested in what we do, it’s cherryleaf.com.
Thank you for listening.
Leave a Reply