A more human way to handle “not-approved” themes in the WordPress Theme Review Team

The WordPress Theme Review Team (TRT) is the team that reviews and approves WordPress themes for inclusion in the wordpress.org themes repository.

Anyone can submit a theme for review and inclusion at https://wordpress.org/themes/upload/ (make sure it meets the requirements before you do!).

Also anyone can join the Theme Review Team. See Become a Reviewer if you’d like to join.

The Theme Review Process

The TRT reviews themes based on a set of guidelines. These mainly focus on code security, required support for core WordPress functionalities and best practices.

It is worth noting that reviewers do not judge the visual quality of the design.

So yes, an objectively ugly theme can be approved as long as is it properly coded. But who can define “objectively ugly” anyway?

Reviewers are not bouncers that will just cold-heartedly reject anything that is not up to the standards.

What most reviewers do not look like.

They will take time to educate and help authors to get their themes up to the standard. The end goal is still to get the theme approved in the repository.

Most of the time, the review takes a few days. The reviewer points some issues and gives the author a few days to update the theme, until it is ready for approval.

Though the team is run by volunteers who contribute their free time to this task. The team is not that big, and the task is huge.

For this reason, the wait time in the review queue is pretty long. Currently a new theme takes around two months to reach the top of the queue.

The team needs theme authors to do their part in the process. They should make sure they follow the guidelines and best practices before submitting their themes.

This is why reviewers will occasionally close the ticket as “not-approved” if the theme has a lot of issues and are really far from meeting the requirements.

This way they can move on to review other themes that are sitting in the queue.

The dreaded “not-approved” trac ticket update.

A more human way to deliver bad news

This leads to a pretty delicate topic recently pointed out by @poena on Twitter:

The issue is two-folds: it is certainly not easy for authors to receive the message. It is also not easy for some reviewers to deliver it.

Let alone the fact that many reviewer don’t have English as their first language.

Some will hide behind cold, robotic-like statements: “Theme has many issues => resolving ticket as not-approved”. Others would rather dodge the topic and just say “these are the rules, people should just deal with it”.

I don’t think either is a good answer.

So I thought it would be a good idea to work on a canned introduction for tickets we resolve as not-approved.

It should hopefully help to mitigate the feeling of rejection this can cause.

This would be especially helpful for new authors who don’t have any theme in the repository yet. Many do not know or understand the ins and outs of the theme review process.

It is already hard to have a theme rejected after two months of anticipation; and we don’t want it to be any more harsh.

We also don’t want authors to feel rejected as designers and developers, or to be discouraged and stop trying.

Therefore it should be clear that:

  • Closing a ticket as not-approved is only rejecting the current version of the theme. Not the theme altogether, not the author themselves.
  • The guidelines we base reviews on are only about code quality, security, best practices and usability.
  • A review is NOT a judgement of the design quality. This is often the most important in the eyes of authors. We understand some authors might be emotionally attached to it.
  • Closing tickets with more than a set number of issues (currently 3) is not to be harsh. The goal is to save reviewers time and prevent the queue to be even slower. The idea is to encourage authors to acknowledge the guidelines. Authors should also self review their own themes to help make the overall review process faster for everyone.
  • A “not-approved” ticket is not a hard rejection, but an encouragement to improve. TRT members are willing to help and to answer any question about requirements, expectations and best practices.
  • We don’t expect perfect themes submissions. When there is only a reasonable number of issues, reviewers will help and work closely with authors. The goal is to end with a the theme that is is ready to go into the repository.

A canned introduction to carry the message without being too harsh

With this in mind I suggest we work on a canned message that carries these ideas. Every reviewer can then simply copy/paste before listing issues and closing tickets.

It doesn’t have to be too long or complicated. But it should be nice and communicate the bad news in a not too harsh way.

Hopefully this should help authors to deal with the bad news.

It should help reviewers to deliver this bad news. This is not always an easy task either. Even more so for reviewers who are not native English speakers, as @poena also pointed out.

Here’s my attempt at such an introduction to be used in tickets closed as not-approved: https://docs.google.com/document/d/1IoePkAt33eWDTsV8Xcp-gHM4_pgMnibSPizVP2MoffM/edit?usp=sharing

It is ready to copy/paste in trac format, feel free to use it during your reviews (don’t forget to replace [AUTHOR] with the author’s name!).

I’m sure we can make it even better, so any suggestion is welcome. The Google docs link above allows you to make suggestions, so please go ahead!

This topic was discussed on Slack during the team meeting on December 12th, 2017: https://wordpress.slack.com/archives/C02RP4Y3K/p1513099934000138

Share the love!