Every WordPress release party includes a phase dedicated to community testing. Contributors from around the world come together synchronously, run through a set of testing scenarios, and report the results in the #core Slack Slack is a Collaborative Group Chat Platform https://slack.com/. The WordPress community has its own Slack Channel at https://make.wordpress.org/chat/ channel. It’s a great tradition and one of the most accessible ways to contribute to a WordPress release.
But as the process has evolved and the project has scaled over the years, some repeating pain points have become clear that are worth addressing. With WordPress 7.0 set to be released during Contributor Day at WordCamp Asia on April 9th, there’s a valuable opportunity to experiment with a large group of contributors in the same room. This feels like the right moment to ask the community for ideas.
This post is a call for feedback aimed at improving how release party testing works, both for those participating synchronously on release day and for contributors who want to keep testing after the party concludes.
What Makes Release Party Testing Great
Before getting into the areas to improve, it’s worth acknowledging why this part of the release process matters and what it gets right.
Release party testing is one of the easiest ways to get involved as a contributor. You only need to know how to update a WordPress site and run through a few basic scenarios that are outlined in the Core Core is the set of software required to run WordPress. The Core Development Team builds WordPress. Handbook. There’s no deep technical knowledge required. That low barrier to entry is not something to be sacrificed in pursuit of more structured or rigorous testing. Lowering barriers to contribution is one of the most important things an open source Open Source denotes software for which the original source code is made freely available and may be redistributed and modified. Open Source **must be** delivered via a licensing model, see GPL. project can do, and any changes made here should make the process more approachable, not more intimidating.
With that in mind, here are some of the areas where there’s room to improve.
Current Pain Points
The channel can become difficult to follow
During the testing phase, the #core channel fills up quickly. For contributors whose first language isn’t English, a fast-moving channel can be especially overwhelming. And when off-topic messages or late test reports get mixed in, it becomes that much harder for everyone to keep up with what matters.
Testing scenarios lack release-specific context
A list of common scenarios to test is always shared. While these are frequently improved and are helpful as a starting point, they tend to be fairly generic (“test creating a new post”), without any guidance on what specifically changed in this release that a tester should be looking for or confirming. Without that context, it’s harder for testers to know where to focus their attention or what a successful test actually looks like for a given release.
It’s not always clear what was tested or how
Testing reports typically include details like installation type, method of updating, version of WP being updated from, PHP The web scripting language in which WordPress is primarily architected. WordPress requires PHP 7.4 or higher version, which is all useful. But there’s often less clarity about how specific things were tested. What steps did the tester follow? What browser were they using? Were they testing as an admin (and super admin), an editor, or a subscriber? That level of detail can matter a lot when trying to triage The act of evaluating and sorting bug reports, in order to decide priority, severity, and other factors. or reproduce a reported issue.
It’s hard to gauge the overall coverage of testing
When it’s unclear how many contributors are actively running through scenarios versus following along, it’s difficult to know how thoroughly the release has been tested. Without that visibility, it’s hard to identify gaps in coverage or know which environments and workflows still need attention.
Testing reports continue after the testing phase ends
The emcee running the release process announces when the testing portion has concluded and asks that only failures be reported going forward. Despite this, testing reports continue to come in, which disrupts the coordination work that follows. This is a process and communication challenge as much as anything, but it’s worth addressing directly.
There’s no clear path forward after the party ends
This is probably the most significant gap for contributors. Once the release party concludes, all of the testing feedback that was shared becomes another message buried within the Slack archives. There’s no clean summary of what was tested, what was found, and what still needs attention. And there’s no obvious next step for contributors who want to keep testing the release after it ships.
Testing shouldn’t stop at the release party. Every new major version of WordPress deserves continued testing across a broader range of environments, configurations, and workflows than any single party can cover.
Goals
With all of that in mind, here’s what improvements to this process should aim to accomplish:
- Make better use of everyone’s valuable time. Contributors are generous in showing up for release parties. The process should respect that.
- Test more scenarios that are relevant to a given release. Testers should know what changed and what to look for, not just what to click.
- Keep the release process on track. When off-topic messages or late test reports flood the channel, critical information can get lost at exactly the wrong moment.
- Make it more clear exactly what was tested and how. Structured reporting, even in a lightweight form, leads to more actionable results.
- Provide contributors with a clear path to continue contributing. The release party should be a starting point, not the end of the road.
- Connect release party testing to the broader testing cycle. Pre-release calls for testing cover a lot of ground, and the party should build on that work rather than exist in isolation from it.
- Leverage tools that contributors already know and use. Building on existing and familiar tools, such as Playground and the Test Reports plugin, reduces friction, helps standardize the process, and creates a more consistent experience.
None of these goals should come at the cost of making testing harder to participate in as that’s not a compromise worth making.
An Opportunity at WordCamp WordCamps are casual, locally-organized conferences covering everything related to WordPress. They're one of the places where the WordPress community comes together to teach one another what they’ve learned throughout the year and share the joy. Learn more. Asia
WordPress 7.0 is planned to be released during Contributor Day Contributor Days are standalone days, frequently held before or after WordCamps but they can also happen at any time. They are events where people get together to work on various areas of https://make.wordpress.org/ There are many teams that people can participate in, each with a different focus. https://2017.us.wordcamp.org/contributor-day/ https://make.wordpress.org/support/handbook/getting-started/getting-started-at-a-contributor-day/ at WordCamp Asia on April 9th. Having a large group of contributors physically present in the same room is a genuinely rare opportunity. Experiments that are difficult to run when 100% of participants are distributed become a lot more practical when a meaningful portion of the group is co-located. That makes the 7.0 release party an ideal time to create a plan and try out something new.
Summary
This post is an open call for any ideas. There’s no one perfect answer or solution, and it may take several releases to find what works best. What adjustments should be made to the testing phase of a release party? Are there practices from other open source projects or testing communities worth borrowing from?
Conversation Starters
Here are a few rough ideas to get the conversation started:
- A “How to Test During a Release Party” course on Learn WordPress: A short course or guide on Learn WordPress that walks first-time participants through what to expect, how to run through a scenario, and what makes a useful test report. Helps contributors show up prepared rather than figuring it out on the fly.
- Pair newer contributors with more experienced ones during the testing phase: The experienced contributor can help guide the approach, model what a useful report looks like, and answer questions in real time. This should result in higher quality results while naturally supporting onboarding at the same time.
- End each release party with a clear “what’s next”: Close out the testing phase with a specific, actionable next step for contributors who want to keep going.
- A designated release party testing board: A shared, visible checklist of the specific scenarios that need to be covered during the party. This is updated in real time as contributors claim and complete each one. Rather than everyone independently deciding what to test, the board makes it clear what’s been covered and what still needs attention, helping the group collectively ensure nothing important gets missed.
Please leave your thoughts and ideas in the comments!
Props to @annezazu, @jeffpaul, and @karmatosed for reviewing this post.
Claude (Sonnet 4.6) was used to create an initial draft from a detailed list of pain points, pros, cons, and surrounding context, and also to perform proofreading.
#feedback
