Introducing WP Test: the best tests for WordPress

Building for WordPress means accounting for endless variables: WordPress versions, plugin versions, plugin conflicts, host configurations, browsers, and all the ways people customize WordPress — post formats, custom post types, theme options, theme templates.

That variety is both the fun and the pain of shipping themes and plugins for mass distribution.

Today I want to start putting an end to one of those variables. Meet WP Test, the best tests for WordPress.

WP Test is an exhaustive set of test data to measure the quality and integrity of your plugins and themes. It builds on the WordPress Theme Unit Test data, plus three years of leading quality, support, and testing for 8BIT — years that uncovered corner cases in how people use WordPress.

Torque and Tom McFarlin have already covered WP Test (thank you, both), but I want to expand on why I created it.

Transparency

The WordPress Theme Unit Test data is a good place to start. My frustration was the lack of updates. It could go more than a year without attention, even as major WordPress releases shipped features with no examples or testing coverage. And when the data did update, nothing indicated what changed.

WP Test ships with a changelog listing what was added, removed, or changed in each release. The website has a release notes page with the same history.

Contribution

Contributing to WordPress is straightforward. Contributing to the test data isn't. WP Test lives on GitHub, so anyone can contribute by opening a pull request or filing an issue. Not comfortable with git or GitHub? There's a contact page on the website.

I left the word "comprehensive" out of the description on purpose. The test data is most likely missing coverage somewhere — that's where the community comes in. Tell me about a test that isn't covered and I'll add it to the data so we can all account for it in our products.

Life

Testing is tedious enough without mundane test data. Lorem ipsum works (it simulates text flow well), but it's boring.

I wanted content with life and character. There's no testing benefit to it. But it injects personality and fun into traditionally dry content, and that's reason enough.

Purpose

The test data isn't only funny — it's smart and helpful. Wherever possible, it references the WordPress Codex for best practices and points to function references. That integration will only deepen over time, lowering the barrier to fixing the issues the test data uncovers.

Consolidation

I had test cases scattered across three servers and ten WordPress installations. Remembering where everything lived was a chore, so I centralized it. Now everyone on the 8BIT team develops against the same WordPress scenarios, for internal and personal projects alike.

Community

I was consolidating these test cases anyway, so the choice was to keep them private for our team or release them for the greater good of the WordPress community. All it cost me was a domain name — a small price for continuity. Making it public was the obvious call.

The future

WP Test would be good enough as test data alone, but I have bigger plans for it. I'd love to see it grow into an all-encompassing suite of WordPress testing tools and data. Who knows if it will get there — at the very least, we can all share and contribute to an impressive set of test data now.

Have ideas, additional test cases, or feedback? Let me know. Let's make WordPress testing easier and more resilient together.