r/testanythingprotocol

I apologize for the rather extreme title.

I've recently come across TAP and think the idea is fantastic. The first use i saw of it was the BATS framework for testing bash. I've never heard of a comparable specification and all test report generators I've seen so far were in some way based on jUnit xml output.

After looking into it for a bit I wanted to suggest an implementation for go in the popular test runner gotestsum. To provide some arguments on why they should implement and include TAP, I looked into consumers and the actual benefits and report generators that are currently out there and I was disappointed to see, that there aren't many.

The concept is good, but there seems to be virtually no ecosystem and no real benefit to using it without one. Am I overlooking something? Do you think it's still worth it to try? Implementing a producer for go should be easy, but what then, if I can't do much with it?

Edit: After posting this I found node-tap, which seems promising as a consumer. But most projects won't want to install a tool that brings an entire interpreter and the largest supply-chain-attack-vector in the IT world with it.

Many test frameworks have ways to select tests via some kind of filter or predicate over test names.

• takes a substring: "You can also run a specific test by passing a filter: $ cargo test foo This will run any test with foo in its name."

• "you can use a filter expression to run selected tests"

• "-run regexp Run only those tests, examples, and fuzz tests matching the regular expression."

• : "With Gradle’s test filtering you can select tests to run based on: (qualified names, simple names, or globs...)"

• allows testing with the grep, fgrep and invert options.

Each of these has their own bespoke conventions for test filtering.

It'd be nice if a developer looking at some output in a TAP test dashboard could come up with a filter to rerun the tests they want.

If one is crafting a testing framework as a TAP producer, any recommendations on how to allow test filtering / selection based on names visible in test output?

iiuc, doesn't impose any structure on test names. Are there any prevailing conventions around subtest names that might be useful in filtering?