FediverseTests

Checks a Fediverse server against the specifications, from the outside, the way another server sees it.

Point it at any account on any server. It reads what that server publishes about the account and compares it with what WebFinger (RFC 7033), ActivityPub and ActivityStreams actually require, then sends the handful of deliveries an inbox is supposed to refuse.

php run.php @[email protected]

That is the whole setup. No dependencies, no configuration file, no database, no cooperation from the server under test - it never has to know it is being tested.

• PHP 8.1 or newer, with curl, json and openssl. All three ship with a default PHP.
• Outbound HTTPS to the server you are testing.
• An existing account on that server to test against. Any public account will do, including your own.

Nothing is installed and nothing is written. The tool is a directory of PHP files you can run from anywhere.

php run.php @[email protected]            # everything
php run.php [email protected]             # the @ is optional
php run.php @[email protected] --no-colour        # for a log or a CI job
php run.php @[email protected] --no-deliveries    # reads only, sends nothing

The exit code is 0 when nothing failed and 1 when something did, so it drops into CI as it is.

Three outcomes:

• PASS - the server did what the specification says.
• FAIL - it did something the specification does not allow. The line under it says what was expected, what arrived, and which part of which document says so.
• SKIP - nothing was proved. An optional feature the server does not offer, a collection it keeps private, or a check that needed something an earlier one could not find. A skip is never a complaint.

It creates nothing. Every check is either a read, or a delivery the server is meant to turn away.

The four inbox checks each POST one activity that is deliberately invalid - unsigned, signed with the wrong key, carrying a digest that does not match the body, or dated a day ago. Each one is a Delete naming an object on a reserved .invalid domain, so even a server that wrongly accepted one would go looking for something it has never seen and do nothing. A correct server ends the run exactly as it started it.

Even so, --no-deliveries is the polite way to test a server that is not yours. It runs the read-only checks and sends nothing at all.

WebFinger (RFC 7033) - the account resolves; the answer is typed as a JRD; the subject is the account that was asked about; a self link points at the ActivityPub actor over https; a handle nobody holds is a 404 rather than a cheerful 200; a request naming no resource is refused; a handle belonging to another host is not answered for.

Actor (ActivityPub §3, §4) - served as ActivityPub JSON; the id agrees with the address it was fetched from; a type other servers understand; an inbox and an outbox; preferredUsername matches the handle; a public key that parses, owned by this actor; and a browser asking for the same address gets a readable page rather than a download.

Collections (ActivityPub §5) - the outbox, followers and following are collections; the outbox either lists its items or says where its first page is, and that page can be fetched; a total, where given, is a number. A collection kept private is a skip, not a failure - that is a server's choice to make.

Objects (ActivityPub §3.1) - something the account has published is followed to its own address: it can be fetched there, it names itself with the address it was fetched from, and it is attributed to somebody on the same host. An id here is a promise that whoever holds it can be dereferenced - every reply, boost and quote elsewhere is a server following one of these back - and an account publishing ids nobody can fetch is one nobody can thread, which looks fine from the inside.

Inbox (ActivityPub §B.1) - a delivery is refused when it carries no signature, when it is signed by a key other than the one it claims, when its digest does not match its body, and when it is dated a day ago. And the inbox does not hand its contents to a stranger.

These say a delivery was refused, not why. A server can refuse for the wrong reason and still pass. A server that accepts has no defence at all.

NodeInfo - how a server introduces itself to directories and relays: the well-known document points at a real one, which declares its version, names its software, and says it speaks ActivityPub. Optional, so a server without it skips - but everything downstream reads what it does say as fact.

  FAIL  the answer is typed as JRD
        Content-Type: expected one of application/jrd+json, application/json,
        got "application/activity+json"
        RFC 7033 §10.2

Every failure names the document and section it comes from, so the argument is with the specification rather than with this tool. If you think a check is wrong, it may well be - open an issue with the server and account you ran it against.

A skip is worth reading too. "This account has published nothing yet" means a whole group proved nothing, and testing against an account with some history in it will tell you more.

A check is a name, the part of the specification it comes from, and a closure that either returns or throws:

$suite -> check('Actor', 'the actor carries an inbox', 'ActivityPub §5.2', function (Target $target): void {
    Assert::present($target -> actor() -> at('inbox'), 'inbox');
});

Put it in the right file under checks/. Target does the discovery once and caches it - webfinger(), actorURI(), actor(), actorEndpoint('outbox') - so twenty checks cost one lookup. Assert::skip('why') where nothing can be proved.

Two rules for anything added here:

1. Never create anything on the server under test. Read, or send something that must be refused.
2. Cite the specification. A check that cannot point at a line in a document is an opinion about how servers ought to behave, and this tool is not for those.

It does not drive the server. Nothing here asks a server to post something, follow someone, or accept a delivery, because doing that needs an adapter per application - which is where testing efforts in this space have historically run aground. Everything here works against any server, unchanged, because it only ever asks what the server already publishes.

So it will not tell you whether two servers can talk to each other. It tells you whether one of them is holding up its end of the specifications.

MIT.