dendrite/docs/development/coverage.md at beea2432e6144a98370138f8d3f6334c19a044bb

mirror of https://github.com/matrix-org/dendrite synced 2024-11-18 15:50:52 +01:00

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->
I was reading through the Dendrite documentation on
https://matrix-org.github.io/dendrite/development/contributing and
noticed the installation link leads to a 404 error. This link works fine
if it is viewed directly from
[docs/CONTRIBUTING.md](https://github.com/matrix-org/dendrite/blob/main/docs/CONTRIBUTING.md)
but this might not be very obvious to new contributors who are reading
through the [contribution
page](https://matrix-org.github.io/dendrite/development/contributing)
directly.

This PR is mainly a small re-organization of the online documentation
mainly in the
[Development](https://matrix-org.github.io/dendrite/development) tab
along with any links throughout the doc that may be impacted by the
change. This does not contain any Go unit tests as this does not
actually touch core dendrite functionality.

* [ ] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Kento Okamoto <kentokamoto@proton.me>`

2022-12-12 16:46:37 +00:00

4.1 KiB

Raw Blame History

title	parent	permalink
Coverage	Development	/development/coverage

To generate a test coverage report for Sytest, a small patch needs to be applied to the Sytest repository to compile and use the instrumented binary:

diff --git a/lib/SyTest/Homeserver/Dendrite.pm b/lib/SyTest/Homeserver/Dendrite.pm
index 8f0e209c..ad057e52 100644
--- a/lib/SyTest/Homeserver/Dendrite.pm
+++ b/lib/SyTest/Homeserver/Dendrite.pm
@@ -337,7 +337,7 @@ sub _start_monolith
 
    $output->diag( "Starting monolith server" );
    my @command = (
-      $self->{bindir} . '/dendrite-monolith-server',
+      $self->{bindir} . '/dendrite-monolith-server', '--test.coverprofile=' . $self->{hs_dir} . '/integrationcover.log', "DEVEL",
       '--config', $self->{paths}{config},
       '--http-bind-address', $self->{bind_host} . ':' . $self->unsecure_port,
       '--https-bind-address', $self->{bind_host} . ':' . $self->secure_port,
diff --git a/scripts/dendrite_sytest.sh b/scripts/dendrite_sytest.sh
index f009332b..7ea79869 100755
--- a/scripts/dendrite_sytest.sh
+++ b/scripts/dendrite_sytest.sh
@@ -34,7 +34,8 @@ export GOBIN=/tmp/bin
 echo >&2 "--- Building dendrite from source"
 cd /src
 mkdir -p $GOBIN
-go install -v ./cmd/dendrite-monolith-server
+# go install -v ./cmd/dendrite-monolith-server
+go test -c -cover -covermode=atomic -o $GOBIN/dendrite-monolith-server -coverpkg "github.com/matrix-org/..." ./cmd/dendrite-monolith-server
 go install -v ./cmd/generate-keys
 cd -

Then run Sytest. This will generate a new file integrationcover.log in each server's directory e.g server-0/integrationcover.log. To parse it, ensure your working directory is under the Dendrite repository then run:

go tool cover -func=/path/to/server-0/integrationcover.log

which will produce an output like:

...
github.com/matrix-org/util/json.go:83:											NewJSONRequestHandler				100.0%
github.com/matrix-org/util/json.go:90:											Protect						57.1%
github.com/matrix-org/util/json.go:110:											RequestWithLogging				100.0%
github.com/matrix-org/util/json.go:132:											MakeJSONAPI					70.0%
github.com/matrix-org/util/json.go:151:											respond						61.5%
github.com/matrix-org/util/json.go:180:											WithCORSOptions					0.0%
github.com/matrix-org/util/json.go:191:											SetCORSHeaders					100.0%
github.com/matrix-org/util/json.go:202:											RandomString					100.0%
github.com/matrix-org/util/json.go:210:											init						100.0%
github.com/matrix-org/util/unique.go:13:										Unique						91.7%
github.com/matrix-org/util/unique.go:48:										SortAndUnique					100.0%
github.com/matrix-org/util/unique.go:55:										UniqueStrings					100.0%
total:															(statements)					53.7%

The total coverage for this run is the last line at the bottom. However, this value is misleading because Dendrite can run in many different configurations, which will never be tested in a single test run (e.g sqlite or postgres, monolith or polylith). To get a more accurate value, additional processing is required to remove packages which will never be tested and extension MSCs:

# These commands are all similar but change which package paths are _removed_ from the output.

# For Postgres (monolith)
go tool cover -func=/path/to/server-0/integrationcover.log | grep 'github.com/matrix-org/dendrite' | grep -Ev 'inthttp|sqlite|setup/mscs|api_trace' > coverage.txt

# For Postgres (polylith)
go tool cover -func=/path/to/server-0/integrationcover.log | grep 'github.com/matrix-org/dendrite' | grep -Ev 'sqlite|setup/mscs|api_trace' > coverage.txt

# For SQLite (monolith)
go tool cover -func=/path/to/server-0/integrationcover.log | grep 'github.com/matrix-org/dendrite' | grep -Ev 'inthttp|postgres|setup/mscs|api_trace' > coverage.txt

# For SQLite (polylith)
go tool cover -func=/path/to/server-0/integrationcover.log | grep 'github.com/matrix-org/dendrite' | grep -Ev 'postgres|setup/mscs|api_trace' > coverage.txt

A total value can then be calculated using:

cat coverage.txt | awk -F '\t+' '{x = x + $3} END {print x/NR}'

We currently do not have a way to combine Sytest/Complement/Unit Tests into a single coverage report.

4.1 KiB Raw Blame History

4.1 KiB

Raw Blame History