mirror of
https://github.com/matrix-org/dendrite
synced 2024-11-18 07:40:53 +01:00
a97b8eafd4
* Use a fork of pq which supports userCurrent on wasm * Use sqlite3_js driver when running in JS * Add cmd/dendritejs to pull in sqlite3_js driver for wasm only * Update to latest go-sqlite-js version * Replace prometheus with a stub. sigh * Hard-code a config and don't use opentracing * Latest go-sqlite3-js version * Generate a key for now * Listen for fetch traffic rather than HTTP * Latest hacks for js * libp2p support * More libp2p * Fork gjson to allow us to enforce auth checks as before Previously, all events would come down redacted because the hash checks would fail. They would fail because sjson.DeleteBytes didn't remove keys not used for hashing. This didn't work because of a build tag which included a file which no-oped the index returned. See https://github.com/tidwall/gjson/issues/157 When it's resolved, let's go back to mainline. * Use gjson@1.6.0 as it fixes https://github.com/tidwall/gjson/issues/157 * Use latest gomatrixserverlib for sig checks * Fix a bug which could cause exclude_from_sync to not be set Caused when sending events over federation. * Use query variadic to make lookups actually work! * Latest gomatrixserverlib * Add notes on getting p2p up and running Partly so I don't forget myself! * refactor: Move p2p specific stuff to cmd/dendritejs This is important or else the normal build of dendrite will fail because the p2p libraries depend on syscall/js which doesn't work on normal builds. Also, clean up main.go to read a bit better. * Update ho-http-js-libp2p to return errors from RoundTrip * Add an LRU cache around the key DB We actually need this for P2P because otherwise we can *segfault* with things like: "runtime: unexpected return pc for runtime.handleEvent" where the event is a `syscall/js` event, caused by spamming sql.js caused by "Checking event signatures for 14 events of room state" which hammers the key DB repeatedly in quick succession. Using a cache fixes this, though the underlying cause is probably a bug in the version of Go I'm on (1.13.7) * breaking: Add Tracing.Enabled to toggle whether we do opentracing Defaults to false, which is why this is a breaking change. We need this flag because WASM builds cannot do opentracing. * Start adding conditional builds for wasm to handle lib/pq The general idea here is to have the wasm build have a `NewXXXDatabase` that doesn't import any postgres package and hence we never import `lib/pq`, which doesn't work under WASM (undefined `userCurrent`). * Remove lib/pq for wasm for syncapi * Add conditional building to remaining storage APIs * Update build script to set env vars correctly for dendritejs * sqlite bug fixes * Docs * Add a no-op main for dendritejs when not building under wasm * Use the real prometheus, even for WASM Instead, the dendrite-sw.js must mock out `process.pid` and `fs.stat` - which must invoke the callback with an error (e.g `EINVAL`) in order for it to work: ``` global.process = { pid: 1, }; global.fs.stat = function(path, cb) { cb({ code: "EINVAL", }); } ``` * Linting
81 lines
3.3 KiB
Markdown
81 lines
3.3 KiB
Markdown
## Peer-to-peer Matrix
|
|
|
|
These are the instructions for setting up P2P Dendrite, current as of March 2020. There's both Go stuff and JS stuff to do to set this up.
|
|
|
|
|
|
### Dendrite
|
|
|
|
- The `master` branch has a WASM-only binary for dendrite: `./cmd/dendritejs`.
|
|
- Build it and copy assets to riot-web.
|
|
```
|
|
$ GOOS=js GOARCH=wasm go build -o main.wasm ./cmd/dendritejs
|
|
$ cp main.wasm ../riot-web/src/vector/dendrite.wasm
|
|
```
|
|
|
|
### Rendezvous
|
|
|
|
This is how peers discover each other and communicate.
|
|
|
|
By default, Dendrite uses the IPFS-hosted websocket star **Development** relay server at `/dns4/ws-star.discovery.libp2p.io/tcp/443/wss/p2p-websocket-star`.
|
|
This is currently hard-coded in `./cmd/dendritejs/main.go` - you can also use a local one if you run your own relay:
|
|
|
|
```
|
|
$ npm install --global libp2p-websocket-star-rendezvous
|
|
$ rendezvous --port=9090 --host=127.0.0.1
|
|
```
|
|
|
|
Then use `/ip4/127.0.0.1/tcp/9090/ws/p2p-websocket-star/`. We'll probably run our own relay server at some point.
|
|
|
|
### Riot-web
|
|
|
|
You need to check out these repos:
|
|
|
|
``
|
|
$ git clone git@github.com:matrix-org/go-http-js-libp2p.git
|
|
$ git clone git@github.com:matrix-org/go-sqlite3-js.git
|
|
```
|
|
|
|
Make sure to `yarn install` in both of these repos. Then:
|
|
|
|
- `$ cp "$(go env GOROOT)/misc/wasm/wasm_exec.js" ./src/vector/`
|
|
- Comment out the lines in `wasm_exec.js` which contains:
|
|
```
|
|
if (!global.fs && global.require) {
|
|
global.fs = require("fs");
|
|
}
|
|
```
|
|
- Add the diff at https://github.com/vector-im/riot-web/compare/matthew/p2p?expand=1 - ignore the `package.json` stuff.
|
|
- Add the following symlinks: they HAVE to be symlinks as the diff in `webpack.config.js` references specific paths.
|
|
```
|
|
$ cd node_modules
|
|
$ ln -s ../../go-sqlite-js # NB: NOT go-sqlite3-js
|
|
$ ln -s ../../go-http-js-libp2p
|
|
```
|
|
|
|
NB: If you don't run the server with `yarn start` you need to make sure your server is sending the header `Service-Worker-Allowed: /`.
|
|
|
|
TODO: Make a Docker image with all of this in it and a volume mount for `dendrite.wasm`.
|
|
|
|
## Running
|
|
|
|
You need a Chrome and a Firefox running to test locally as service workers don't work in incognito tabs.
|
|
- For Chrome, use `chrome://serviceworker-internals/` to unregister/see logs.
|
|
- For Firefox, use `about:debugging#/runtime/this-firefox` to unregister. Use the console window to see logs.
|
|
|
|
Assuming you've `yarn start`ed Riot-Web, go to `http://localhost:8080` and wait a bit. Then refresh the page (this is required
|
|
because the fetch interceptor races with setting up dendrite. If you don't refresh, you won't be able to contact your HS). After
|
|
the refresh, click Register and use `http://localhost:8080` as your HS URL.
|
|
|
|
TODO: Fix the race so we don't need multiple refreshes.
|
|
|
|
You can join rooms by room alias e.g `/join #foo:bar`.
|
|
|
|
### Known issues
|
|
|
|
- When registering you may be unable to find the server, it'll seem flakey. This happens because the SW, particularly in Firefox,
|
|
gets killed after 30s of inactivity. When you are not registered, you aren't doing `/sync` calls to keep the SW alive, so if you
|
|
don't register for a while and idle on the page, the HS will disappear. To fix, unregister the SW, and then refresh the page *twice*.
|
|
|
|
- The libp2p layer has rate limits, so frequent Federation traffic may cause the connection to drop and messages to not be transferred.
|
|
I guess in other words, don't send too much traffic?
|
|
|