1. Adds a user-agent string to the node announcement. This lets us keep
track of what software version everyone is running. Especially useful
to see what percentage of the network has upgraded to a new release.
For old nodes, this defaults to the string `/radicle/`.
The user agent format is roughly based on BIP 0014.
2. Advertize protocol version in node announcement.
By advertizing the protocol version, nodes can decide to only connect to
peers with a compatible version. The default and current version is `1`.
We bundle these two changes as they both modify the node announcement
and node database.
3. Swap the order of the fields in announcements to make the message
extensible.
This prevents potential memory leaks. We also ensure that sends fail
instead of blocking, in case the channels are full.
Additionally, we add some metrics to report on channel size.
Previously, our advertized inventory was not always correct: when we
stopped seeding a repository, it was still advertised as part of our
inventory until node restart, because the `Storage` type doesn't have
access to seeding policies.
In this change, we remove the concept of inventory from `Storage` and
make the authoritative place for it the routing table in our database.
To make this work, we have to add our node to the database on profile
creation, to not violate the foreign key constraint on the routing
table. Hence, the tests are changed to include our alias which is now
always available.
`--env` is a podman-specific feature. While it is clear that podman is a
requirement here, switching these to build arguments better expresses
intent.
These values are meant to be configurable at build time, which is
exactly what build arguments are for, while environment variables are
meant to persist in the image built.
Signed-off-by: Yorgos Saslis <yorgos.work@proton.me>
```
--- Expected
++++ actual: stdout
1 - branch 'master' set up to track 'origin/master'.
1 + Branch 'master' set up to track remote branch 'master' from 'origin'.
Exit status: 0
```
As per @fintohaps' suggestion, this gets the tests passing again on git 2.34.
Co-Authored-By: fintohaps <fintan@monoidal.tech>
Signed-off-by: Yorgos Saslis <yorgos.work@proton.me>
Instead of requiring to specify a commit to obtain a file through the
/raw route, this commit allows to prefix `/head` to get the file at the
current canonical head commit.
This is a change to the `config.json` file which removes certain options
that are best not touched and changes some options to make them easier
to work with.
1. We change the default journaling mode to "wal" and remove the config
option.
2. We remove the option to set the peer connection "target", as it's not
a good idea to set this to a different value, as it affects the
network.
3. We combine seeding policy & scope, since there's no need to set the
scope when the policy is "block". In that case, we always want to
block "all" remotes. The new policy configuration has the following
schema:
{ seedingPolicy: { default: "block" | "allow", scope?: "all" | "followed" } }
The "scope" key is not used when "default" is set to "block".
4. We add an `extra` field to the node config for options that are not
recognized. We use this to warn the user.
We use the *staggered broadcast* technique when relaying gossip messages
to reduce message amplification. In our basic simulation, it brings
amplification down from 3.5 to 1.5.
The idea is simply to accumulate gossip messages and relay them after a fixed
interval of time. This has two effects:
1) Old messages that haven't been sent yet are replaced by newer ones,
so only one message is sent after the delay.
2) Because nodes have their own intervals, messages are not all sent at
the same time, which reduces the chance of messages crossing paths.
For now we only turn this on for inventory messages, since this is the
least likely to be noticed by users, and also the most problematic
currently.
After the Dockerfiles for radicle-node and radicle-httpd have been
removed, these files can also be removed.
Signed-off-by: Yorgos Saslis <yorgos.work@proton.me>
Ensures dead connections are not kept around longer than necessary.
Note that the ping/pong mechanism is more expensive and checks that the
session is active and healthy, while TCP_KEEPALIVE is lower level,
cheaper, and checks that the connection is still alive.
On FreeBSD systems, the file limits are of type `i64` in comparison to
`u64` on Linux systems.
Also fix `rad web` command name on Windows and FreeBSD.
Co-authored-by: cloudhead <cloudhead@radicle.xyz>
Co-authored-by: Shawn Webb <shawn.webb@hardenedbsd.org>
These make the Debian package checker, lintian, complain less. This
makes any other warning and errors stand out better.
Signed-off-by: Lars Wirzenius <liw@liw.fi>
We don't actually need to use it, and it doesn't help in any way. In
fact, it makes things more difficult, as it insists on using the
Debian-packaged cargo.
Signed-off-by: Lars Wirzenius <liw@liw.fi>
In the case of an out-of-sync local node, we were showing the wrong tip.
Additionally, mark your tip as unannounced if it isn't announced. This
isn't fool-proof, but can be useful to debug certain issues.
From empirical testing, the `FETCH_TIMEOUT` can be set to a lower value of 3s.
For a repository that is about 300MiB, the process will successfully send all
packfile bytes and the references will be written.
The `--timeout` option is also used in the `git-upload-pack` process for good
measure.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `join` method is equivalent to the custom loop that was written for joining
the `reader` thread in upload-pack.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Introduce an `UploadPack` type for keeping track of relevant data from
upload-pack events and displaying progress to the terminal.
It keeps track of the number of remotes that are being uploaded to as well as
the throughput of transmitted data.
The `Progress` events are logged rather than being displayed since they don't
provide any useful details to the user.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Breaking change: when writing to the node socket, command names should
be specified via the `"command"` field instead of the `"type"` field.
This was changed to be more readable.
Upgrade the family of `gix` crates and resolve any breaking API changes.
Fixes RUSTEC-2024-0335[[0]].
[0]: https://rustsec.org/advisories/RUSTSEC-2024-0335.html
Co-authored-by: Fintan Halpenny <fintan.halpenny@gmail.com>
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
* Remove empty `inherit`s
These were not doing anything.
{ inherit (f) x; }
is equivalent to
{ x = f.x; }
but
{ inherit (f); }
is equivalent to
{}
since there is no attribute name given.
* Refactor `srcFilters` to use `any`
That's more lighweight and easier to extend.
* Use `installShellFiles` for `buildManPages`
* Produce a more stable version
* Move package `radicle` next to the others
It's nothing special!
* Refactor Flake
More code reuse, and a little more direct, basically.
* Set environment variable on derivation
* Compress even more by removing special crate
* Use `with pkgs` before lists of packages
* Use system-independent `lib`
* Clean up
* Update `flake.lock`
* Enable checks `audit`, `deny`
A `.keep` file in the `objects/pack` directory acts as a lock to ensure that
operations like `git gc` do not clean up `.idx` files while a fetch is being
performed. It is safe to remove the `.keep` file once a fetch has completed and
a reference points to a given object in the packfile.
To ensure this happens in `radicle-fetch`, a newtype `Keepfile` is introduced.
It uses special `Drop` semantics by attempting to remove the file when the
object is dropped -- logging a warning if it fails to do so.
`FetchState` is augmented to keep a set of the `Keepfile`s. This ensures that
when the fetch completes, either successfully or with a failure, when
`FetchState` is dropped then so are the `Keepfile`s.
We ensure that the expected behaviour of removing the `.keep`files occurs by
adding a check in `e2e::test_replication` to check Alice's repository's packs.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
* Remove `rad sync --force` flag
* Remove `rad clone --private` flag
These flags are no longer necessary. Simplify specifying which seed
to fetch from with `--seed` is enough.
cloudhead
Yorgos Saslis
Sebastian Martinez
Alexis Sellier
Lars Wirzenius
Lorenz Leutgeb
Fintan Halpenny