Implement caching for the `Issue` and `Patch` COB. This is achieved by
having the `Transaction::initial` and `Transaction::commit` methods
take a cache which can update the newly created/updated object. The
`Store::remove` method also takes the cache for removing the object
from the cache, as well as the repository.
This meant that the `*Mut` types for the respective COBs are required
to carry a cache alongside the repository store. Identities will not
be cached, and so this is always set to `NoCache` -- which performs no
caching, and always succeeds.
To perform cache reading for issues and patches, the `cache::Issues`
and `cache::Patches` traits are introduced. They capture the minimal
amount of methods that are needed for the `rad issue` and `rad patch`
CLI commands.
The implementor for each of these traits is there respective `Cache`
types, which combines their backing repository store and the SQLite
database. They both provide convenience methods:
- `create` (and `draft` for patches)
- `get_mut`
- `remove`
which can be used instead of their repository store counterparts.
All uses of the repository stores (where needed) are replaced with
`Cache` types.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
1. Simplify latency calculation to be more predictable, otherwise,
setting a latency is not super useful.
2. Turn a hard error into a log. It's not worth fixing it now though, as
it isn't a problem for the tests.
This is going to be useful for testing message amplification in the
network protocol.
We also reset the state of a few things everytime the simulator is run.
Shows you updates on local repositories.
Included here is a basic implementation of a notification system. A new
database is created and updated after every fetch, with refs that have
changed.
The `rad inbox` command can then list items in the database and delete
them.
Most of these changes should be self-explanatory.
* Atomic fetch was no longer used
* The default policy when initializing should be 'all', as it is with
`rad init`.
Dequeued fetches were always fetching all refs, instead of fetching the
specific refs that were initially requested. We fix that by storing the
refs to fetch in the fetch queue.
When using `network: "test"`, we shouldn't default to any public seed
node. This change makes sure that the "preferredSeeds" configuration var
is only given a value when `network` is `main`.
The previous change to the wire protocol introduced a bug: outbound
peers that failed to connect fully would not notify the service on
disconnect. This change fixes it.
* Make sure we cleanup pending peer states when disconnecting before
handshake.
* Make sure we don't panic if a peer disconnects before being fully
connected.
Introduce pinned repositories to the `profile::Config`, i.e.
`config.json`.
This is used for pinning repositories in the httpd `/projects` handle.
To allow returning all repositories, an `all` query parameter is also
included.
This is non-backwards compatible change, since we introduce the `show`
query parameter which, if not supplied, will default to `pinned`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
When a DNS error occured, the `dial` call failed and we didn't get a
connection attempt, just a disconnect. We fix that by always calling
back into the service when we attempt a connection.
There was an issue with the old version due to the use of `RawFd` as
peer IDs, since they are not unique for the lifetime of a process.
With this change, we use the new `ResourceId` type, though these are not
immediately available, as they are generated when the resource is
registered.
Allow some margin of error when sending multiple inventory announcements
at a time. Previously, they might end up with the same timestamp, so
only the first announcement would be processed by nodes. Now, we ensure
that the timestamp is updated.
To be able to use log setup across the different components of
heartwood, expose a `logger` module from `radicle`. This also includes
a `test` logger.
These modules are guarded behind a `logger` feature flag, so that
other crates are not bringing any uneeded dependencies.
This allows logging to be setup in `radicle-cli-test` so that when a
test fails, the log output is present for debugging purposes.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
When a peer is performing a background and foreground fetch from two
different peers, the background fetch result was being used as the
result for the foreground fetch. This results in inconsistencies,
since the foreground peer has new updates from themselves.
The reason for this issue was the fetches that were queued were not
correctly being handled, and the other peer's results were being
returned instead.
This fixes the queueing and dequeueing by ensuring that the queued
fetch occurs, passing the correct result channel. This fetch will then
be subsequently dequeued correctly and the foreground fetch obtains
the correct result.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
This test is flakey and requires further investigation. Some extra
assertions are added to ensure that the expected behaviour is
happening upon fetching. It is unknown if the fetch queue is behaving
properly, and it's suspected that some existing fetch results might be
used for queued fetches.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Previously, we might ban a node if we're having trouble connecting to
it, but this could be due to internet connectivity issues.
We introduce a penalty score for nodes, that increases if there are
connection issues or the node misbehaves, and decreases on successful
connections.
When connected to new nodes, we sort them so that low-penalty nodes are
attempted first.
Some test flakes would occur due to the `converge` function looping
forever. It was noticed that in these cases the log output would look
like:
test: Node z6MkqTKLVnFdVF24PQiDoQffbTPT7U5igghS3XWuFJLfah1P has
{(Id(rad:z39WuDs9hrqmCr9shjoz6dn2Abqyz),
PublicKey(z6MkemMxfWhRQFKy4UPX7FsN8GjLAZ9T65rbmDsUn7fchioP)),
(Id(rad:z39WuDs9hrqmCr9shjoz6dn2Abqyz),
PublicKey(z6MkpdPJp7naAurUjCpwef1BDD5jfPhAvwfinaGd7pVh227j)),
(Id(rad:z39WuDs9hrqmCr9shjoz6dn2Abqyz),
PublicKey(z6MkqTKLVnFdVF24PQiDoQffbTPT7U5igghS3XWuFJLfah1P))}
with further investigation, the variable `all_routes` was found to be
smaller than the calculated routes.
Instead of checking `==`, instead check that
`routes.is_superset(&all_routes)`, which means that `routes` must be
equal to `all_routes`, but allows it have extra entries.
Signed-off-by: Fintan Halpenny <Fintan Halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Since node announcements are stored and retrieved from disk (due to the
PoW), the timestamp on it is never updated. This means it can be too old
to cache and not sent/received by nodes. Fix this by having a max age.
Instead of creating the DB policies handle when needed, construct the
handle when the `Worker` pool is being constructed.
This replaces the usage of the DB handle with the `Worker`'s field
instead.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
If a node had removed a seeding policy and another node fetched from
it, then it would perform the fetch.
Instead, check the policy during the authorization step and prevent
the fetch is the resulting policy is `Block`.
Note that the policy will be `Block` iff the `Block` policy was
specifically set or the `Block` is the default policy for the node.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Nix Flakes[[0]] are a more modular way of expressing nix derivations.
The `flake.nix` file describes:
- How to build the project, or components of a projet, via `nix build`
- What "checks" can run for a project, which can be executed via
`nix flake check`
- What binaries can be run, e.g. `nix run .#rad` to execute the CLI
This approach is preferable since a lot of the Nix ecosystem is using
Flakes -- despite it still being marked as experimental.
One interesting benefit, that needs confirming, is that a check can be
run from a remote Git source. So if someone creates a patch on
Radicle, someone else with the `nix` binary can execute:
nix flake check git+https://app.radicle.xyz/nodes/seed.radicle.xyz/rad:z3gqcJUoA1n9HaHKufZs5FCSGazv5?refs=patches/<patch id>
This means that anyone with `nix` can check patches without having to
fetch the patch itself.
Another obvious benefit will be that it allows the Radicle binaries to
enter the Nix ecosystem.
The big caveat to both of the above is that, currently, the tests are
failing while running `nix flake check`. They are currently disabled
with `doCheck = false`. They are potentially failing due to some
strange sandboxing issues which are being looked into.
[0]: https://nixos.wiki/wiki/Flakes
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Since forking creates your `rad/sigrefs`, which then needs to be backed
up forever, we avoid forking on `rad clone`. This should make little or
no difference for users, as the "fork" will be created on `git push`.
Alexis Sellier
cloudhead
Fintan Halpenny
Fintan Halpenny