Previously, we only kept tracked of unsynced nodes we were connected to.
Now we track all unsynced nodes, since announcements are relayed.
We also cap the replicas to exit earlier in case of very low replica
count.
Remove the borrow of `rid` to fix the clippy warning.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Instead of re-loading the full inventory on `rad init`, we simply add
the new repository to the in-memory cache.
We also keep track of whether a fetch was a full clone, to know when
to update our routing table.
For large seeds with many repos, it's expensive to traverse the
inventory everytime the node connects to a new peer, for example.
To improve the situation, we cache the inventory per `Storage` instance,
and add a `refresh` method to refresh the inventory.
This method is called when the node is told to synchronize its
inventory.
This change limits the amount of `wants` and `haves` data that is sent
to the serving side of a fetch.
When a `RefsAt` announcement is sent, the fetching peer can calculate
the difference between the `rad/sigrefs` they're aware of -- if it
exists -- and the newly advertised `rad/sigrefs`. This means they can
efficiently ask for the `wants` and `haves` of references that have
changed -- saving some data sent over to the serving side.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Announcement are logged at the debug level and are very frequent. This
causes a lot of noisy logging in a node.
Downgrade the log levels for these messages from debug to trace.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
For larger fetches, a timeout for the channels and socket control are
required. Bump the timeout from 9s to 30s.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The timeout was never being used, so it is being removed for the time
being.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Two concurrent fetches can occur from the same peer, while also being
for a different set of references. This was seen in the
`test_outdated_sigrefs` test. Alice would have an ongoing fetch from
Eve, and concurrently, Eve will create a new issue -- thus creating a
new `RefsAt` message -- and Alice would attempt to fetch this.
The previous logic would only check that Alice is fetching from Eve
and ignore the new updates -- meaning the test could fail. However, if
the `RefsAt` are checked then the other fetch can be recognised as
new, and queue it.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
If a peer deletes a COB, it does not mean that the COB should be
deleted. This was incorrectly assumed in the `radicle-node` fetch
code.
To fix this, when a `RefUpdate::Deleted` is seen after a fetch, the
cache is updated with the new state of the COB.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Implement caching for the `Issue` and `Patch` COB. This is achieved by
having the `Transaction::initial` and `Transaction::commit` methods
take a cache which can update the newly created/updated object. The
`Store::remove` method also takes the cache for removing the object
from the cache, as well as the repository.
This meant that the `*Mut` types for the respective COBs are required
to carry a cache alongside the repository store. Identities will not
be cached, and so this is always set to `NoCache` -- which performs no
caching, and always succeeds.
To perform cache reading for issues and patches, the `cache::Issues`
and `cache::Patches` traits are introduced. They capture the minimal
amount of methods that are needed for the `rad issue` and `rad patch`
CLI commands.
The implementor for each of these traits is there respective `Cache`
types, which combines their backing repository store and the SQLite
database. They both provide convenience methods:
- `create` (and `draft` for patches)
- `get_mut`
- `remove`
which can be used instead of their repository store counterparts.
All uses of the repository stores (where needed) are replaced with
`Cache` types.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
1. Simplify latency calculation to be more predictable, otherwise,
setting a latency is not super useful.
2. Turn a hard error into a log. It's not worth fixing it now though, as
it isn't a problem for the tests.
This is going to be useful for testing message amplification in the
network protocol.
We also reset the state of a few things everytime the simulator is run.
Shows you updates on local repositories.
Included here is a basic implementation of a notification system. A new
database is created and updated after every fetch, with refs that have
changed.
The `rad inbox` command can then list items in the database and delete
them.
Most of these changes should be self-explanatory.
* Atomic fetch was no longer used
* The default policy when initializing should be 'all', as it is with
`rad init`.
Dequeued fetches were always fetching all refs, instead of fetching the
specific refs that were initially requested. We fix that by storing the
refs to fetch in the fetch queue.
When using `network: "test"`, we shouldn't default to any public seed
node. This change makes sure that the "preferredSeeds" configuration var
is only given a value when `network` is `main`.
The previous change to the wire protocol introduced a bug: outbound
peers that failed to connect fully would not notify the service on
disconnect. This change fixes it.
* Make sure we cleanup pending peer states when disconnecting before
handshake.
* Make sure we don't panic if a peer disconnects before being fully
connected.
Introduce pinned repositories to the `profile::Config`, i.e.
`config.json`.
This is used for pinning repositories in the httpd `/projects` handle.
To allow returning all repositories, an `all` query parameter is also
included.
This is non-backwards compatible change, since we introduce the `show`
query parameter which, if not supplied, will default to `pinned`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
When a DNS error occured, the `dial` call failed and we didn't get a
connection attempt, just a disconnect. We fix that by always calling
back into the service when we attempt a connection.
There was an issue with the old version due to the use of `RawFd` as
peer IDs, since they are not unique for the lifetime of a process.
With this change, we use the new `ResourceId` type, though these are not
immediately available, as they are generated when the resource is
registered.
cloudhead
Fintan Halpenny
Alexis Sellier