When deciding whether or not to fetch a repo based on a refs
announcement, we need to know whether our signed refs are stale or not,
compared to the ones in the announcement. This can be quite expensive as
it requires multiple reads from Git's ODB, which often results in
packfile loading.
To remedy this, we introduce a refs cache in our node database that is
consulted when an announcement is received.
We also remove the logic that generates `info` messages, since they are
rarely needed and introduced more complexity.
Additionally, we revert commit 3ad2b4431f,
which introduced logic to minimze wants and haves with additional logic,
since it potentially also increases the load on the ODB.
This change limits the amount of `wants` and `haves` data that is sent
to the serving side of a fetch.
When a `RefsAt` announcement is sent, the fetching peer can calculate
the difference between the `rad/sigrefs` they're aware of -- if it
exists -- and the newly advertised `rad/sigrefs`. This means they can
efficiently ask for the `wants` and `haves` of references that have
changed -- saving some data sent over to the serving side.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The protocol requires that the fetcher signals to the remote that it
is done with the fetching process, so that the remote can exit out of
its upload-pack process.
The first change is that the `done` signal can happen before the
validation, since nothing else needs to be fetched.
The second change is that the fetcher does not care if the remote
responds to this signal since it has the data required for validation
and applying the changes to its refdb. Thus, if an error is
encountered then it is logged, but the rest of the process proceeds as
normal.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
When performing a handshake, the error source is hidden in the error
variant and never shown.
To get insight as to what the error was, it is logged when it is
encounterd.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Log when different stages of the fetch have completed, while also
providing the elapsed time, in seconds.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The following scenario was noticed:
If a node has a `rad/id` that is behind the one it is fetching from,
where a delegate was added, then verification will fail since it will
not fetch the new delegate -- if the scope is set to `Followed`.
Irregardless if the `rad/id` is ahead or behind, the reference should
be fetched to use as the delegates anchor since it will tell the
fetching side which delegates in can fetch from the remote. If any of
those delegates are missing then the remote is indeed lying.
This is safe since the `refs/rad/id` reference is only updated by
computing the COB rather than setting the OID, so it will not update
through the fetch protocol itself.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Ref announcements contain the namespace (NID) and the SHA of the new
`rad/sigrefs`. This information can be used by the fetch protocol to
fetch only those `Oid`s for those namespaces.
Add a new stage `SigrefsAnnouncement` that captures this behaviour by
only asking for the advertised `Oid`s and updating the corresponding
`rad/sigrefs` for each namespace. Note that this is a special case of
the `SpecialRefs` stage and so the protocol chooses between these two
by checking if there was announcement passed through. The following
stage, `DataRefs`, stays the same.
Note that this stage can only be activated via a `pull`, and `clone`
stays the same.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `radicle-fetch` crate introduces a method for fetching from a
server using Git protocol V2, which provides the implementation
control over performing a sequence of fetches and validation over
domain data.
The protocol is built on top of the suite of Gitoxide crates[[0]]. It
builds wrappers on top of Gitoxide's refdb, odb, and transport layers
-- as well as taking the ls-refs and fetch code straight from the code
base to simplify for the needs of this crate.
[0]: https://github.com/Byron/gitoxide
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
cloudhead
Fintan Halpenny