The extension of the test shows that Node B's fetch cannot be dequeued because
the RID is in the active fetch set.
It can then be dequeued once Node A has been fetched from.
- Clear active[rid] only when the result's node matches the node that
started the fetch; mismatched results now report NotFound
- Prevent a late completion from a disconnected peer clearing a newer
fetch for the same repo started by a different node
- Add test covering the stale-from mismatch path
The configuration of the fetch process was only achieved at the setup
of the of the node.
This limits the ability to pass configuration options to the fetch
worker while the node service is running.
This change introduces `radicle_protocol::fetcher::state::FetchConfig`
to allow configuration options to be kept track of. This includes the
previous `timeout` option, but now also includes the
`radicle_fetch::Config`.
The service now produces the `FetchConfig` and passes it through to
the fetch worker.
The construction of the debug object is unwieldy, and error prone
(for example, renamed struct members have to be manually renamed
in the serialization code, see "refs" vs. "refsAt").
Use derived serializers where possible to make this easier to maintain.
Using empty `Vec`s to mean "all refs" can be confusing. Avoid the chance
of confusion by introduction of a new type that clearly describes which
refs should be fetched.
It was found that if a fetch was started for a specific `Vec<RefsAt>`
and a following general, i.e. empty, `Vec<RefsAt>` was queued, the
subscriber for the general one would be removed when the specific
fetch completed.
This behaviour is undesirable, since the general fetch could
potentially fetch more data.
To prevent this from happening, the `Vec<RefsAt>` is added to the
`FetchKey`, so that `subscribers` are matched with the set of
references they want to hear results for.
A test was added for the above scenario.
Wire up the new `FetcherService`. This reduces the `Service` fetch
methods down to:
- `fetch`
- `fetched`
- `dequeue_fetches`
- `fetch_refs_at`
This simplifies the code by off-loading the logic to the `fetcher`
family of types, rather than handling the intricacies in the `Service`
itself.
The `Service` now just performs the necessary I/O based on the
returned events from the `fetcher`.
This also removes the fetching state from the node sessions. This
follows the single responsibility, and sessions now only care about
their connectivity.
Breaking changes:
- The `Connected` state of a peer no longer contains fetching
information, which was being returned as part of the JSON payload
result.
- The `rad debug` information for ongoing fetches contained the number
listeners awaiting for results, this was removed.
The `FetcherService` wraps the `FetcherState` alongside keeping track
of subscribers for a given fetch.
Typically, this is a channel for which the fetch result should be sent
to.
These subscribers are coalesced by maintaing a map from `(rid, node)`
pairs to the subscriber.
This continues the sans-IO approach where the type of subscribers are
not specified, but rather are kept as a type parameter – to be
specified by the caller.
This patch introduces the `FetcherState` which encapsulates the logic
and state for keeping track of fetches.
It uses a sans-IO approach, where the state of fetches transition
based on the current state and provided input.
Callers can then decide to perform I/O based on the returned event.
Daniel Norman
Fintan Halpenny
Josh Soref
Lorenz Leutgeb