On FreeBSD systems, the file limits are of type `i64` in comparison to
`u64` on Linux systems.
Also fix `rad web` command name on Windows and FreeBSD.
Co-authored-by: cloudhead <cloudhead@radicle.xyz>
Co-authored-by: Shawn Webb <shawn.webb@hardenedbsd.org>
In the case of an out-of-sync local node, we were showing the wrong tip.
Additionally, mark your tip as unannounced if it isn't announced. This
isn't fool-proof, but can be useful to debug certain issues.
From empirical testing, the `FETCH_TIMEOUT` can be set to a lower value of 3s.
For a repository that is about 300MiB, the process will successfully send all
packfile bytes and the references will be written.
The `--timeout` option is also used in the `git-upload-pack` process for good
measure.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `join` method is equivalent to the custom loop that was written for joining
the `reader` thread in upload-pack.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Introduce an `UploadPack` type for keeping track of relevant data from
upload-pack events and displaying progress to the terminal.
It keeps track of the number of remotes that are being uploaded to as well as
the throughput of transmitted data.
The `Progress` events are logged rather than being displayed since they don't
provide any useful details to the user.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
A `.keep` file in the `objects/pack` directory acts as a lock to ensure that
operations like `git gc` do not clean up `.idx` files while a fetch is being
performed. It is safe to remove the `.keep` file once a fetch has completed and
a reference points to a given object in the packfile.
To ensure this happens in `radicle-fetch`, a newtype `Keepfile` is introduced.
It uses special `Drop` semantics by attempting to remove the file when the
object is dropped -- logging a warning if it fails to do so.
`FetchState` is augmented to keep a set of the `Keepfile`s. This ensures that
when the fetch completes, either successfully or with a failure, when
`FetchState` is dropped then so are the `Keepfile`s.
We ensure that the expected behaviour of removing the `.keep`files occurs by
adding a check in `e2e::test_replication` to check Alice's repository's packs.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
* Remove `rad sync --force` flag
* Remove `rad clone --private` flag
These flags are no longer necessary. Simplify specifying which seed
to fetch from with `--seed` is enough.
*Breaking change* The `tor` configuration field is no longer
supported. Users must remove that field from their config.
Allows for a mixed mode, where regular addresses bypass the Tor proxy,
while onion addresses go through it.
When `onion.mode` is set to "forward", Tor connections are fowarded to
the global proxy, and if it isn't set, to the OS's DNS resolution.
For mixed mode, a global proxy simply isn't set, so that IP/DNS is not
proxied.
The previous banning implementation didn't work, since it included port
numbers in the address value.
Here, we keep a separate table with (node, ip) pairs and update this
table when a node connects or is banned.
After empirically testing HardenedBSD[[0]], which has a total blob size of
42GiB, and its largest checkout is 1.26GiB (using `git-sizer`[[1]]), it was
found that letting the background process fetch the repository with a timeout of
1 hour is enough (a very generous estimate).
This deemed sufficient, since cases smaller than this will report back that they
are done before this time. For cases larger than this, more research will be
needed.
[0]: https://git.hardenedbsd.org/hardenedbsd/HardenedBSD
[1]: https://github.com/github/git-sizer
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The timeout for a `UnixStream` will interrupt a `read` even when more data is
available. The preference would be that the timeout applies to each read, where
the timeout is reset again once a read is successful. This better supports
streaming semantics expected for the `Handle::subscribe` method.
To achieve this, the `LineIter` type holds the `UnixStream` and timeout
`Duration`. The `Iterator` implementation can then use `read_line` to progress
the stream one line at a time, and the timeout only applies to each read.
Co-authored-by: Alexis Sellier <alexis@radicle.xyz>
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Emit the `UploadPack` events from the `upload_pack` method.
A wrapper type, `Reporter`, is introduced for implementing a `Write` instance
that will parse the given bytes as `gix_protocol::RemoteProgress`, if possible,
and emit it via the `Emitter`, and then write the bytes to the underlying
writer. This allows any subscribers to see the progress of an upload-pack, i.e.
`Compressing objects`, `Counting object`, `Enumerating objects`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Move the events Emitter to the `radicle::node::events` module to allow it to be
used outside of `radicle-node`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Instead of using `DEFAULT_CHANNEL_TIMEOUT`, the timeout is dynamically set
per-fetch by the caller.
This passed via the `Io::Fetch` variant -- when initiating a fetch, and the
`Control::Open` variant when responding to a fetch.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Instead of resetting the number of attempts on connection, reset it when
the connection is stable, ie. after some time has passed.
This prevents connection retries from happening in succession when a
peer is disconnected during the initial handshake, as is the case with
invalid subscribe timestamps.
If our inventory hasn't updated, don't issue a new inventory
announcement, simply use the previous message.
This avoids identical inventories propagating over the network and using
up resources.
Relay now has three values: "always", "never" and "auto".
The first two operate like the previous `true`/`false`. The
new (default) value turns relaying on for nodes with public (external)
addresses and off for nodes without. This should reduce redundant traffic on
the network, that was causing the rate limiters to trigger.
Many (most?) system clocks are not monotonic, and even though we're
getting a `LocalTime`, which *is* monotonic when using `now()`, it is
constructed from a `SystemTime` which isn't.
* Try to use constants instead of strings
* Move crypto seed override out of radicle-crypto
* Use GIT_COMMITTER_DATE for commit overrides
* Use RAD_LOCAL_TIME as general time override
* Allow variables to be used in release mode
There are rare cases where ref announcements may be sent in close
succession, and the current clock time is re-used. This will cause the
second (newer) announcement to be ignored by peers.
We add a `timestamp` method that checks if the timestamp is re-used, and
increments it if so, ensuring that we always use a fresh one.
cloudhead
Fintan Halpenny
Alexis Sellier
Dr Maxim Orlovsky