Commit Graph

1947 Commits

Author SHA1 Message Date
Lorenz Leutgeb 660236225e
signals: Add license to calm `cargo deny` 2024-05-16 12:05:22 +02:00
cloudhead 6dd29d4b50
build: Move `ulimit` flag to correct location 2024-05-15 14:47:57 +02:00
Sebastian Martinez 5727359319
httpd: Validate blob sizes 2024-05-15 13:03:28 +02:00
Rūdolfs Ošiņš 3b342ab385
httpd: Set cache header for immutable blob endpoint
It is safe to do so because it is content-addressed.
2024-05-15 12:28:30 +02:00
cloudhead b195a3eb35
build: Set ulimit on container
This was necessary on some machines.
2024-05-15 12:24:44 +02:00
cloudhead a6cb4d0f48
node: Implement IP-based banning
The previous banning implementation didn't work, since it included port
numbers in the address value.

Here, we keep a separate table with (node, ip) pairs and update this
table when a node connects or is banned.
2024-05-15 12:10:23 +02:00
Fintan Halpenny 72c715016f
fetch: add trace logging to fetch protocol
Log the following at the trace level:
- Receiving the pack along with the amount of time it took
- The closing of the connection

Also organising the imports, while in the neighbourhood.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-15 10:07:56 +01:00
Fintan Halpenny f038b7ee5c
cli: watch for upload-pack events in `rad init`
To improve the user experience of `rad init`, the events listener waits for
upload-pack messages indefinitely. The spinner reports back the upload progress
over time.

This process can be cancelled via Ctrl+C without affecting the announcement,
since that is happening in the background.

To also improve the experience, the remote NIDs that have successfully synced
are reported to the terminal.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-15 10:07:56 +01:00
Fintan Halpenny 344fe3a901
node: increase `FETCH_TIMEOUT`
After empirically testing HardenedBSD[[0]], which has a total blob size of
42GiB, and its largest checkout is 1.26GiB (using `git-sizer`[[1]]), it was
found that letting the background process fetch the repository with a timeout of
1 hour is enough (a very generous estimate).

This deemed sufficient, since cases smaller than this will report back that they
are done before this time. For cases larger than this, more research will be
needed.

[0]: https://git.hardenedbsd.org/hardenedbsd/HardenedBSD
[1]: https://github.com/github/git-sizer

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-15 10:07:56 +01:00
Fintan Halpenny 027bfbef67
node: stream timeout semantics for `Node::call`
The timeout for a `UnixStream` will interrupt a `read` even when more data is
available. The preference would be that the timeout applies to each read, where
the timeout is reset again once a read is successful. This better supports
streaming semantics expected for the `Handle::subscribe` method.

To achieve this, the `LineIter` type holds the `UnixStream` and timeout
`Duration`. The `Iterator` implementation can then use `read_line` to progress
the stream one line at a time, and the timeout only applies to each read.

Co-authored-by: Alexis Sellier <alexis@radicle.xyz>
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-15 10:07:56 +01:00
Fintan Halpenny c8a24d5584
node: report upload-pack progress
Emit the `UploadPack` events from the `upload_pack` method.

A wrapper type, `Reporter`, is introduced for implementing a `Write` instance
that will parse the given bytes as `gix_protocol::RemoteProgress`, if possible,
and emit it via the `Emitter`, and then write the bytes to the underlying
writer. This allows any subscribers to see the progress of an upload-pack, i.e.
`Compressing objects`, `Counting object`, `Enumerating objects`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-15 10:07:56 +01:00
Fintan Halpenny 89627a8b6a
radicle: add upload-pack events
Add a variant for upload-pack events to allow reporting from the upload-pack
process.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-15 10:07:56 +01:00
Fintan Halpenny f0648c2a18
radicle: move Emitter to events module
Move the events Emitter to the `radicle::node::events` module to allow it to be
used outside of `radicle-node`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-15 10:07:56 +01:00
Fintan Halpenny fd4c4cd0af
node: dynamic fetch timeout via channels
Instead of using `DEFAULT_CHANNEL_TIMEOUT`, the timeout is dynamically set
per-fetch by the caller.

This passed via the `Io::Fetch` variant -- when initiating a fetch, and the
`Control::Open` variant when responding to a fetch.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-15 10:07:55 +01:00
cloudhead cadd996a76
Move prepared statements out of transaction
There's no need to create these on each loop iteration.
2024-05-14 16:36:04 +02:00
cloudhead 21cced05c1
node: Allow for "WAL" mode in sqlite
This solves concurrency issues between the node and http daemon.
2024-05-14 16:34:30 +02:00
cloudhead 25c6660a59
node: Use inventory cache for checking missing
Use the cache when checking whether we're missing an inventory.
This is a lot faster.
2024-05-13 16:37:02 +02:00
cloudhead cce46a3d8b
Try to improve performance of repository open
By using `open_ext` with `BARE`, we defer loading of the Git config,
which has been shown to be expensive.
2024-05-13 16:37:02 +02:00
cloudhead 8e541dcf58
node: Set TCP NO_DELAY for node connections
Just experimenting. I think this will be a better default for the type
of data exchanged between nodes.
2024-05-13 16:12:41 +02:00
cloudhead 2ca1e74707
cli: Have way to delete payload field in `rad id`
Allow for deleting a field by setting it to `null`.
2024-05-13 16:12:41 +02:00
Alexis Sellier 930f9b0a7e
node: Don't always connect to Tor addresses
Only connect if Tor is configured.
2024-05-10 13:38:02 +02:00
Alexis Sellier c5395f0674
cli: Improve `rad stats` output
Use daily/weekly figures.
2024-05-10 13:13:03 +02:00
Alexis Sellier 9766c67671
node: Fix tor proxy issue for regular addresses
This should allow using a Tor proxy for regular connections.
2024-05-09 21:45:48 +02:00
cloudhead 6ab3bfcba0
node: Use banning for repeated offenders
When the penalty score for a node reaches a certain threshold, we ban
the node.

Banned nodes are not allowed to connect to us.
2024-05-07 14:49:02 +02:00
cloudhead 9a82aae993
node: Reset connection attempts when stable
Instead of resetting the number of attempts on connection, reset it when
the connection is stable, ie. after some time has passed.

This prevents connection retries from happening in succession when a
peer is disconnected during the initial handshake, as is the case with
invalid subscribe timestamps.
2024-05-07 14:49:02 +02:00
cloudhead ef4efe3200
node: Improve logging for storage check 2024-05-07 14:49:02 +02:00
cloudhead 7484d737d5
node: Re-use inventory message timestamps
If our inventory hasn't updated, don't issue a new inventory
announcement, simply use the previous message.

This avoids identical inventories propagating over the network and using
up resources.
2024-05-07 14:49:02 +02:00
cloudhead 88a4029789
node: Use `BoundedVec` for inbox 2024-05-07 14:49:02 +02:00
cloudhead f212dbb2ee
node: Make sure we bound peer message inboxes 2024-05-07 14:49:02 +02:00
Fintan Halpenny e8f2f88bf6
cli: Add test for unknown field in identity doc
Add a test to confirm that the identity can still be deserialized when an
unknown field is added.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-07 10:21:07 +02:00
Fintan Halpenny 9b719d0cca
radicle: allow for unknown fields in Project deserialization
Allow unknown fields to pass through when deserializing the `Project` payload.
This allows the deserialization to be more resilient when unknown fields are
added, e.g. from the CLI, and improves backwards-compatability when new fields
may be added in the future.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-05-07 10:19:59 +02:00
cloudhead 43b0b2b426
node: Always go through proxy with Tor
Force the proxy for regular connections, when Tor is configured.
2024-05-06 22:27:46 +02:00
cloudhead 153be9ede4
node: Improve session type definition 2024-05-06 22:22:39 +02:00
Dr Maxim Orlovsky a3cce22f7a
node: Patch dependencies for Tor/socks5 support 2024-05-06 22:19:36 +02:00
cloudhead 72db6dd7dd
docs: Fix example in `rad-id.1` 2024-05-06 13:40:43 +02:00
cloudhead fddd0680fa
cli: Make ruby syntax highlighting non-optional
The current verison of the crate no longer requires C++ and builds fine.
2024-05-05 00:50:54 +02:00
cloudhead 2f3d1284ca
docs: Fix download link in README.md 2024-05-04 22:58:39 +02:00
cloudhead 6dd52c94fb
cli: Use new pager for `rad diff` 2024-05-03 14:31:08 +02:00
cloudhead 40f4383bbc
term: Implement content-aware pager
This is a pager (like `less`) that takes an `Element`, and dynamically
re-renders the element when the terminal size changes. This ensures that
the content is always properly wrapped and rendered.
2024-05-03 14:31:05 +02:00
cloudhead 2801403182
signals: Move signal handling to its own crate
We're going to share this code with `radicle-term`.
2024-05-03 14:30:51 +02:00
cloudhead 425e7b5092
docs: Fix docstring variable 2024-05-03 13:26:33 +02:00
cloudhead 104a1b96d0
docs: Specify `--root` for source installation
This places the binaries in the common location for Radicle.
2024-05-03 12:07:45 +02:00
cloudhead 227e13d5b1
build: Push tags as part of upload 2024-05-02 21:37:15 +02:00
cloudhead 2c6fa989cb
scripts: Script for outputting markdown changelog 2024-05-02 17:00:14 +02:00
cloudhead 0d880e12e1
node: Respect relay config on subscribe
Don't relay when a subscribe message is received, unless our relay
configuration allows us.
2024-05-02 13:48:47 +02:00
cloudhead a54180199e
node: Further increase the rate-limit defaults
The rate-limitter gets hit on connect, due to historical gossip messages
being exchanged. This results in a high initial burst that needs to be
accounted for.
2024-05-02 13:43:09 +02:00
Sebastian Martinez 5ffbf7545f
httpd: Add apiVersion to root handler and node information
This will help consumers of the httpd to check if they are on the right
version of the API
2024-05-02 13:09:00 +02:00
Sebastian Martinez d04d9d7d1f
httpd: Add threshold to project info 2024-05-02 14:43:12 +02:00
Sebastian Martinez c4d4ad7008
httpd: Add alias to project delegates 2024-05-02 14:43:12 +02:00
Sebastian Martinez 854fc9af8e
httpd: Decouple repo stats from tree endpoint 2024-05-02 14:41:27 +02:00