Commit Graph

890 Commits

Author SHA1 Message Date
Alexis Sellier 636d9e55ee
ci: Remove `ci.yaml`
We are no longer relying on GitHub CI.
2023-04-17 15:58:55 +02:00
Alexis Sellier 51b372ecad
node: Share channel type between modules
Shares the `Channels` type between the `wire::protocol` and
`worker` modules.
2023-04-17 15:56:24 +02:00
Alexis Sellier d0e24bd0ce
node: Simplify worker data piping 2023-04-17 15:55:31 +02:00
Alexis Sellier c99497eb21
Update `netservices` to 0.2.2
This includes a small change in how the EOF is sent as well.
2023-04-17 15:53:31 +02:00
Alexis Sellier cb79ebef76
Fix storage documentation
The link was invalid.
2023-04-17 12:08:47 +02:00
Alexis Sellier a8f0329391
node: Downgrade some logs to `trace` level
These logs were a bit noisy and don't help much.
2023-04-17 11:21:22 +02:00
Alexis Sellier 0e7ffcc916
node: Properly validate signed refs on fetch
With this change, we not only verify the signed refs before
the repository "transfer", but also validate that all refs in the
repo are signed and there is no discrepancy with the signed refs
file.
2023-04-15 00:17:17 +02:00
Alexis Sellier d9dd28b347
scripts: Add `delete-remote-refs.sh`
Useful for deleting refs of a corrupted remote.
2023-04-15 00:15:02 +02:00
Alexis Sellier 56e41fadd9
node: Update `netservices` and `io-reactor`
This fixes hanging issues we were experiencing.
2023-04-14 23:30:18 +02:00
Alexis Sellier 9729a23559
node: Use `StagedRepository` to know what to do
To know whether or not we're cloning, use `StagedRepository` instead
of checking manually.
2023-04-14 13:56:52 +02:00
Alexis Sellier 3e05939d14
node: Don't fail fetch for existing repositories
If `git-fetch` returns a non-zero code on exit, we keep going in the
case of an existing repository.

The reason is that `git-fetch` will return an error code if any ref
is rejected during the fetch. But this can happen if someone
pushes conflicting `rad/sigrefs` branches for example, and the node
already has a copy of that branch.
2023-04-14 13:56:16 +02:00
Alexis Sellier 18616a16bd
node: Match error by value
There wasn't a good reason to use a reference. This also allows us
to explicitly drop the session.
2023-04-14 13:55:47 +02:00
Alexis Sellier 17651e259d
node: Don't panic if peer disconnected 2023-04-14 13:55:32 +02:00
Alexis Sellier 8c92282578
node: Fix some error messages
A "block" policy doesn't necessarily mean an active block.
2023-04-14 13:55:22 +02:00
Alexis Sellier 992878fab4
cli: Update `rad untrack` command
This command was using very old code that needed updating.
2023-04-14 13:54:33 +02:00
Alexis Sellier eecea67ffd
cob: Deduplicate change parents 2023-04-14 13:54:27 +02:00
Alexis Sellier 97f076e3cf
Include dependencies for certain COB actions
We ensure that for example, the given commits are fetched before the
revision or merge actions in a patch.
2023-04-14 13:22:10 +02:00
Alexis Sellier 489370dcbd
cob: Add ability to add more parents to change
This allows change commits to have more than one parent that isn't
part of the DAG.

We can use this to create dependencies towards git objects such as
commits, in the case of patch actions.
2023-04-14 13:22:10 +02:00
Alexis Sellier bda5d6f080
node: Tunnel git data without parsing packet-lines
Instead of parsing the worker requests and responses, which was causing issues
around packfile transmission, we use threads and pass the bytes
transparently.
2023-04-14 13:16:32 +02:00
Alexis Sellier a05de1fbf3
node: Test fetch with large repository
This is for stress-testing the replication.
2023-04-14 13:15:52 +02:00
Alexis Sellier ac3a03fea6
node: Set git-daemon keep-alive
Send a keep-alive packet every 3 seconds to make sure the client doesn't
timeout during pack building.
2023-04-14 13:15:52 +02:00
Alexis Sellier 273ce2744a
node: Add more information to worker result log 2023-04-14 13:15:52 +02:00
Alexis Sellier f4257eb953
node: Add setting for limiting fetch concurrency
Can be passed via `--limit-fetch-concurrency <num>`.
2023-04-14 13:15:52 +02:00
Alexis Sellier ea4294c79f
node: More realistic concurrent fetch test
This implements a test which can optionally use a lot more data by
setting `RAD_TEST_SCALE` to a higher value.
2023-04-14 13:15:52 +02:00
Alexis Sellier 8205f8ad4e
Update `fastrand` dependency to 1.9.0 2023-04-14 13:15:52 +02:00
Alexis Sellier 31647521fa
node: Properly close worker channels
We need to send a 'close' event to the worker channel for it to exit
without an error.

Also updated and improved a bunch of the logs.
2023-04-14 13:15:52 +02:00
Alexis Sellier 267aba5901
node: Announce inventory only if it changed
Previously we were announcing inventories all the time, due to
relying on the timestamp changing on insertion.

We change this to only announce our inventory when it actually changes.
2023-04-14 13:15:52 +02:00
Alexis Sellier 7cbfde1e9b
node: Don't needlessly fetch or announce
* We shouldn't announce if no refs were updated.
* We shouldn't blindly fetch if we don't know if we have an item in
  storage.
2023-04-14 13:15:52 +02:00
Alexis Sellier f3d513783c
node: Don't fetch owned refs unless cloning
We shouldn't be fetching our own refs unless we are creating a clone.
2023-04-12 16:40:53 +02:00
Alexis Sellier 9db63a8801
cli: Cleanup patch, checkout and clone output
Remove extra whitespace, start moving away from sprout.
2023-04-12 12:02:38 +02:00
Alexis Sellier c8e548fb1d
node: Make sure we have all refs in staging copy
Previous to this change, we just created a clone, which was missing
all the refs.
2023-04-12 11:43:43 +02:00
Alexis Sellier 097b5b4ca4
node: Improve worker code logging and errors 2023-04-12 11:28:12 +02:00
Alexis Sellier eeec2008e8
node: Allow most refs to be force-updated
Previously, during the "transfer" stage of a fetch, we used `force:
false`, which didn't allow refs to be force-updated.

This change makes it possible to fetch force-updated refs, but ensures
that the sigrefs branch is never force-updated.
2023-04-12 11:09:05 +02:00
Alexis Sellier 89b9eb53b7
node: Implement protocol multiplexing
To improve the reliability and flexibility of the protocol, we introduce
multiplexing over peer connections. This involves a new `Frame` type
that carries a stream-id and payload.

Three stream types are made available:

1. Control
2. Gossip
3. Git

This change brings the following improvements:

* Removed need to queue fetch requests
* Removed need to queue gossip messages
* Removed need for `Fetch` and `FetchOk` messages
* Service doesn't need to know about inbound fetches
* Removed one round-trip for fetch negotiation
* Removed special `done` git packet
* Removed session logic and state around Git/Fetch protocols
* Removed code around upgrading/downgrading transport
* Worker in responder mode is able to process any number of fetches
* Connections support any number of concurrent fetches

We had to introduce a few extra things however to make it all work:

* A `VarInt` type for variable-length integers, since we want the frames
  to be lightweight.
* A custom "tunnel" implementation, since we couldn't use the existing
  one anymore.

Overall the change removes more complexity than it adds, while improving
the protocol along the way.
2023-04-10 14:41:55 +02:00
Adam Szkoda 0f47dc9057
cli: Make `rad rm` work without a fork 2023-04-10 10:42:22 +02:00
Alexis Sellier 3d40a606f7
cli: Fix `rad track` help output
Even though we accept `did:key` as a substitute for NIDs, we don't
generally accept DIDs in this command.
2023-04-10 10:28:46 +02:00
Alexis Sellier c1face7cd8
cli: Fix and simplify `rad review`
* Properly remove review help message from review
* Remove `--no-confirm` and confirmation step

The CLI should be mostly non-interactive for day-to-day commands,
keeping the interactivity for the TUI.
2023-04-10 10:27:52 +02:00
Alexis Sellier f999a62da2
cli: Show patch status in color in `patch show` 2023-04-10 10:17:38 +02:00
Alexis Sellier 7d1e3d489b
cli: Add `patch archive` sub-command 2023-04-10 10:17:37 +02:00
xphoniex 414477a316
cli: Unify format of help messages
Signed-off-by: xphoniex <dj.2dixx@gmail.com>
2023-04-10 10:06:45 +02:00
Adam Szkoda 992e94ebe4
ci: Improve directory structure 2023-04-07 11:37:17 +02:00
Adam Szkoda f1821e0b99
ci: Update releases volume path 2023-04-07 11:37:15 +02:00
Alexis Sellier 505dd9ea96
Add `install` script
Downloads binaries and installs them, based on your operating system.

Can be used in a one liner, for example:

    sh <(curl https://radicle.xyz/install)
2023-04-06 11:29:42 +02:00
Adam Szkoda b33893741f
ci: Migrate away from Google Cloud Storage 2023-04-06 11:28:46 +02:00
Alexis Sellier 9db91e976e
Add `.gitsigners` file with core team 2023-04-05 11:46:57 +02:00
Alexis Sellier 33461e4be1
Set patch status to `merged` automatically
When a merge is applied to a patch, we check whether the threshold
of delegates has been reached, at which point we set the patch status
to `merged`.

This requires threading the identity document oid through the "apply"
process.
2023-04-05 10:32:45 +02:00
Alexis Sellier 6b44955cd9
cli: Show timeline in `patch show` 2023-04-04 21:19:50 +02:00
Alexis Sellier ed4c266a51
node: Don't prune owned refs
Owned refs were being pruned on fetch.
2023-04-03 22:09:41 +02:00
Alexis Sellier 9e50d9053a
cli: Additional fixes in the merge process 2023-04-03 16:06:24 +02:00
Alexis Sellier b6fa4a4b2a
cli: Some improvements to `rad merge` 2023-04-03 16:06:24 +02:00