Commit Graph

609 Commits

Author SHA1 Message Date
cloudhead dbf47fe4e9
Implement new versioning system
See `VERSIONING.md` for details.
2024-03-25 11:54:08 +01:00
cloudhead e6728db15a
cli: Unify all syncing code
We were using similar but slightly different syncing code in the
remote-helper, issues/patches and `rad sync`. This commit unifies them.
2024-03-22 16:35:06 +01:00
cloudhead 8f89f088c6
cli: Correctly honor sync timeout 2024-03-22 16:35:06 +01:00
cloudhead 38a76a42de
radicle: Don't update sync status redundantly
We were updating it even when the `head` was the same.
2024-03-22 16:35:06 +01:00
Sebastian Martinez 0581c017ae
cli: Add `rad patch archive --undo`
Allows archived patches to be reverted to open state.
2024-03-21 20:33:15 +01:00
cloudhead 6288dd0a24
radicle: Load inventory lazily
Instead of loading the inventory cache on storage open, load it
when the `inventory` function is called and the cache is `None`.
2024-03-21 18:03:18 +01:00
cloudhead 057af40ea3
radicle: Fix config loading with bad alias
Previously, a config would load fine with an invalid alias,
due to how serde is configured for the `Alias` type. This fixes it.
2024-03-21 14:44:04 +01:00
cloudhead d771ce8522
node: Make `Storage::refresh` private 2024-03-21 12:40:58 +01:00
cloudhead 2bcb03b021
node: Re-think inventory update code
Instead of re-loading the full inventory on `rad init`, we simply add
the new repository to the in-memory cache.

We also keep track of whether a fetch was a full clone, to know when
to update our routing table.
2024-03-21 12:40:58 +01:00
cloudhead 0e880e12e6
node: Cache node inventory
For large seeds with many repos, it's expensive to traverse the
inventory everytime the node connects to a new peer, for example.

To improve the situation, we cache the inventory per `Storage` instance,
and add a `refresh` method to refresh the inventory.

This method is called when the node is told to synchronize its
inventory.
2024-03-21 12:40:58 +01:00
Alexis Sellier f15afa84be
cli: Add timing information to `rad-sync` 2024-03-19 17:27:18 +01:00
Alexis Sellier ddd4bde302
node: Make worker count configurable 2024-03-19 16:34:15 +01:00
cloudhead d7c88f12ab
radicle: Fix some doc comment issues 2024-03-12 16:13:08 +01:00
Fintan Halpenny 3ad2b4431f
fetch: only send changed wants and haves
This change limits the amount of `wants` and `haves` data that is sent
to the serving side of a fetch.

When a `RefsAt` announcement is sent, the fetching peer can calculate
the difference between the `rad/sigrefs` they're aware of -- if it
exists -- and the newly advertised `rad/sigrefs`. This means they can
efficiently ask for the `wants` and `haves` of references that have
changed -- saving some data sent over to the serving side.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-12 13:01:03 +01:00
cloudhead 93ff59ae25
cli: Add `rad version --json` flag
This will be used to automatically populated metadata around releases
for the homepage.
2024-03-11 17:36:31 +01:00
Fintan Halpenny 864ca656c0
cli: add Unknown variant for NotificationKind
When the `rad inbox` command would come across a reference name that
it did not recognise as a `Cob` or `Branch`, then it would fail with
an `Unknown` error.

Instead, add an `Unknown` variant to `NotificationKind` which can be
displayed in the `inbox` CLI, by using the `--show-unknown`
flag. Otherwise, it will be skipped and the inbox will work as usual.

While refactoring, the `TypedId` type is used for the `Cob` variant to
simplify some of its uses.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-11 12:57:53 +01:00
Fintan Halpenny 71d4cb0128
radicle: TypedId improvements
- Add `Serialize` implementation
- Add `is_identity` method

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-11 12:57:53 +01:00
Fintan Halpenny 68339d86b9
node: increase timeouts
For larger fetches, a timeout for the channels and socket control are
required. Bump the timeout from 9s to 30s.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-11 11:28:38 +01:00
Fintan Halpenny 96c65eb2b3
cli: add replies to patch show output
If any replies to patches were made, it was not possible to see them
in the output of `rad patch show`.

Add the replies to show output, but only if the `--verbose` flag is
set.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-03-08 23:43:19 +01:00
Alexis Sellier 786ce40f95
node: Add connection limits
The limits are roughly modeled after the Bitcoin network.
2024-03-08 23:24:34 +01:00
cloudhead 9c9bbbe7bb
node: Expose listen addresses 2024-02-26 12:35:20 +01:00
cloudhead 09f2befa58
radicle: Authorize certain changes if no-op
For backwards compatibility, if a change is a no-op, we allow it even if
it's normall not authorized.
2024-02-23 18:29:07 +01:00
cloudhead 449790e252
radicle: Fix indentation of SQL file 2024-02-23 17:57:32 +01:00
Fintan Halpenny 985b0af3f6
radicle: implement caching for issues and patches
Implement caching for the `Issue` and `Patch` COB. This is achieved by
having the `Transaction::initial` and `Transaction::commit` methods
take a cache which can update the newly created/updated object. The
`Store::remove` method also takes the cache for removing the object
from the cache, as well as the repository.

This meant that the `*Mut` types for the respective COBs are required
to carry a cache alongside the repository store. Identities will not
be cached, and so this is always set to `NoCache` -- which performs no
caching, and always succeeds.

To perform cache reading for issues and patches, the `cache::Issues`
and `cache::Patches` traits are introduced. They capture the minimal
amount of methods that are needed for the `rad issue` and `rad patch`
CLI commands.

The implementor for each of these traits is there respective `Cache`
types, which combines their backing repository store and the SQLite
database. They both provide convenience methods:
- `create` (and `draft` for patches)
- `get_mut`
- `remove`
which can be used instead of their repository store counterparts.

All uses of the repository stores (where needed) are replaced with
`Cache` types.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-02-23 16:03:21 +01:00
Fintan Halpenny b26842d4d8
radicle: add cob::cache module
Add the cob::cache module which describes an SQLite store and schema
for caching COBs. It also provides in-memory and no-cache types for
testing purposes.

The intention is for the store to be used alongside the existing COB
types so that a write-thru cache can be implemented and used for
efficient reads.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-02-23 13:56:19 +01:00
Fintan Halpenny f7ef9cc8b3
radicle: serialization and deserialization improvements
Add derivations of `Serialize` and `Deserialize` for the `Issue` and
`Thread` cob types.

It's necessary to circumvent `Infallible` for the case of the
`Comment` type. This is because `Infallible` does not have these
traits implemented, and there does not seem to be a plan to do so.
Instead, we can use a custom `Infallible` equivalent, which is dubbed
`Never` here.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-02-23 13:56:19 +01:00
Fintan Halpenny 8239a1d0ac
radicle: add cob storage for MockRepository
Ensure that MockRepository can be used where the COB storage traits
are required. Currently, the implementations of the methods themselves
are not required -- they are left as `todo!()`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-02-23 13:56:19 +01:00
Fintan Halpenny a1b41542b8
radicle: rename resolved to is_resolved
The standard in the codebase for boolean methods is to prefix them
with `is_`.

Change `resolved` to `is_resolved`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-02-23 13:56:19 +01:00
Fintan Halpenny 25ff5876ec
radicle: introduce cob::TypedId
The combination of a COB's `TypeName` and `ObjectId` is useful for
parsing a Radicle COB refname, i.e. `refs/cobs/<typename>/<object
id>`.

Introduce `TypedId` for this and provide helper functions for parsing
from the git refname types, as well as being able to tell what kind of
COB it is.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-02-23 13:56:19 +01:00
cloudhead 17744a03fd
cob: Properly authorize root actions
Previously, authorization was only checked for ops after the root op.
We also avoid creating empty actions for `assign` and `label`, which
was needed for non-authorized users to be able to create issues/patches.
2024-02-23 12:31:13 +01:00
cloudhead c63aaaceac
cli: Fix `id` COB notifications in `rad inbox` 2024-02-23 11:35:54 +01:00
cloudhead 6b4f055c36
cli: Don't allow missing config file
It's best to just error if the user is missing the config file, since we
create it on profile init. This also gets rid of the privacy leak.
2024-02-22 16:02:40 +01:00
cloudhead e06dd208f4
cli: `rad seed` output improvement
Better colors, and re-order fields to have policy before scope.
2024-02-22 16:02:38 +01:00
cloudhead d6d90c6fdc
node: Track peer latency
This is useful for a bunch of different network optimizations we're
going to be working on.
2024-02-12 11:36:35 +01:00
cloudhead 664cf20c92
Change default seeding scope to 'all'
This makes the default policy `block` with scope `all`. The good thing
about this is that you don't have to manage individual nodes in the
beginning. You simply seed the repos you want, and everything is
followed within those repos. This matches user expectations, as `rad
seed` does what you expect it to do.

For open-policy seed nodes, it also means that simply changing `block`
to `allow` from the default config is all you need.

This leaves the non-default `followed` scope as a more "advanced"
setting, since it is much more cumbersome: it forces you to follow a
user before you can see their issues/patches, which is not something
most users expect coming from old-shcool forges.
2024-02-12 10:56:21 +01:00
cloudhead fe55de181d
cli: Add new `rad inbox` command
Shows you updates on local repositories.

Included here is a basic implementation of a notification system. A new
database is created and updated after every fetch, with refs that have
changed.

The `rad inbox` command can then list items in the database and delete
them.
2024-02-06 11:40:58 +01:00
cloudhead 98c94de5e3
Various small improvements across crates
Most of these changes should be self-explanatory.

* Atomic fetch was no longer used
* The default policy when initializing should be 'all', as it is with
  `rad init`.
2024-02-06 10:37:44 +01:00
Sebastian Martinez db6cb55d6c
httpd: Add review ids to review json 2024-02-01 17:25:18 +01:00
cloudhead 15d1709831
node: Ensure dequeued fetches are correct
Dequeued fetches were always fetching all refs, instead of fetching the
specific refs that were initially requested. We fix that by storing the
refs to fetch in the fetch queue.
2024-01-31 13:40:13 +01:00
cloudhead f38d2b2ef8
Prevent multiple events per synced peer
We were not properly guarding against the fact that we could receive
multiple sync events from the same node. Normally though, this wouldn't
happen, as further announcements would be stale.
2024-01-31 13:40:13 +01:00
cloudhead 1af505c5c3
Create databases on profile init
Ensures that the databases exist for commands like `rad ls`, even
before the node is started for the first time.
2024-01-31 13:16:09 +01:00
Sebastian Martinez 5333f8217a
httpd: Add reactions to revisions, improve json for comment reactions
Instead of using tuples and lots of arrays, this commit tries to use
`BTreeMap`s to provide more context to review, issues, patch and other
comments.

Also the returned object by the httpd is more aligned with what is
needed by the consumer, so less recomputation is needed for display.
2024-01-31 12:14:05 +01:00
cloudhead 3560e0066d
Fix preferred seeds configuration
When using `network: "test"`, we shouldn't default to any public seed
node. This change makes sure that the "preferredSeeds" configuration var
is only given a value when `network` is `main`.
2024-01-31 12:09:09 +01:00
cloudhead b994c4a6ef
cli: Rework `rad sync`
The command now works with both a replica target *and* a seeds target.
This is especially useful to eg. ensure that a preferred seed has been
synced, while still trying to hit a higher replica count.
2024-01-26 15:22:10 +01:00
cloudhead 131103cb53
Fix code around default policies
Make the code a little clearer and ensure that we are always calling the
`Config` and not the `Store`, so that we fallback on the default policy.
2024-01-25 21:11:08 +01:00
Fintan Halpenny 49584f4e73
radicle: pinned repositories configuration
Introduce pinned repositories to the `profile::Config`, i.e.
`config.json`.

This is used for pinning repositories in the httpd `/projects` handle.

To allow returning all repositories, an `all` query parameter is also
included.

This is non-backwards compatible change, since we introduce the `show`
query parameter which, if not supplied, will default to `pinned`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-01-24 10:55:27 +01:00
cloudhead 47799cbab2
cob: Fix some clippy warnings 2024-01-23 15:50:37 +01:00
cloudhead ae981ded6e
cli: Add `rad stats` command
Returns aggregate stats as JSON. Useful for tracking seed node metrics.
2024-01-23 15:50:32 +01:00
cloudhead 92d6fd4c7e
cob: Re-introduce some of the ignored tests
This reverts commit d8f8d29605.
2024-01-22 21:20:01 +01:00
cloudhead eee304e0b7
cob: Expose operation concurrency
When evaluating a COB, expose to the `apply` function the operations
concurrent to the current one being applied. These are essentially
"related" nodes in the graph, that are neither ancestors or descendants.

This will let us for example have different failure modes depending on
whether there were conflicting operations, as well as expose conflicts
to the user.
2024-01-22 21:20:01 +01:00