Commit Graph

689 Commits

Author SHA1 Message Date
cloudhead 9c9bbbe7bb
node: Expose listen addresses 2024-02-26 12:35:20 +01:00
cloudhead 09f2befa58
radicle: Authorize certain changes if no-op
For backwards compatibility, if a change is a no-op, we allow it even if
it's normall not authorized.
2024-02-23 18:29:07 +01:00
cloudhead 449790e252
radicle: Fix indentation of SQL file 2024-02-23 17:57:32 +01:00
Fintan Halpenny 985b0af3f6
radicle: implement caching for issues and patches
Implement caching for the `Issue` and `Patch` COB. This is achieved by
having the `Transaction::initial` and `Transaction::commit` methods
take a cache which can update the newly created/updated object. The
`Store::remove` method also takes the cache for removing the object
from the cache, as well as the repository.

This meant that the `*Mut` types for the respective COBs are required
to carry a cache alongside the repository store. Identities will not
be cached, and so this is always set to `NoCache` -- which performs no
caching, and always succeeds.

To perform cache reading for issues and patches, the `cache::Issues`
and `cache::Patches` traits are introduced. They capture the minimal
amount of methods that are needed for the `rad issue` and `rad patch`
CLI commands.

The implementor for each of these traits is there respective `Cache`
types, which combines their backing repository store and the SQLite
database. They both provide convenience methods:
- `create` (and `draft` for patches)
- `get_mut`
- `remove`
which can be used instead of their repository store counterparts.

All uses of the repository stores (where needed) are replaced with
`Cache` types.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-02-23 16:03:21 +01:00
Fintan Halpenny b26842d4d8
radicle: add cob::cache module
Add the cob::cache module which describes an SQLite store and schema
for caching COBs. It also provides in-memory and no-cache types for
testing purposes.

The intention is for the store to be used alongside the existing COB
types so that a write-thru cache can be implemented and used for
efficient reads.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-02-23 13:56:19 +01:00
Fintan Halpenny f7ef9cc8b3
radicle: serialization and deserialization improvements
Add derivations of `Serialize` and `Deserialize` for the `Issue` and
`Thread` cob types.

It's necessary to circumvent `Infallible` for the case of the
`Comment` type. This is because `Infallible` does not have these
traits implemented, and there does not seem to be a plan to do so.
Instead, we can use a custom `Infallible` equivalent, which is dubbed
`Never` here.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-02-23 13:56:19 +01:00
Fintan Halpenny 8239a1d0ac
radicle: add cob storage for MockRepository
Ensure that MockRepository can be used where the COB storage traits
are required. Currently, the implementations of the methods themselves
are not required -- they are left as `todo!()`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-02-23 13:56:19 +01:00
Fintan Halpenny a1b41542b8
radicle: rename resolved to is_resolved
The standard in the codebase for boolean methods is to prefix them
with `is_`.

Change `resolved` to `is_resolved`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-02-23 13:56:19 +01:00
Fintan Halpenny 25ff5876ec
radicle: introduce cob::TypedId
The combination of a COB's `TypeName` and `ObjectId` is useful for
parsing a Radicle COB refname, i.e. `refs/cobs/<typename>/<object
id>`.

Introduce `TypedId` for this and provide helper functions for parsing
from the git refname types, as well as being able to tell what kind of
COB it is.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-02-23 13:56:19 +01:00
cloudhead 17744a03fd
cob: Properly authorize root actions
Previously, authorization was only checked for ops after the root op.
We also avoid creating empty actions for `assign` and `label`, which
was needed for non-authorized users to be able to create issues/patches.
2024-02-23 12:31:13 +01:00
cloudhead c63aaaceac
cli: Fix `id` COB notifications in `rad inbox` 2024-02-23 11:35:54 +01:00
cloudhead 6b4f055c36
cli: Don't allow missing config file
It's best to just error if the user is missing the config file, since we
create it on profile init. This also gets rid of the privacy leak.
2024-02-22 16:02:40 +01:00
cloudhead e06dd208f4
cli: `rad seed` output improvement
Better colors, and re-order fields to have policy before scope.
2024-02-22 16:02:38 +01:00
cloudhead d6d90c6fdc
node: Track peer latency
This is useful for a bunch of different network optimizations we're
going to be working on.
2024-02-12 11:36:35 +01:00
cloudhead 664cf20c92
Change default seeding scope to 'all'
This makes the default policy `block` with scope `all`. The good thing
about this is that you don't have to manage individual nodes in the
beginning. You simply seed the repos you want, and everything is
followed within those repos. This matches user expectations, as `rad
seed` does what you expect it to do.

For open-policy seed nodes, it also means that simply changing `block`
to `allow` from the default config is all you need.

This leaves the non-default `followed` scope as a more "advanced"
setting, since it is much more cumbersome: it forces you to follow a
user before you can see their issues/patches, which is not something
most users expect coming from old-shcool forges.
2024-02-12 10:56:21 +01:00
cloudhead fe55de181d
cli: Add new `rad inbox` command
Shows you updates on local repositories.

Included here is a basic implementation of a notification system. A new
database is created and updated after every fetch, with refs that have
changed.

The `rad inbox` command can then list items in the database and delete
them.
2024-02-06 11:40:58 +01:00
cloudhead 98c94de5e3
Various small improvements across crates
Most of these changes should be self-explanatory.

* Atomic fetch was no longer used
* The default policy when initializing should be 'all', as it is with
  `rad init`.
2024-02-06 10:37:44 +01:00
Sebastian Martinez db6cb55d6c
httpd: Add review ids to review json 2024-02-01 17:25:18 +01:00
cloudhead 15d1709831
node: Ensure dequeued fetches are correct
Dequeued fetches were always fetching all refs, instead of fetching the
specific refs that were initially requested. We fix that by storing the
refs to fetch in the fetch queue.
2024-01-31 13:40:13 +01:00
cloudhead f38d2b2ef8
Prevent multiple events per synced peer
We were not properly guarding against the fact that we could receive
multiple sync events from the same node. Normally though, this wouldn't
happen, as further announcements would be stale.
2024-01-31 13:40:13 +01:00
cloudhead 1af505c5c3
Create databases on profile init
Ensures that the databases exist for commands like `rad ls`, even
before the node is started for the first time.
2024-01-31 13:16:09 +01:00
Sebastian Martinez 5333f8217a
httpd: Add reactions to revisions, improve json for comment reactions
Instead of using tuples and lots of arrays, this commit tries to use
`BTreeMap`s to provide more context to review, issues, patch and other
comments.

Also the returned object by the httpd is more aligned with what is
needed by the consumer, so less recomputation is needed for display.
2024-01-31 12:14:05 +01:00
cloudhead 3560e0066d
Fix preferred seeds configuration
When using `network: "test"`, we shouldn't default to any public seed
node. This change makes sure that the "preferredSeeds" configuration var
is only given a value when `network` is `main`.
2024-01-31 12:09:09 +01:00
cloudhead b994c4a6ef
cli: Rework `rad sync`
The command now works with both a replica target *and* a seeds target.
This is especially useful to eg. ensure that a preferred seed has been
synced, while still trying to hit a higher replica count.
2024-01-26 15:22:10 +01:00
cloudhead 131103cb53
Fix code around default policies
Make the code a little clearer and ensure that we are always calling the
`Config` and not the `Store`, so that we fallback on the default policy.
2024-01-25 21:11:08 +01:00
Fintan Halpenny 49584f4e73
radicle: pinned repositories configuration
Introduce pinned repositories to the `profile::Config`, i.e.
`config.json`.

This is used for pinning repositories in the httpd `/projects` handle.

To allow returning all repositories, an `all` query parameter is also
included.

This is non-backwards compatible change, since we introduce the `show`
query parameter which, if not supplied, will default to `pinned`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-01-24 10:55:27 +01:00
cloudhead 47799cbab2
cob: Fix some clippy warnings 2024-01-23 15:50:37 +01:00
cloudhead ae981ded6e
cli: Add `rad stats` command
Returns aggregate stats as JSON. Useful for tracking seed node metrics.
2024-01-23 15:50:32 +01:00
cloudhead 92d6fd4c7e
cob: Re-introduce some of the ignored tests
This reverts commit d8f8d29605.
2024-01-22 21:20:01 +01:00
cloudhead eee304e0b7
cob: Expose operation concurrency
When evaluating a COB, expose to the `apply` function the operations
concurrent to the current one being applied. These are essentially
"related" nodes in the graph, that are neither ancestors or descendants.

This will let us for example have different failure modes depending on
whether there were conflicting operations, as well as expose conflicts
to the user.
2024-01-22 21:20:01 +01:00
cloudhead a46bcb2247
cob: Distinguish between commit parents
Previously, COB change commits would not distinguish between their
parents. This means that they would try to load source code commits
(eg. for patch COBs) and then fail since those commits wouldn't have a
manifest or `Rad-Resource`.

We introduce a backwards compatible change here which is that parent
oids that are not COB commits are accompanied by a commit trailer called
`Rad-Related`. This is for all related content that should be pulled in
but is not part of the change graph proper.

When building the change graph, we simply ignore commits that are marked
as "related".
2024-01-22 21:20:01 +01:00
cloudhead 7feb3e3b10
Use "repository" consistently over "project"
We were using the two pretty interchangeably. "Project" should only be
used to refer to the repository payload. "Repository" should be used
when referring to the resource that is fetched, synced, cloned, checked
out etc.
2024-01-22 21:18:54 +01:00
cloudhead f87dfba9cd
Rename `Id` to `RepoId`
Rename the type to make it clearer and consistent with `NodeId`.
2024-01-22 15:51:44 +01:00
cloudhead 7208d89c31
radicle: Remove unused import 2024-01-22 10:51:44 +01:00
cloudhead 0fbfbc5721
Make sure `preferredSeeds` has a default
If the user already has a config, the preferred seeds list is not
populated. We fix that here by giving it a default value.
2024-01-19 16:30:24 +01:00
Fintan Halpenny 5dbd9bb2a3
radicle: ensure clean up on rad::init error
It is possible that there can be errors while configuring a Radicle
`Repository` during `rad::init`, for example:
- configuring working copy remote fails
- signing references fails
- setting the identity head fails
- setting the project head fails

If any of these occur, a best effort is made to remove the
`Repository` and the working copy remote, so that the caller may fix
any issues and retry.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-01-19 11:35:14 +01:00
cloudhead ebdaf0edd9
General dependency update
Left out some of the `gix-*` dependencies to be done separately after
review.
2024-01-18 13:30:26 +01:00
cloudhead 640d302345
Don't ever decrease rlimit
In case the rlimit is already higher than the desired number, we leave
it as-is.
2024-01-17 17:38:52 +01:00
cloudhead ccc675b304
cli: Have a way of enabling logging in helper
Log to stderr if RUST_LOG is enabled in the remote helper.
2024-01-17 17:34:30 +01:00
cloudhead 045b3e7175
node: Increase process ulimit on start
Users have been having issues with the process soft file limit.
We set that to a reasonably high value to avoid problems.
2024-01-17 17:34:25 +01:00
cloudhead 76302ffec0
cli: Attempt to fix flaky test
We also move the seed config to the test module.
2024-01-17 12:35:04 +01:00
Fintan Halpenny f7b53139f4
radicle: add Config::seed constructor
Add a `Config` constructor for testing seed nodes that uses
`usize::MAX` for capacity.

This allows test scenarios to not drop messages due to rate limiting
-- all nodes are on the same IP address and rate limiting is performed
per IP address.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-01-12 16:21:49 +01:00
cloudhead 1e2776cb08
cli: Output explore URLs on push 2024-01-12 13:24:11 +01:00
cloudhead fd38d88e14
cli: Hint about how to start node
In some cases, we can hint to the user about how they can start their
node.

We also remove the `CallError` type that was redundant.
2024-01-12 11:20:42 +01:00
Fintan Halpenny 2e781b1efd
cli: add control flow for announcement
To allow `rad sync` to break early from the call to `announce`,
control flow is introduced for the callback.

In `rad sync`, the number of synced nodes is checked against the
`replicas` -- if specified -- and sends a `Break`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-01-10 16:03:14 +01:00
Fintan Halpenny 93a6f90d6e
radicle: impl AliasStore for Profile
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-01-09 13:37:04 +01:00
Fintan Halpenny a8cca57c0a
cli: inline patch revision updates
Instead of having a timeline of a patch be global and sorted by
timestamp, organise it via revisions. Each revision has an entry in
the timeline, sorted by timestamp. Under each revision, a series of
updates are inlined, also sorted by their timestamps. Currently, these
updates can either be reviews or merges.

For example:

```
├────────────────────────────────────────────────────────────────────────────┤
│ ● opened by fintohaps (you) 1 week ago                                     │
│   └── ✓ accepted by fintohaps (you) 1 week ago                             │
│ ↑ updated to 5f2a123e3f2aa60f2e03815df283563495b73085 (a1c3c0f) 1 week ago │
│   └── ✓ accepted by fintohaps (you) 1 week ago                             │
│   └── ✓ merged by cloudhead (z6Mkt67…v4N1tRk) now                          │
╰────────────────────────────────────────────────────────────────────────────╯
```

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-01-09 13:06:03 +01:00
cloudhead e4f06e9c9d
cli: Improve `rad remote add`
By default, this command now:
* Follows the peer if not already followed
* Syncs refs of the peer, if it wasn't followed before
* Fetches refs into working copy

This functionality can be disabled via `--no-sync` and `--no-fetch`.
2024-01-08 17:28:01 +01:00
cloudhead 7169e97bd5
cli: Tell the user when canonical head is updated
This is useful especially in multi-delegate repos, since patches can be
merged without the head being updated.
2024-01-05 12:09:30 +01:00
cloudhead 50460842f4
cli-test: Expand env vars in commands 2024-01-05 11:03:24 +01:00
cloudhead 2d71093608
radicle: Small error message improvements 2024-01-04 14:23:53 +01:00
cloudhead 91684cc9ca
cli: Add `rad patch diff` command 2024-01-04 11:49:01 +01:00
Fintan Halpenny 6baefdb13f
cli: use `git` for anonymous fetch
Shell out to `git`, avoiding using `git2`. This is to avoid an
issue where somewhere within the fetch there is an attempt to lookup a
`rad/sigrefs` object, which says that the object is missing. It is
suspected that this is due to the object being localised in the same
packfile as other objects we are fetching. Since the `rad/sigrefs`
object is never needed nor used in the working copy, this will always
result in the object missing. This seems to only be an issue with
`libgit2`/`git2` and not `git` itself.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-01-02 15:04:49 +01:00
Sebastian Martinez 1bb400c278
httpd: Add edits field to revision json 2023-12-22 15:17:25 +01:00
Sebastian Martinez ee3dcfacf0
httpd: Allow edit revision embeds 2023-12-22 15:09:20 +01:00
Sebastian Martinez 5331cf3bcf
radicle: Move `DataUri` to `common::DataUri` 2023-12-22 15:09:20 +01:00
Sebastian Martinez a3a8060e5c
radicle: Add `Vec<Edit>` to revision description
This allows us to have embeds on a patch revision, and eventually also
show the edits that have been made to a revision description.
2023-12-22 15:08:35 +01:00
Fintan Halpenny 2dea54c6ee
cob: mandatory `Oid` in `CodeLocation`
To make `CodeLocation` reusable across different components, require
that a commit's `Oid` is part of the struct.

An `Oid` is necessary since the `path` can refer to a path at any
point in time throughout the history of the repository. By pinning the
commit that is being referred to, the path will refer to the path
within the associated tree [of the commit].

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-22 15:00:42 +01:00
Sebastian Martinez 1f36879aa2
cob: improve Comment generics
`Infallible` is a more appropriate type in comparsion to `()` for
comments that do not require a `CodeLocation`.

That is because if the type for `location` is `Option<()>`, this means
that `Some(())` can be constructed, however, we never intend to
construct that variant. On the other hand, `Infallible` can never be
constructed, and so, only `None` can be used.

This requires updates to some functions to use the generic `T` and
also handle the different kinds of `Comment`s -- `Infallible` and
`CodeLocation`.

X-Co-Authored-By: Fintan Halpenny <fintan.halpenny@gmail.com>
2023-12-22 14:09:02 +01:00
Fintan Halpenny 47a440a384
radicle: expose shared logger
To be able to use log setup across the different components of
heartwood, expose a `logger` module from `radicle`. This also includes
a `test` logger.

These modules are guarded behind a `logger` feature flag, so that
other crates are not bringing any uneeded dependencies.

This allows logging to be setup in `radicle-cli-test` so that when a
test fails, the log output is present for debugging purposes.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-20 15:11:10 +01:00
Fintan Halpenny a6ba1b0103
node: inspect outdated sigrefs test
This test is flakey and requires further investigation. Some extra
assertions are added to ensure that the expected behaviour is
happening upon fetching. It is unknown if the fetch queue is behaving
properly, and it's suspected that some existing fetch results might be
used for queued fetches.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-19 15:02:14 +01:00
cloudhead 13db55014b
cli: Show URL on `rad init` 2023-12-19 14:15:30 +01:00
cloudhead 6eb2772235
node: Improve flexibility of test environment
Allows us to pass full configs (profile or node) when creating test
nodes or profiles.
2023-12-19 14:15:30 +01:00
cloudhead 59f506dbb5
radicle: Fix bug in node.db query 2023-12-19 13:26:17 +01:00
cloudhead 6f7c2dca91
node: Introduce connection penalty system
Previously, we might ban a node if we're having trouble connecting to
it, but this could be due to internet connectivity issues.

We introduce a penalty score for nodes, that increases if there are
connection issues or the node misbehaves, and decreases on successful
connections.

When connected to new nodes, we sort them so that low-penalty nodes are
attempted first.
2023-12-15 16:00:09 +01:00
cloudhead d9a84de5ee
radicle: Simple database migration system
We create a system for migrating the our databases, as well as a first
migration, for adding a `penalty` field to the `nodes` table, which we
are going to need soon, as an example.
2023-12-15 12:04:54 +01:00
cloudhead cc380532c6
cli: Improve messaging when config doesn't load
Instead of telling users to setup their profile, point out the actual
problem with the config.
2023-12-13 12:30:11 +01:00
cloudhead ecb59d687b
cob: Return `RevisionId` instead of `Revision`
Previously we panicked if a revision wasn't found after an update. But
it can happen that the update goes through yet a concurrent update
prevents the revision from being valid.

To partially deal with this, we don't try to lookup the revision after
an update, we simply return the ID. This means a subsequent lookup can
fail, and the error can be handled there.
2023-12-13 12:14:06 +01:00
cloudhead 6ca7da2768
radicle: Rename many more methods using new naming 2023-12-06 10:02:00 +01:00
cloudhead 1b026ae0ee
node: Rename node `tracking` module 2023-12-06 10:00:45 +01:00
cloudhead e65595c4c7
node: Rename commands with new "seed" nomenclature 2023-12-06 09:56:58 +01:00
cloudhead f453090e27
cli: Consolidate policy/scope variants
* Policy::Track -> Policy::Allow
* Scope::Trusted -> Scope::Followed
2023-12-06 09:56:58 +01:00
Fintan Halpenny dce5316cbb
radicle: rename Issue::assigned -> Issue::assignees
Align the naming with the field itself and Patch::assignees.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-06 09:48:02 +01:00
Sebastian Martinez f8005c74c6
httpd: Add comment edits to serialized json
This allows the display of any edits done to comments,
replies and issue descriptions.

This commit doesn't remove the body property
which is the latest comment body but adds a edits property.

Also adds an author property to the `Edit` struct,
which allows us to see the author of an edit action.
2023-12-05 16:01:50 +01:00
Fintan Halpenny 924a2624a4
git: update git2-0.18.1
Update to git2-0.18.1, which also requires updating radicle-git-ext
and radicle-surf.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-04 11:11:36 +00:00
cloudhead af095d53ca
radicle: Add foreign-key constraint for routing 2023-11-30 13:00:59 +01:00
cloudhead 6b04eff34c
radicle: Consolidate databases
Instead of having a routing database, and an addresses database,
consolidate that in a single database called `node.db`.

We also simplify database access, by providing a single type for opening
the database, and use traits to access the various functions.

We create a new `seed::Store` that has seed-related methods split out of
the `address::Store`.

Note that tracking is still in its own database currently, as it isn't
ephemeral state. This makes it easy to delete the new `node.db` without
risk of losing any important information.

To avoid renaming all database methods, we create a `Stores` wrapper
that allows us to access routing, addresses, seeds etc. without type
ambiguity.

This is a breaking change and requires `addresses.db` and `routing.db`
to be deleted.
2023-11-30 13:00:59 +01:00
Fintan Halpenny 72b00e3d41
cli: sort-by flag for rad sync
Allow the caller of `rad sync` to `--sort-by` the NID, alias, or
status of the seeds.

This prevents random ordering, due to timestamps, in CLI testing.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-30 12:46:18 +01:00
cloudhead 538ca1e44d
cob: Don't lose state when lifecycling
It was previously possible to switch a merged patch to "open", losing
the merge state.
2023-11-30 10:53:09 +01:00
Fintan Halpenny 5b1cdbaa4a
radicle: remove repo iff there's no local rad/sigrefs
If the local peer does not have any `rad/sigrefs` then there is no
risk of any data corruption. In this case the repository can be
removed entirely.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-30 10:48:14 +01:00
Fintan Halpenny 5f96429005
radicle: remove `WriteStorage::remove`
The `remove` method is no longer being used, with `clean` taking its
place.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-30 10:48:14 +01:00
cloudhead ad7a3addb6
cli: Add `rad config` command
Displays the current configuration.

Also move to our tree-sitter highlighter for pretty-printing JSON and
remove the `json-color` dependency.
2023-11-30 10:46:32 +01:00
cloudhead 59a08f21a4
radicle: Explorer and preferred seed configuration
Allows users to configure the default node explorer to use.
Allows users to configure their preferred seeds.
2023-11-30 10:46:32 +01:00
Fintan Halpenny d23120ca8b
radicle: add clean method
Add a `clean` method to remove the remotes of a repository that are
not the local peer nor the delegates.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-24 10:48:05 +00:00
cloudhead 53013f306f
cli: Improve `rad ls`
Distinguish between repos that are simply tracked, vs. repos that are
cloned (and thus forked). By default, we don't show repos that don't
have a local fork.
2023-11-23 13:51:47 +01:00
cloudhead 2917198d1c
radicle: Fix `config.json` issue
This allows the config to load without a `cli` key.
2023-11-23 13:51:47 +01:00
cloudhead 8e76ce03d5
cli: Hints system
Enable hints by default, but allow users to turn them off via
configuration.

Add a hint when pushing while offline.
2023-11-23 11:07:07 +01:00
cloudhead dfe0cb4342
cli: Polish the `rad sync status` output 2023-11-22 17:40:51 +01:00
Fintan Halpenny 81dc849bce
cli: add name option to rad patch checkout
Add the `--name` option to `rad patch checkout`. This allows the user
to name the branch they are checking out, rather than use the
`patch/<short-id>` format.

This change includes returning early for setting the upstream branch,
if the upstream already existed.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-21 10:19:25 +01:00
cloudhead 358324883c
node: Announce offline ref updates
Currently, if refs are updated while the node is stopped, these refs are
not immediately announced when the node starts, due to the node being
unaware of the updates.

To remedy this, on startup, we compare the refs we have in storage with
the refs we announced to the network. Since only our own refs could have
changed while the node was stopped, this change only applies to owned
refs. If we detect that storage has changed while the node was stopped,
we create ref announcements for the updated repos and store them in our
gossip database. These announcements will be included with other
historical gossip announcements when nodes connect to us.

To keep track of what was announced, we store our local node's sync
status in the database. This is also useful when using the `rad sync status`
command, as it tells us what oid our local node is at, and whether it
is aware of our local changes yet.

In addition to this, we make a small fix: previously, inventory and ref
announcements originating from our local node were not stored in the
announcement database. We ensure that they are via the new `announce`
method.
2023-11-20 17:54:36 +01:00
cloudhead b140d8580f
cli: Don't announce if already in sync
In this change, we check whether we are in sync with a node before
announcing refs to it.
2023-11-17 17:28:51 +01:00
cloudhead d4ec2176bb
Update Rust toolchain to 1.74 2023-11-17 16:58:24 +01:00
cloudhead d3b0483f42
cli: Implement `rad sync status` command
This command shows the sync status of a repository by consulting the
database via the node.

To make it work, we call into the address store to update the
`repo-sync-status` table when receiving ref announcements that are
relevant to us.
2023-11-17 16:56:19 +01:00
cloudhead ca557909c2
radicle: Fix some serialization issues
For some inexplicable reason, deserializing doesn't work unless we use
our custom de-serializer for `LocalTime`.
2023-11-17 16:54:27 +01:00
cloudhead aa13f57e7d
radicle: Fix issues with the address store SQL 2023-11-17 16:54:27 +01:00
cloudhead bf64fde8a1
cli: Improve timestamp format function 2023-11-17 16:54:27 +01:00
Fintan Halpenny 5cb4ad4622
node: `info` announcements
An informational message for the protocol is useful for sending
messages to a node that is not required to be relayed.

This message is introduced by starting with an ACK message for a set
of references being synced. This fixes an issue for announcing to a
node that is already synced, since it can now ACK that it received the
message and replies that it is synced.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-17 16:29:29 +01:00
Fintan Halpenny 85d452b222
radicle: return RefsAt for announce_refs
It can be useful to return the `RefsAt` that was announced for this
node, for example, to compare it to node events.

Change the `Handle` method to return `RefsAt` so that when it is
called it returns the local node's `RefsAt`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-17 16:29:02 +01:00
Fintan Halpenny ac849f1fd5
radicle: add ReadRepository::contains
It can be useful to be able to check if a repository contains a given
commit. Add a `contains` method to the `ReadRepository` trait to
handle this.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-17 16:09:42 +01:00
cloudhead cdcc7cfaf9
node: Emit events for gossip messages 2023-11-17 14:56:42 +01:00
cloudhead 39f65e1b29
Update `netservices` and `cyphernet` dependencies 2023-11-17 12:23:47 +01:00
Fintan Halpenny e71f07e592
radicle: temporary repository helper
It's possible that a temporary directory can be created on a different
mount, and when moved during a clone, creates an I/O error.

To prevent this, add a method to `Storage` which creates a temporary
directory and `Repository` with the path in the form
`<rid>.lock`.

This `.lock` path name can easily be skipped when listing
repositories. Further tooling can be added for detecting leftover
`.lock` directories if, for some reason, they were not cleaned up by
the `tempfile` crate.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-16 12:01:10 +00:00
Fintan Halpenny fbc5c5a027
radicle: Check signed refs are unchanged
Instead of creating the tree to check if `rad/sigrefs` has changed, it
is possible to check that the signature of the signed refs being
updated is the same as the old one. If so, then there are no
changes. Otherwise, there must be changes and the update can continue
to create the necessary Git objects.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-16 11:05:41 +01:00
cloudhead ea4104dbd7
node: Only process messages from known nodes
Previously, we would process any valid inventory or refs announcement.
This could cause problems for keeping sync statuses, as we could get
a refs announcement from a node that isn't in our database.

In practice, there is no good reason why we should receive a refs or
inventory announcement before the announcer's node announcement, unless
nodes are not following the protocol.

Implementing this change meant that many of the tests needed updating,
as it wasn't assumed that a node announcement was needed before
receiving inventories etc. from a peer.

We also had to implement a basic form of node address banning to make
the tests pass after this change.
2023-11-13 16:15:57 +01:00
cloudhead 0ae3c00cd7
Use cascading deletes for removal of node rows
Instead of manually removing all rows in the address table, use
cascading deletes.
2023-11-13 16:01:18 +01:00
cloudhead 269d21deca
node: Rewrite `seeds` method to use sync-status
We update the `seeds` method to use not only the routing table, but also
the sync status information for our owned refs.
2023-11-13 12:06:37 +01:00
cloudhead 9f45405c64
Add table and types for tracking sync status
We add the necessary database table to keep track of which node has
synced which of our repos. This will be useful for giving the user
information on the sync status of every local repository.
2023-11-13 12:03:24 +01:00
cloudhead b675185538
node: Improve control socket API
We improve the API so that error messages can be sent back and are
always expected as potential responses to commands. This means that
`CommandResult` is always the type that is sent for commands.

This includes some breaking changes, for example the default "success"
message is simply `{}` instead of `{"status":"ok"}`.
2023-11-13 11:02:45 +01:00
cloudhead d88cce484a
radicle: Simplify canonical head quorum
Instead of trying to handle cases where delegate branches diverge, we
simply throw an error in those cases, which prevents the canonical head
from updating.

This simplifies the code a lot, and forces the user to take action when
such an event occures.
2023-11-10 13:42:16 +01:00
Fintan Halpenny d27cd0db0c
node: minimise refs announcement to key and Oid
The RefsAnnouncement previous announced its complete set of
`rad/sigrefs`. This was useful in that verification can occur on the
receiveing side immediately by verifying the refs and proceeding with
the fetch -- note that it did nothing with the ref information it
received.

This can be minimised to only advertising the namespace, i.e. the
NID, and the new SHA of the `rad/sigrefs`. This is represented by a
new message type, `RefsAt`, which contains the NID and the Oid.

For checking if an announcement is stale or fresh, the receiving
side can check the existence of the SHA in its Git repository.

The tests are appropriately modified to use NIDs and Oids.

To prepare for a follow-up change, the announcement data is also
threaded through for the fetch protocol to focus fetching only the
advertised set of Oids.

N.B.: because the refs aren't communicated, the verification happens
when fetching since the data needs to be fetched before
verifying. This is a trade-off with minimising the message payload for
verifying earlier.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-09 17:47:09 +01:00
Fintan Halpenny 61b06553b4
radicle: use git::raw::Error for reference_oid
The minimal error for the `ReadRepository::reference_oid` method is
`git::raw::Error`.

Change the error type from `git::ext::Error` to `git::raw::Error`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-09 17:47:07 +01:00
Fintan Halpenny 82a53e3a53
radicle: UserInfo for Repository::create
The fetch V2 improvements introduced a `UserInfo` type that is used to
set the Git config's name and email when cloning a repository. This
`UserInfo` can also be used when creating a new `Repository`.

The `Storage` type now takes a `UserInfo` so that it can be passed to
`Repository::create`, for creating a new repository.

For testing, this requires adding a new `fixtures::user` so that
testing `Storage` instances can be easily constructed.

It also means that the `radicle-cli` tests all changed with new SHAs
since the name and email are used as part of the hashing input.

N.b. the order of the timelines while testing the identity cob
changed. It's not clear why that is.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-09 16:43:34 +01:00
cloudhead 9b7c48ab40
cli: Improve `rad diff`
* Properly show the user when a file was moved or copied.
* Don't require a radicle project in the current repo.
2023-11-09 13:04:40 +01:00
cloudhead f0754330c9
Update `sqlite` dependency to 0.32.0 2023-11-08 15:08:01 +01:00
cloudhead f9df360171
node: Persist historical gossip messages
Instead of storing received gossip messages in an in-memory `BTreeMap`,
we persist them to the database (`addresses.db`). There are two reasons
for this:

1. Node restarts do not wipe the state, meaning that it is much less
   likely that some gossip message will get lost before reaching all
   nodes.
2. This will allow us in the next commit to know how far behind we are
   after a restart, to be able to request the correct time range of
   gossip messages that we missed while offline.

Note that we now share the addresses.db file between two connections and
stores, which we don't do anywhere else. Having multiple connections to
the same database is fine, but the way we're having to do it is a bit
awkward. In a future patch, we could pull out the database schema from the
`address` module and rename the database to make things clearer.
2023-11-08 13:15:56 +01:00
Sebastian Martinez 193ed2f675
radicle: Add missing embeds in `RevisionComment` 2023-11-06 14:51:33 +01:00
Fintan Halpenny 981172aa4f
radicle-fetch
The `radicle-fetch` crate introduces a method for fetching from a
server using Git protocol V2, which provides the implementation
control over performing a sequence of fetches and validation over
domain data.

The protocol is built on top of the suite of Gitoxide crates[[0]]. It
builds wrappers on top of Gitoxide's refdb, odb, and transport layers
-- as well as taking the ls-refs and fetch code straight from the code
base to simplify for the needs of this crate.

[0]: https://github.com/Byron/gitoxide

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny 87187d47e5
radicle: add remove to WriteStorage
Add the ability to remove a repository, given its RID.

This is useful in the case of attempting to clone a repository, and if
that process fails -- for example, if the repository is private --
then it's necessary to clean up the initially created repository.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny f1f32d8574
radicle: return verified doc
The `identity_doc_at` and `identity_doc` methods on `ReadRepository`
returned the unverified identity document. Every usage of these
methods would always verify the document after.

Simplify the code by calling verified in the methods and directly return the
verified identity document.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny adf3130431
radicle: add Special type
Add an enum to represent the special references, `rad/id` and
`rad/sigrefs`. These can be used for cases where it's more manageable
to work with the enum type over checking for reference string equality.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny 75b52e28b2
radicle: separate repository validation
Make the validation logic customisable by extracting it out into a
separate trait that can be defined by other crates.

The Repository validation logic stays the same.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny b503aa6be4
radicle: validations
Instead of returning the unsigned references, the validation of a
repository's remotes returns the set of validation errors. The caller
is then expected to handle these validation errors appropriately, for
example, by logging them and re-raising an error.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny dfdf2e20cc
radicle: add UserInfo to git
Add a type to capture user information that can be used for Git
configuration and signatures.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny ef6ae0fa84
radicle: move service tracking config to radicle::node
The `service::tracking` module has nothing specific to the
radicle-node code. This means that it can be moved to radicle under
the `node::tracking` module.

This makes it more resuable in other contexts, for example, for
fetching logic.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny c06b225877
radicle: add sigrefs ref name constructor
Similar to the `id` function, add a `sigrefs` function to construct
the reference name for a given remote's `rad/sigrefs`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny 030419932e
radicle: Add SignedRefsAt type for signed refs
Introduce a SignedRefsAt type that combines a SignedRefs along with the Oid
it was loaded at.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
cloudhead d150938b38
cli: Update tracking without a running node
This change allows users to track/untrack peers and repos without a
running node. This fixes a bug when running `rad init` without the node
running, where the repo would be tracked with a different scope than
what was specified on the CLI.
2023-11-03 17:17:07 +01:00
cloudhead a70664368e
radicle: Move some functions to `Home` type
Certain functions in `Profile` did not need to be in there and could be
moved in the more basic `Home` type.
2023-11-03 17:08:18 +01:00
cloudhead 84a95848b0
cli: Add pager to `rad diff`
Invokes a pager when the output doesn't fit on the screen.
2023-11-03 14:20:16 +01:00
cloudhead d8f8d29605
radicle: Fix timestamp issue with feature flag
The `stable-commit-ids` feature flag was being included in non-dev,
non-test builds because of how cargo works (or doesn't work).

For now, we ignore the tests that this affects, and they can be run with
the `RAD_COMMIT_TIME=1514817556` environment.
2023-11-01 17:24:47 +01:00
Michalis Zampetakis 0b6ede69e7
node: Add `listen` option to node config 2023-11-01 12:59:29 +01:00
Fintan Halpenny c82344496b
radicle: Improve cwd to search parents
If a `rad` command that requires an RID is executed within a
subdirectory of a Radicle project, then it will fail saying:

    ✗ Error: Current directory is not a radicle project

Improve this flow by searching parent directories and/or using the
GIT_DIR environment variable.

This is safe as long as the operations are read-only when calling
`cwd`. Currently they are:
- `Repository::open_ext` -- will not create a repository, only open
  one.
- `Repository::find_remote` -- this will only read the remote from the
  config and get the URL to parse the RID.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-10-30 15:29:37 +01:00
Fintan Halpenny 199fa07a79
radicle: Apply clippy suggestions
`cargo clippy --all --all-targets` is warning that there are multiple
redundant clones.

Apply the suggestions by removing the calls to `clone`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-10-30 14:51:21 +01:00
Fintan Halpenny 2584ecdb0a
radicle: Pin timestamp for testing
When testing the CLI, "since"-calculations will result in different
values over time:

    now
    1 month ago
    2 months ago
    10 months ago
    1 year ago
    10 years ago

Initially, the solution was to use the wildcard match from `snapbox`,
`[..]`, to match on these values. However, since these values change
in length it means that they will affect the width of the tables
produced by the CLI.

To ensure the width is stable across runs over time it's possible to
use a time that is the same as the commit time used at testing.

To do this the `Timestamp` type is now a newtype which has a
constructor `now` which will use the `RAD_COMMIT_TIME` env variable
for construction, if it's available and in debug mode. This ensure
that all "since"-calculations will result in `now`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-10-30 14:48:38 +01:00
cloudhead 1d167581f2
cob: Introduce `xyz.radicle.id` COB
Introduces a new COB type to store repository identity documents.

The reason for this change is to:

1. Simplify the code, as the identity document logic resembled a COB,
   yet it had custom logic. This allows existing COB code to be used for
   identities.
2. Make identity document update logic more flexible, since COB actions
   can be added in the future.
3. Re-purpose existing tools around COBs to work on identities, eg. `rad
   cob`.
4. Unify the concept of an identity change proposal, with regular identity
   changes. This means we can remove the `id.proposal` COB in favor of
   using the `id` COB itself.

Notes
-----

* Each repository has one Identity COB.
* The `Proposal` COB has been repurposed into the `Identity` COB.
* Identity documents are stored as *embeds* inside the Identity COB
  actions.
* The action that contains new document versions is called `revision`,
  just like the Patches COB.
* The namespaced `rad/id` ref is a symbolic reference to that Identity
  COB.
* The canonical `rad/id` ref is a direct reference to the commit in the
  Identity COB that contains the latest *accepted* revision of the
  document.
* All commands for managing identities have been folded into `rad id`.
  Hence `rad delegate` and `rad edit` are removed.
* The concept of "rebasing" an identity document is gone.
* The `rad id` output has been updated to match the style of other
  commands.
* When a revision has enough signatures, it is automatically adopted as
  the current identity, there is no longer the need for a "commit"
  action.
* When an identity revision is proposed, and the current identity has a
  threshold of `1`, that identity is automatically accepted due to the
  above point.
* The idea of "verifying a peer's identity branch" no longer applies, as
  COBs cannot be verified one history at a time.
* Since the root commit of the Identity COB does not have a "Resource"
  to point to (it would have to point to itself, which is impossible),
  we've made the resource id optional for COBs.
2023-10-12 17:32:33 +02:00
cloudhead c99d9a6aed
radicle: Make mock transport thread-safe
The mock transport was not thread-safe when multiple threads used the
same node ids. This is fixed by including the thread id as part of the
storage key.
2023-10-11 14:53:59 +02:00
cloudhead 5709f5d886
cob: Improve stability of test setup
We employ various tricks to ensure that tests are reproducible.
In this case, having stable commit ids allows us to test specific
traversal orders of a COB history in cases of concurrent updates.
2023-10-11 14:53:59 +02:00
Sebastian Martinez 78e6833672
httpd: Add missing COB actions
Handles missing issue and patch actions. Adds embeds to patch comments.

Signed-off-by: Sebastian Martinez <me@sebastinez.dev>
2023-10-11 13:36:48 +02:00
Alexis Sellier c0271c303f
Prevent divergences in canonical head
On push, we check whether the resulting state would cause a
divergence/fork in the canonical head, and if so, prevent the push from
happening.

This is to avoid situations where delegates have to then rollback their
heads.

Note that this doesn't prevent forks from happening altogether, as they
could happen asychronously, but it mitigates the problem.
2023-09-27 12:25:14 +02:00
cloudhead 883b548c61
Improve computation of canonical head
We employ a better algorithm for determining which commit should be
canonical when there are multiple possible options.

The algorithm still doesn't work in certain cases when there are
multiple merge commits as candidates, with shared parents.

We leave some (ignored) failing tests to be resolved in the future.
2023-09-27 12:25:14 +02:00
cloudhead 1376ab63c0
cli: Allow checking out a specific revision
Allow `rad patch checkout` to specify a revision to checkout that may
not be the patch's latest.
2023-09-21 16:18:24 +02:00
cloudhead c94bd50666
cob: Better distinguish patch revisions
The `Patch` API and semantics are changed a little bit:

* It's now valid again to propose a revision without being the patch
  author.
* To get revisions by the patch author, `Patch::updates` is used.
* `Patch::latest` now only looks at revisions by the patch author.
* Added `Patch::revisions_by`, `Patch::authors` and `Patch::latest_by`.
* The merge behavior on git-push is improved to favor revisions that
  merge more recent commits. We also allow for merging the non-latest
  revision.
2023-09-21 13:51:55 +02:00
cloudhead 5078396028
cli: Move commenting to patch/issue commands
Instead of having `rad comment`, move the command to the respect COBs,
ie. `rad issue comment` and `rad patch comment`.
2023-09-19 14:33:40 +02:00
cloudhead 8cae60371c
cli: Simplified issue opening
We remove the YAML front-matter, since that was annoying to use.
Issues are now edited like patches and commit messages.

Labeling and assigning are done via dedicated commands.
2023-09-19 14:22:45 +02:00
Alexis Sellier 1abce45d5d
node: Slightly increase rate limit defaults 2023-09-18 11:14:19 +02:00
Alexis Sellier a7cd7b25f5
node: Allow rate-limitting to be configurable 2023-09-18 11:14:19 +02:00
Alexis Sellier af724eeb0d
cli: Add `node config` command
Shows the current (loaded) node configuration.
2023-09-18 11:14:19 +02:00
cloudhead 457b82f723
cob: Evaluate COB state eagerly
Instead of returning histories from `radicle-cob`, we return an
evaluated object which we build during initial graph traversal.

We do this so that branches can be correctly pruned when operations are
invalid at the application level. This way, new operations are not
building on top of invalid ones contained in the history.

To achieve this, we introduce a new `Evaluate` trait that is implemented
by all COBs, and we make change graph evaluation fallible.

Doing this means that we get `apply` errors returned for free when an
invalid transaction is applied, and no longer need to check for action
validity in multiple places.

We also implement a new `Dag::prune` method to avoid having to copy
graph nodes during traversal.
2023-09-11 23:25:28 +02:00
cloudhead 65fa3cbb1b
radicle: Make `DraftStore` generic over the repo
This will allow using it inside of COBs.
2023-09-10 20:44:38 +02:00
cloudhead 6695074479
cob: Don't require full `Entry` for update
Instead of taking the full entry type, we only need the id.
2023-09-10 20:44:38 +02:00