Commit Graph

556 Commits

Author SHA1 Message Date
cloudhead 98c94de5e3
Various small improvements across crates
Most of these changes should be self-explanatory.

* Atomic fetch was no longer used
* The default policy when initializing should be 'all', as it is with
  `rad init`.
2024-02-06 10:37:44 +01:00
Sebastian Martinez db6cb55d6c
httpd: Add review ids to review json 2024-02-01 17:25:18 +01:00
cloudhead 15d1709831
node: Ensure dequeued fetches are correct
Dequeued fetches were always fetching all refs, instead of fetching the
specific refs that were initially requested. We fix that by storing the
refs to fetch in the fetch queue.
2024-01-31 13:40:13 +01:00
cloudhead f38d2b2ef8
Prevent multiple events per synced peer
We were not properly guarding against the fact that we could receive
multiple sync events from the same node. Normally though, this wouldn't
happen, as further announcements would be stale.
2024-01-31 13:40:13 +01:00
cloudhead 1af505c5c3
Create databases on profile init
Ensures that the databases exist for commands like `rad ls`, even
before the node is started for the first time.
2024-01-31 13:16:09 +01:00
Sebastian Martinez 5333f8217a
httpd: Add reactions to revisions, improve json for comment reactions
Instead of using tuples and lots of arrays, this commit tries to use
`BTreeMap`s to provide more context to review, issues, patch and other
comments.

Also the returned object by the httpd is more aligned with what is
needed by the consumer, so less recomputation is needed for display.
2024-01-31 12:14:05 +01:00
cloudhead 3560e0066d
Fix preferred seeds configuration
When using `network: "test"`, we shouldn't default to any public seed
node. This change makes sure that the "preferredSeeds" configuration var
is only given a value when `network` is `main`.
2024-01-31 12:09:09 +01:00
cloudhead b994c4a6ef
cli: Rework `rad sync`
The command now works with both a replica target *and* a seeds target.
This is especially useful to eg. ensure that a preferred seed has been
synced, while still trying to hit a higher replica count.
2024-01-26 15:22:10 +01:00
cloudhead 131103cb53
Fix code around default policies
Make the code a little clearer and ensure that we are always calling the
`Config` and not the `Store`, so that we fallback on the default policy.
2024-01-25 21:11:08 +01:00
Fintan Halpenny 49584f4e73
radicle: pinned repositories configuration
Introduce pinned repositories to the `profile::Config`, i.e.
`config.json`.

This is used for pinning repositories in the httpd `/projects` handle.

To allow returning all repositories, an `all` query parameter is also
included.

This is non-backwards compatible change, since we introduce the `show`
query parameter which, if not supplied, will default to `pinned`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-01-24 10:55:27 +01:00
cloudhead 47799cbab2
cob: Fix some clippy warnings 2024-01-23 15:50:37 +01:00
cloudhead ae981ded6e
cli: Add `rad stats` command
Returns aggregate stats as JSON. Useful for tracking seed node metrics.
2024-01-23 15:50:32 +01:00
cloudhead 92d6fd4c7e
cob: Re-introduce some of the ignored tests
This reverts commit d8f8d29605.
2024-01-22 21:20:01 +01:00
cloudhead eee304e0b7
cob: Expose operation concurrency
When evaluating a COB, expose to the `apply` function the operations
concurrent to the current one being applied. These are essentially
"related" nodes in the graph, that are neither ancestors or descendants.

This will let us for example have different failure modes depending on
whether there were conflicting operations, as well as expose conflicts
to the user.
2024-01-22 21:20:01 +01:00
cloudhead a46bcb2247
cob: Distinguish between commit parents
Previously, COB change commits would not distinguish between their
parents. This means that they would try to load source code commits
(eg. for patch COBs) and then fail since those commits wouldn't have a
manifest or `Rad-Resource`.

We introduce a backwards compatible change here which is that parent
oids that are not COB commits are accompanied by a commit trailer called
`Rad-Related`. This is for all related content that should be pulled in
but is not part of the change graph proper.

When building the change graph, we simply ignore commits that are marked
as "related".
2024-01-22 21:20:01 +01:00
cloudhead 7feb3e3b10
Use "repository" consistently over "project"
We were using the two pretty interchangeably. "Project" should only be
used to refer to the repository payload. "Repository" should be used
when referring to the resource that is fetched, synced, cloned, checked
out etc.
2024-01-22 21:18:54 +01:00
cloudhead f87dfba9cd
Rename `Id` to `RepoId`
Rename the type to make it clearer and consistent with `NodeId`.
2024-01-22 15:51:44 +01:00
cloudhead 7208d89c31
radicle: Remove unused import 2024-01-22 10:51:44 +01:00
cloudhead 0fbfbc5721
Make sure `preferredSeeds` has a default
If the user already has a config, the preferred seeds list is not
populated. We fix that here by giving it a default value.
2024-01-19 16:30:24 +01:00
Fintan Halpenny 5dbd9bb2a3
radicle: ensure clean up on rad::init error
It is possible that there can be errors while configuring a Radicle
`Repository` during `rad::init`, for example:
- configuring working copy remote fails
- signing references fails
- setting the identity head fails
- setting the project head fails

If any of these occur, a best effort is made to remove the
`Repository` and the working copy remote, so that the caller may fix
any issues and retry.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-01-19 11:35:14 +01:00
cloudhead 640d302345
Don't ever decrease rlimit
In case the rlimit is already higher than the desired number, we leave
it as-is.
2024-01-17 17:38:52 +01:00
cloudhead ccc675b304
cli: Have a way of enabling logging in helper
Log to stderr if RUST_LOG is enabled in the remote helper.
2024-01-17 17:34:30 +01:00
cloudhead 045b3e7175
node: Increase process ulimit on start
Users have been having issues with the process soft file limit.
We set that to a reasonably high value to avoid problems.
2024-01-17 17:34:25 +01:00
cloudhead 76302ffec0
cli: Attempt to fix flaky test
We also move the seed config to the test module.
2024-01-17 12:35:04 +01:00
Fintan Halpenny f7b53139f4
radicle: add Config::seed constructor
Add a `Config` constructor for testing seed nodes that uses
`usize::MAX` for capacity.

This allows test scenarios to not drop messages due to rate limiting
-- all nodes are on the same IP address and rate limiting is performed
per IP address.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-01-12 16:21:49 +01:00
cloudhead 1e2776cb08
cli: Output explore URLs on push 2024-01-12 13:24:11 +01:00
cloudhead fd38d88e14
cli: Hint about how to start node
In some cases, we can hint to the user about how they can start their
node.

We also remove the `CallError` type that was redundant.
2024-01-12 11:20:42 +01:00
Fintan Halpenny 2e781b1efd
cli: add control flow for announcement
To allow `rad sync` to break early from the call to `announce`,
control flow is introduced for the callback.

In `rad sync`, the number of synced nodes is checked against the
`replicas` -- if specified -- and sends a `Break`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-01-10 16:03:14 +01:00
Fintan Halpenny 93a6f90d6e
radicle: impl AliasStore for Profile
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-01-09 13:37:04 +01:00
Fintan Halpenny a8cca57c0a
cli: inline patch revision updates
Instead of having a timeline of a patch be global and sorted by
timestamp, organise it via revisions. Each revision has an entry in
the timeline, sorted by timestamp. Under each revision, a series of
updates are inlined, also sorted by their timestamps. Currently, these
updates can either be reviews or merges.

For example:

```
├────────────────────────────────────────────────────────────────────────────┤
│ ● opened by fintohaps (you) 1 week ago                                     │
│   └── ✓ accepted by fintohaps (you) 1 week ago                             │
│ ↑ updated to 5f2a123e3f2aa60f2e03815df283563495b73085 (a1c3c0f) 1 week ago │
│   └── ✓ accepted by fintohaps (you) 1 week ago                             │
│   └── ✓ merged by cloudhead (z6Mkt67…v4N1tRk) now                          │
╰────────────────────────────────────────────────────────────────────────────╯
```

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-01-09 13:06:03 +01:00
cloudhead e4f06e9c9d
cli: Improve `rad remote add`
By default, this command now:
* Follows the peer if not already followed
* Syncs refs of the peer, if it wasn't followed before
* Fetches refs into working copy

This functionality can be disabled via `--no-sync` and `--no-fetch`.
2024-01-08 17:28:01 +01:00
cloudhead 7169e97bd5
cli: Tell the user when canonical head is updated
This is useful especially in multi-delegate repos, since patches can be
merged without the head being updated.
2024-01-05 12:09:30 +01:00
cloudhead 50460842f4
cli-test: Expand env vars in commands 2024-01-05 11:03:24 +01:00
cloudhead 2d71093608
radicle: Small error message improvements 2024-01-04 14:23:53 +01:00
cloudhead 91684cc9ca
cli: Add `rad patch diff` command 2024-01-04 11:49:01 +01:00
Fintan Halpenny 6baefdb13f
cli: use `git` for anonymous fetch
Shell out to `git`, avoiding using `git2`. This is to avoid an
issue where somewhere within the fetch there is an attempt to lookup a
`rad/sigrefs` object, which says that the object is missing. It is
suspected that this is due to the object being localised in the same
packfile as other objects we are fetching. Since the `rad/sigrefs`
object is never needed nor used in the working copy, this will always
result in the object missing. This seems to only be an issue with
`libgit2`/`git2` and not `git` itself.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2024-01-02 15:04:49 +01:00
Sebastian Martinez 1bb400c278
httpd: Add edits field to revision json 2023-12-22 15:17:25 +01:00
Sebastian Martinez ee3dcfacf0
httpd: Allow edit revision embeds 2023-12-22 15:09:20 +01:00
Sebastian Martinez 5331cf3bcf
radicle: Move `DataUri` to `common::DataUri` 2023-12-22 15:09:20 +01:00
Sebastian Martinez a3a8060e5c
radicle: Add `Vec<Edit>` to revision description
This allows us to have embeds on a patch revision, and eventually also
show the edits that have been made to a revision description.
2023-12-22 15:08:35 +01:00
Fintan Halpenny 2dea54c6ee
cob: mandatory `Oid` in `CodeLocation`
To make `CodeLocation` reusable across different components, require
that a commit's `Oid` is part of the struct.

An `Oid` is necessary since the `path` can refer to a path at any
point in time throughout the history of the repository. By pinning the
commit that is being referred to, the path will refer to the path
within the associated tree [of the commit].

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-22 15:00:42 +01:00
Sebastian Martinez 1f36879aa2
cob: improve Comment generics
`Infallible` is a more appropriate type in comparsion to `()` for
comments that do not require a `CodeLocation`.

That is because if the type for `location` is `Option<()>`, this means
that `Some(())` can be constructed, however, we never intend to
construct that variant. On the other hand, `Infallible` can never be
constructed, and so, only `None` can be used.

This requires updates to some functions to use the generic `T` and
also handle the different kinds of `Comment`s -- `Infallible` and
`CodeLocation`.

X-Co-Authored-By: Fintan Halpenny <fintan.halpenny@gmail.com>
2023-12-22 14:09:02 +01:00
Fintan Halpenny 47a440a384
radicle: expose shared logger
To be able to use log setup across the different components of
heartwood, expose a `logger` module from `radicle`. This also includes
a `test` logger.

These modules are guarded behind a `logger` feature flag, so that
other crates are not bringing any uneeded dependencies.

This allows logging to be setup in `radicle-cli-test` so that when a
test fails, the log output is present for debugging purposes.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-20 15:11:10 +01:00
Fintan Halpenny a6ba1b0103
node: inspect outdated sigrefs test
This test is flakey and requires further investigation. Some extra
assertions are added to ensure that the expected behaviour is
happening upon fetching. It is unknown if the fetch queue is behaving
properly, and it's suspected that some existing fetch results might be
used for queued fetches.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-19 15:02:14 +01:00
cloudhead 13db55014b
cli: Show URL on `rad init` 2023-12-19 14:15:30 +01:00
cloudhead 6eb2772235
node: Improve flexibility of test environment
Allows us to pass full configs (profile or node) when creating test
nodes or profiles.
2023-12-19 14:15:30 +01:00
cloudhead 59f506dbb5
radicle: Fix bug in node.db query 2023-12-19 13:26:17 +01:00
cloudhead 6f7c2dca91
node: Introduce connection penalty system
Previously, we might ban a node if we're having trouble connecting to
it, but this could be due to internet connectivity issues.

We introduce a penalty score for nodes, that increases if there are
connection issues or the node misbehaves, and decreases on successful
connections.

When connected to new nodes, we sort them so that low-penalty nodes are
attempted first.
2023-12-15 16:00:09 +01:00
cloudhead d9a84de5ee
radicle: Simple database migration system
We create a system for migrating the our databases, as well as a first
migration, for adding a `penalty` field to the `nodes` table, which we
are going to need soon, as an example.
2023-12-15 12:04:54 +01:00
cloudhead cc380532c6
cli: Improve messaging when config doesn't load
Instead of telling users to setup their profile, point out the actual
problem with the config.
2023-12-13 12:30:11 +01:00