Commit Graph

635 Commits

Author SHA1 Message Date
Sebastian Martinez ee3dcfacf0
httpd: Allow edit revision embeds 2023-12-22 15:09:20 +01:00
Sebastian Martinez 5331cf3bcf
radicle: Move `DataUri` to `common::DataUri` 2023-12-22 15:09:20 +01:00
Sebastian Martinez a3a8060e5c
radicle: Add `Vec<Edit>` to revision description
This allows us to have embeds on a patch revision, and eventually also
show the edits that have been made to a revision description.
2023-12-22 15:08:35 +01:00
Fintan Halpenny 2dea54c6ee
cob: mandatory `Oid` in `CodeLocation`
To make `CodeLocation` reusable across different components, require
that a commit's `Oid` is part of the struct.

An `Oid` is necessary since the `path` can refer to a path at any
point in time throughout the history of the repository. By pinning the
commit that is being referred to, the path will refer to the path
within the associated tree [of the commit].

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-22 15:00:42 +01:00
Sebastian Martinez 1f36879aa2
cob: improve Comment generics
`Infallible` is a more appropriate type in comparsion to `()` for
comments that do not require a `CodeLocation`.

That is because if the type for `location` is `Option<()>`, this means
that `Some(())` can be constructed, however, we never intend to
construct that variant. On the other hand, `Infallible` can never be
constructed, and so, only `None` can be used.

This requires updates to some functions to use the generic `T` and
also handle the different kinds of `Comment`s -- `Infallible` and
`CodeLocation`.

X-Co-Authored-By: Fintan Halpenny <fintan.halpenny@gmail.com>
2023-12-22 14:09:02 +01:00
Fintan Halpenny 47a440a384
radicle: expose shared logger
To be able to use log setup across the different components of
heartwood, expose a `logger` module from `radicle`. This also includes
a `test` logger.

These modules are guarded behind a `logger` feature flag, so that
other crates are not bringing any uneeded dependencies.

This allows logging to be setup in `radicle-cli-test` so that when a
test fails, the log output is present for debugging purposes.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-20 15:11:10 +01:00
Fintan Halpenny a6ba1b0103
node: inspect outdated sigrefs test
This test is flakey and requires further investigation. Some extra
assertions are added to ensure that the expected behaviour is
happening upon fetching. It is unknown if the fetch queue is behaving
properly, and it's suspected that some existing fetch results might be
used for queued fetches.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-19 15:02:14 +01:00
cloudhead 13db55014b
cli: Show URL on `rad init` 2023-12-19 14:15:30 +01:00
cloudhead 6eb2772235
node: Improve flexibility of test environment
Allows us to pass full configs (profile or node) when creating test
nodes or profiles.
2023-12-19 14:15:30 +01:00
cloudhead 59f506dbb5
radicle: Fix bug in node.db query 2023-12-19 13:26:17 +01:00
cloudhead 6f7c2dca91
node: Introduce connection penalty system
Previously, we might ban a node if we're having trouble connecting to
it, but this could be due to internet connectivity issues.

We introduce a penalty score for nodes, that increases if there are
connection issues or the node misbehaves, and decreases on successful
connections.

When connected to new nodes, we sort them so that low-penalty nodes are
attempted first.
2023-12-15 16:00:09 +01:00
cloudhead d9a84de5ee
radicle: Simple database migration system
We create a system for migrating the our databases, as well as a first
migration, for adding a `penalty` field to the `nodes` table, which we
are going to need soon, as an example.
2023-12-15 12:04:54 +01:00
cloudhead cc380532c6
cli: Improve messaging when config doesn't load
Instead of telling users to setup their profile, point out the actual
problem with the config.
2023-12-13 12:30:11 +01:00
cloudhead ecb59d687b
cob: Return `RevisionId` instead of `Revision`
Previously we panicked if a revision wasn't found after an update. But
it can happen that the update goes through yet a concurrent update
prevents the revision from being valid.

To partially deal with this, we don't try to lookup the revision after
an update, we simply return the ID. This means a subsequent lookup can
fail, and the error can be handled there.
2023-12-13 12:14:06 +01:00
cloudhead 6ca7da2768
radicle: Rename many more methods using new naming 2023-12-06 10:02:00 +01:00
cloudhead 1b026ae0ee
node: Rename node `tracking` module 2023-12-06 10:00:45 +01:00
cloudhead e65595c4c7
node: Rename commands with new "seed" nomenclature 2023-12-06 09:56:58 +01:00
cloudhead f453090e27
cli: Consolidate policy/scope variants
* Policy::Track -> Policy::Allow
* Scope::Trusted -> Scope::Followed
2023-12-06 09:56:58 +01:00
Fintan Halpenny dce5316cbb
radicle: rename Issue::assigned -> Issue::assignees
Align the naming with the field itself and Patch::assignees.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-06 09:48:02 +01:00
Sebastian Martinez f8005c74c6
httpd: Add comment edits to serialized json
This allows the display of any edits done to comments,
replies and issue descriptions.

This commit doesn't remove the body property
which is the latest comment body but adds a edits property.

Also adds an author property to the `Edit` struct,
which allows us to see the author of an edit action.
2023-12-05 16:01:50 +01:00
Fintan Halpenny 924a2624a4
git: update git2-0.18.1
Update to git2-0.18.1, which also requires updating radicle-git-ext
and radicle-surf.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-12-04 11:11:36 +00:00
cloudhead af095d53ca
radicle: Add foreign-key constraint for routing 2023-11-30 13:00:59 +01:00
cloudhead 6b04eff34c
radicle: Consolidate databases
Instead of having a routing database, and an addresses database,
consolidate that in a single database called `node.db`.

We also simplify database access, by providing a single type for opening
the database, and use traits to access the various functions.

We create a new `seed::Store` that has seed-related methods split out of
the `address::Store`.

Note that tracking is still in its own database currently, as it isn't
ephemeral state. This makes it easy to delete the new `node.db` without
risk of losing any important information.

To avoid renaming all database methods, we create a `Stores` wrapper
that allows us to access routing, addresses, seeds etc. without type
ambiguity.

This is a breaking change and requires `addresses.db` and `routing.db`
to be deleted.
2023-11-30 13:00:59 +01:00
Fintan Halpenny 72b00e3d41
cli: sort-by flag for rad sync
Allow the caller of `rad sync` to `--sort-by` the NID, alias, or
status of the seeds.

This prevents random ordering, due to timestamps, in CLI testing.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-30 12:46:18 +01:00
cloudhead 538ca1e44d
cob: Don't lose state when lifecycling
It was previously possible to switch a merged patch to "open", losing
the merge state.
2023-11-30 10:53:09 +01:00
Fintan Halpenny 5b1cdbaa4a
radicle: remove repo iff there's no local rad/sigrefs
If the local peer does not have any `rad/sigrefs` then there is no
risk of any data corruption. In this case the repository can be
removed entirely.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-30 10:48:14 +01:00
Fintan Halpenny 5f96429005
radicle: remove `WriteStorage::remove`
The `remove` method is no longer being used, with `clean` taking its
place.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-30 10:48:14 +01:00
cloudhead ad7a3addb6
cli: Add `rad config` command
Displays the current configuration.

Also move to our tree-sitter highlighter for pretty-printing JSON and
remove the `json-color` dependency.
2023-11-30 10:46:32 +01:00
cloudhead 59a08f21a4
radicle: Explorer and preferred seed configuration
Allows users to configure the default node explorer to use.
Allows users to configure their preferred seeds.
2023-11-30 10:46:32 +01:00
Fintan Halpenny d23120ca8b
radicle: add clean method
Add a `clean` method to remove the remotes of a repository that are
not the local peer nor the delegates.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-24 10:48:05 +00:00
cloudhead 53013f306f
cli: Improve `rad ls`
Distinguish between repos that are simply tracked, vs. repos that are
cloned (and thus forked). By default, we don't show repos that don't
have a local fork.
2023-11-23 13:51:47 +01:00
cloudhead 2917198d1c
radicle: Fix `config.json` issue
This allows the config to load without a `cli` key.
2023-11-23 13:51:47 +01:00
cloudhead 8e76ce03d5
cli: Hints system
Enable hints by default, but allow users to turn them off via
configuration.

Add a hint when pushing while offline.
2023-11-23 11:07:07 +01:00
cloudhead dfe0cb4342
cli: Polish the `rad sync status` output 2023-11-22 17:40:51 +01:00
Fintan Halpenny 81dc849bce
cli: add name option to rad patch checkout
Add the `--name` option to `rad patch checkout`. This allows the user
to name the branch they are checking out, rather than use the
`patch/<short-id>` format.

This change includes returning early for setting the upstream branch,
if the upstream already existed.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-21 10:19:25 +01:00
cloudhead 358324883c
node: Announce offline ref updates
Currently, if refs are updated while the node is stopped, these refs are
not immediately announced when the node starts, due to the node being
unaware of the updates.

To remedy this, on startup, we compare the refs we have in storage with
the refs we announced to the network. Since only our own refs could have
changed while the node was stopped, this change only applies to owned
refs. If we detect that storage has changed while the node was stopped,
we create ref announcements for the updated repos and store them in our
gossip database. These announcements will be included with other
historical gossip announcements when nodes connect to us.

To keep track of what was announced, we store our local node's sync
status in the database. This is also useful when using the `rad sync status`
command, as it tells us what oid our local node is at, and whether it
is aware of our local changes yet.

In addition to this, we make a small fix: previously, inventory and ref
announcements originating from our local node were not stored in the
announcement database. We ensure that they are via the new `announce`
method.
2023-11-20 17:54:36 +01:00
cloudhead b140d8580f
cli: Don't announce if already in sync
In this change, we check whether we are in sync with a node before
announcing refs to it.
2023-11-17 17:28:51 +01:00
cloudhead d4ec2176bb
Update Rust toolchain to 1.74 2023-11-17 16:58:24 +01:00
cloudhead d3b0483f42
cli: Implement `rad sync status` command
This command shows the sync status of a repository by consulting the
database via the node.

To make it work, we call into the address store to update the
`repo-sync-status` table when receiving ref announcements that are
relevant to us.
2023-11-17 16:56:19 +01:00
cloudhead ca557909c2
radicle: Fix some serialization issues
For some inexplicable reason, deserializing doesn't work unless we use
our custom de-serializer for `LocalTime`.
2023-11-17 16:54:27 +01:00
cloudhead aa13f57e7d
radicle: Fix issues with the address store SQL 2023-11-17 16:54:27 +01:00
cloudhead bf64fde8a1
cli: Improve timestamp format function 2023-11-17 16:54:27 +01:00
Fintan Halpenny 5cb4ad4622
node: `info` announcements
An informational message for the protocol is useful for sending
messages to a node that is not required to be relayed.

This message is introduced by starting with an ACK message for a set
of references being synced. This fixes an issue for announcing to a
node that is already synced, since it can now ACK that it received the
message and replies that it is synced.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-17 16:29:29 +01:00
Fintan Halpenny 85d452b222
radicle: return RefsAt for announce_refs
It can be useful to return the `RefsAt` that was announced for this
node, for example, to compare it to node events.

Change the `Handle` method to return `RefsAt` so that when it is
called it returns the local node's `RefsAt`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-17 16:29:02 +01:00
Fintan Halpenny ac849f1fd5
radicle: add ReadRepository::contains
It can be useful to be able to check if a repository contains a given
commit. Add a `contains` method to the `ReadRepository` trait to
handle this.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-17 16:09:42 +01:00
cloudhead cdcc7cfaf9
node: Emit events for gossip messages 2023-11-17 14:56:42 +01:00
cloudhead 39f65e1b29
Update `netservices` and `cyphernet` dependencies 2023-11-17 12:23:47 +01:00
Fintan Halpenny e71f07e592
radicle: temporary repository helper
It's possible that a temporary directory can be created on a different
mount, and when moved during a clone, creates an I/O error.

To prevent this, add a method to `Storage` which creates a temporary
directory and `Repository` with the path in the form
`<rid>.lock`.

This `.lock` path name can easily be skipped when listing
repositories. Further tooling can be added for detecting leftover
`.lock` directories if, for some reason, they were not cleaned up by
the `tempfile` crate.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-16 12:01:10 +00:00
Fintan Halpenny fbc5c5a027
radicle: Check signed refs are unchanged
Instead of creating the tree to check if `rad/sigrefs` has changed, it
is possible to check that the signature of the signed refs being
updated is the same as the old one. If so, then there are no
changes. Otherwise, there must be changes and the update can continue
to create the necessary Git objects.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-16 11:05:41 +01:00
cloudhead ea4104dbd7
node: Only process messages from known nodes
Previously, we would process any valid inventory or refs announcement.
This could cause problems for keeping sync statuses, as we could get
a refs announcement from a node that isn't in our database.

In practice, there is no good reason why we should receive a refs or
inventory announcement before the announcer's node announcement, unless
nodes are not following the protocol.

Implementing this change meant that many of the tests needed updating,
as it wasn't assumed that a node announcement was needed before
receiving inventories etc. from a peer.

We also had to implement a basic form of node address banning to make
the tests pass after this change.
2023-11-13 16:15:57 +01:00
cloudhead 0ae3c00cd7
Use cascading deletes for removal of node rows
Instead of manually removing all rows in the address table, use
cascading deletes.
2023-11-13 16:01:18 +01:00
cloudhead 269d21deca
node: Rewrite `seeds` method to use sync-status
We update the `seeds` method to use not only the routing table, but also
the sync status information for our owned refs.
2023-11-13 12:06:37 +01:00
cloudhead 9f45405c64
Add table and types for tracking sync status
We add the necessary database table to keep track of which node has
synced which of our repos. This will be useful for giving the user
information on the sync status of every local repository.
2023-11-13 12:03:24 +01:00
cloudhead b675185538
node: Improve control socket API
We improve the API so that error messages can be sent back and are
always expected as potential responses to commands. This means that
`CommandResult` is always the type that is sent for commands.

This includes some breaking changes, for example the default "success"
message is simply `{}` instead of `{"status":"ok"}`.
2023-11-13 11:02:45 +01:00
cloudhead d88cce484a
radicle: Simplify canonical head quorum
Instead of trying to handle cases where delegate branches diverge, we
simply throw an error in those cases, which prevents the canonical head
from updating.

This simplifies the code a lot, and forces the user to take action when
such an event occures.
2023-11-10 13:42:16 +01:00
Fintan Halpenny d27cd0db0c
node: minimise refs announcement to key and Oid
The RefsAnnouncement previous announced its complete set of
`rad/sigrefs`. This was useful in that verification can occur on the
receiveing side immediately by verifying the refs and proceeding with
the fetch -- note that it did nothing with the ref information it
received.

This can be minimised to only advertising the namespace, i.e. the
NID, and the new SHA of the `rad/sigrefs`. This is represented by a
new message type, `RefsAt`, which contains the NID and the Oid.

For checking if an announcement is stale or fresh, the receiving
side can check the existence of the SHA in its Git repository.

The tests are appropriately modified to use NIDs and Oids.

To prepare for a follow-up change, the announcement data is also
threaded through for the fetch protocol to focus fetching only the
advertised set of Oids.

N.B.: because the refs aren't communicated, the verification happens
when fetching since the data needs to be fetched before
verifying. This is a trade-off with minimising the message payload for
verifying earlier.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-09 17:47:09 +01:00
Fintan Halpenny 61b06553b4
radicle: use git::raw::Error for reference_oid
The minimal error for the `ReadRepository::reference_oid` method is
`git::raw::Error`.

Change the error type from `git::ext::Error` to `git::raw::Error`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-09 17:47:07 +01:00
Fintan Halpenny 82a53e3a53
radicle: UserInfo for Repository::create
The fetch V2 improvements introduced a `UserInfo` type that is used to
set the Git config's name and email when cloning a repository. This
`UserInfo` can also be used when creating a new `Repository`.

The `Storage` type now takes a `UserInfo` so that it can be passed to
`Repository::create`, for creating a new repository.

For testing, this requires adding a new `fixtures::user` so that
testing `Storage` instances can be easily constructed.

It also means that the `radicle-cli` tests all changed with new SHAs
since the name and email are used as part of the hashing input.

N.b. the order of the timelines while testing the identity cob
changed. It's not clear why that is.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-09 16:43:34 +01:00
cloudhead 9b7c48ab40
cli: Improve `rad diff`
* Properly show the user when a file was moved or copied.
* Don't require a radicle project in the current repo.
2023-11-09 13:04:40 +01:00
cloudhead f0754330c9
Update `sqlite` dependency to 0.32.0 2023-11-08 15:08:01 +01:00
cloudhead f9df360171
node: Persist historical gossip messages
Instead of storing received gossip messages in an in-memory `BTreeMap`,
we persist them to the database (`addresses.db`). There are two reasons
for this:

1. Node restarts do not wipe the state, meaning that it is much less
   likely that some gossip message will get lost before reaching all
   nodes.
2. This will allow us in the next commit to know how far behind we are
   after a restart, to be able to request the correct time range of
   gossip messages that we missed while offline.

Note that we now share the addresses.db file between two connections and
stores, which we don't do anywhere else. Having multiple connections to
the same database is fine, but the way we're having to do it is a bit
awkward. In a future patch, we could pull out the database schema from the
`address` module and rename the database to make things clearer.
2023-11-08 13:15:56 +01:00
Sebastian Martinez 193ed2f675
radicle: Add missing embeds in `RevisionComment` 2023-11-06 14:51:33 +01:00
Fintan Halpenny 981172aa4f
radicle-fetch
The `radicle-fetch` crate introduces a method for fetching from a
server using Git protocol V2, which provides the implementation
control over performing a sequence of fetches and validation over
domain data.

The protocol is built on top of the suite of Gitoxide crates[[0]]. It
builds wrappers on top of Gitoxide's refdb, odb, and transport layers
-- as well as taking the ls-refs and fetch code straight from the code
base to simplify for the needs of this crate.

[0]: https://github.com/Byron/gitoxide

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny 87187d47e5
radicle: add remove to WriteStorage
Add the ability to remove a repository, given its RID.

This is useful in the case of attempting to clone a repository, and if
that process fails -- for example, if the repository is private --
then it's necessary to clean up the initially created repository.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny f1f32d8574
radicle: return verified doc
The `identity_doc_at` and `identity_doc` methods on `ReadRepository`
returned the unverified identity document. Every usage of these
methods would always verify the document after.

Simplify the code by calling verified in the methods and directly return the
verified identity document.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny adf3130431
radicle: add Special type
Add an enum to represent the special references, `rad/id` and
`rad/sigrefs`. These can be used for cases where it's more manageable
to work with the enum type over checking for reference string equality.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny 75b52e28b2
radicle: separate repository validation
Make the validation logic customisable by extracting it out into a
separate trait that can be defined by other crates.

The Repository validation logic stays the same.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny b503aa6be4
radicle: validations
Instead of returning the unsigned references, the validation of a
repository's remotes returns the set of validation errors. The caller
is then expected to handle these validation errors appropriately, for
example, by logging them and re-raising an error.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny dfdf2e20cc
radicle: add UserInfo to git
Add a type to capture user information that can be used for Git
configuration and signatures.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny ef6ae0fa84
radicle: move service tracking config to radicle::node
The `service::tracking` module has nothing specific to the
radicle-node code. This means that it can be moved to radicle under
the `node::tracking` module.

This makes it more resuable in other contexts, for example, for
fetching logic.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny c06b225877
radicle: add sigrefs ref name constructor
Similar to the `id` function, add a `sigrefs` function to construct
the reference name for a given remote's `rad/sigrefs`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
Fintan Halpenny 030419932e
radicle: Add SignedRefsAt type for signed refs
Introduce a SignedRefsAt type that combines a SignedRefs along with the Oid
it was loaded at.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-11-06 12:06:13 +01:00
cloudhead d150938b38
cli: Update tracking without a running node
This change allows users to track/untrack peers and repos without a
running node. This fixes a bug when running `rad init` without the node
running, where the repo would be tracked with a different scope than
what was specified on the CLI.
2023-11-03 17:17:07 +01:00
cloudhead a70664368e
radicle: Move some functions to `Home` type
Certain functions in `Profile` did not need to be in there and could be
moved in the more basic `Home` type.
2023-11-03 17:08:18 +01:00
cloudhead 84a95848b0
cli: Add pager to `rad diff`
Invokes a pager when the output doesn't fit on the screen.
2023-11-03 14:20:16 +01:00
cloudhead d8f8d29605
radicle: Fix timestamp issue with feature flag
The `stable-commit-ids` feature flag was being included in non-dev,
non-test builds because of how cargo works (or doesn't work).

For now, we ignore the tests that this affects, and they can be run with
the `RAD_COMMIT_TIME=1514817556` environment.
2023-11-01 17:24:47 +01:00
Michalis Zampetakis 0b6ede69e7
node: Add `listen` option to node config 2023-11-01 12:59:29 +01:00
Fintan Halpenny c82344496b
radicle: Improve cwd to search parents
If a `rad` command that requires an RID is executed within a
subdirectory of a Radicle project, then it will fail saying:

    ✗ Error: Current directory is not a radicle project

Improve this flow by searching parent directories and/or using the
GIT_DIR environment variable.

This is safe as long as the operations are read-only when calling
`cwd`. Currently they are:
- `Repository::open_ext` -- will not create a repository, only open
  one.
- `Repository::find_remote` -- this will only read the remote from the
  config and get the URL to parse the RID.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-10-30 15:29:37 +01:00
Fintan Halpenny 199fa07a79
radicle: Apply clippy suggestions
`cargo clippy --all --all-targets` is warning that there are multiple
redundant clones.

Apply the suggestions by removing the calls to `clone`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-10-30 14:51:21 +01:00
Fintan Halpenny 2584ecdb0a
radicle: Pin timestamp for testing
When testing the CLI, "since"-calculations will result in different
values over time:

    now
    1 month ago
    2 months ago
    10 months ago
    1 year ago
    10 years ago

Initially, the solution was to use the wildcard match from `snapbox`,
`[..]`, to match on these values. However, since these values change
in length it means that they will affect the width of the tables
produced by the CLI.

To ensure the width is stable across runs over time it's possible to
use a time that is the same as the commit time used at testing.

To do this the `Timestamp` type is now a newtype which has a
constructor `now` which will use the `RAD_COMMIT_TIME` env variable
for construction, if it's available and in debug mode. This ensure
that all "since"-calculations will result in `now`.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-10-30 14:48:38 +01:00
cloudhead 1d167581f2
cob: Introduce `xyz.radicle.id` COB
Introduces a new COB type to store repository identity documents.

The reason for this change is to:

1. Simplify the code, as the identity document logic resembled a COB,
   yet it had custom logic. This allows existing COB code to be used for
   identities.
2. Make identity document update logic more flexible, since COB actions
   can be added in the future.
3. Re-purpose existing tools around COBs to work on identities, eg. `rad
   cob`.
4. Unify the concept of an identity change proposal, with regular identity
   changes. This means we can remove the `id.proposal` COB in favor of
   using the `id` COB itself.

Notes
-----

* Each repository has one Identity COB.
* The `Proposal` COB has been repurposed into the `Identity` COB.
* Identity documents are stored as *embeds* inside the Identity COB
  actions.
* The action that contains new document versions is called `revision`,
  just like the Patches COB.
* The namespaced `rad/id` ref is a symbolic reference to that Identity
  COB.
* The canonical `rad/id` ref is a direct reference to the commit in the
  Identity COB that contains the latest *accepted* revision of the
  document.
* All commands for managing identities have been folded into `rad id`.
  Hence `rad delegate` and `rad edit` are removed.
* The concept of "rebasing" an identity document is gone.
* The `rad id` output has been updated to match the style of other
  commands.
* When a revision has enough signatures, it is automatically adopted as
  the current identity, there is no longer the need for a "commit"
  action.
* When an identity revision is proposed, and the current identity has a
  threshold of `1`, that identity is automatically accepted due to the
  above point.
* The idea of "verifying a peer's identity branch" no longer applies, as
  COBs cannot be verified one history at a time.
* Since the root commit of the Identity COB does not have a "Resource"
  to point to (it would have to point to itself, which is impossible),
  we've made the resource id optional for COBs.
2023-10-12 17:32:33 +02:00
cloudhead c99d9a6aed
radicle: Make mock transport thread-safe
The mock transport was not thread-safe when multiple threads used the
same node ids. This is fixed by including the thread id as part of the
storage key.
2023-10-11 14:53:59 +02:00
cloudhead 5709f5d886
cob: Improve stability of test setup
We employ various tricks to ensure that tests are reproducible.
In this case, having stable commit ids allows us to test specific
traversal orders of a COB history in cases of concurrent updates.
2023-10-11 14:53:59 +02:00
Sebastian Martinez 78e6833672
httpd: Add missing COB actions
Handles missing issue and patch actions. Adds embeds to patch comments.

Signed-off-by: Sebastian Martinez <me@sebastinez.dev>
2023-10-11 13:36:48 +02:00
Alexis Sellier c0271c303f
Prevent divergences in canonical head
On push, we check whether the resulting state would cause a
divergence/fork in the canonical head, and if so, prevent the push from
happening.

This is to avoid situations where delegates have to then rollback their
heads.

Note that this doesn't prevent forks from happening altogether, as they
could happen asychronously, but it mitigates the problem.
2023-09-27 12:25:14 +02:00
cloudhead 883b548c61
Improve computation of canonical head
We employ a better algorithm for determining which commit should be
canonical when there are multiple possible options.

The algorithm still doesn't work in certain cases when there are
multiple merge commits as candidates, with shared parents.

We leave some (ignored) failing tests to be resolved in the future.
2023-09-27 12:25:14 +02:00
cloudhead 1376ab63c0
cli: Allow checking out a specific revision
Allow `rad patch checkout` to specify a revision to checkout that may
not be the patch's latest.
2023-09-21 16:18:24 +02:00
cloudhead c94bd50666
cob: Better distinguish patch revisions
The `Patch` API and semantics are changed a little bit:

* It's now valid again to propose a revision without being the patch
  author.
* To get revisions by the patch author, `Patch::updates` is used.
* `Patch::latest` now only looks at revisions by the patch author.
* Added `Patch::revisions_by`, `Patch::authors` and `Patch::latest_by`.
* The merge behavior on git-push is improved to favor revisions that
  merge more recent commits. We also allow for merging the non-latest
  revision.
2023-09-21 13:51:55 +02:00
cloudhead 5078396028
cli: Move commenting to patch/issue commands
Instead of having `rad comment`, move the command to the respect COBs,
ie. `rad issue comment` and `rad patch comment`.
2023-09-19 14:33:40 +02:00
cloudhead 8cae60371c
cli: Simplified issue opening
We remove the YAML front-matter, since that was annoying to use.
Issues are now edited like patches and commit messages.

Labeling and assigning are done via dedicated commands.
2023-09-19 14:22:45 +02:00
Alexis Sellier 1abce45d5d
node: Slightly increase rate limit defaults 2023-09-18 11:14:19 +02:00
Alexis Sellier a7cd7b25f5
node: Allow rate-limitting to be configurable 2023-09-18 11:14:19 +02:00
Alexis Sellier af724eeb0d
cli: Add `node config` command
Shows the current (loaded) node configuration.
2023-09-18 11:14:19 +02:00
cloudhead 457b82f723
cob: Evaluate COB state eagerly
Instead of returning histories from `radicle-cob`, we return an
evaluated object which we build during initial graph traversal.

We do this so that branches can be correctly pruned when operations are
invalid at the application level. This way, new operations are not
building on top of invalid ones contained in the history.

To achieve this, we introduce a new `Evaluate` trait that is implemented
by all COBs, and we make change graph evaluation fallible.

Doing this means that we get `apply` errors returned for free when an
invalid transaction is applied, and no longer need to check for action
validity in multiple places.

We also implement a new `Dag::prune` method to avoid having to copy
graph nodes during traversal.
2023-09-11 23:25:28 +02:00
cloudhead 65fa3cbb1b
radicle: Make `DraftStore` generic over the repo
This will allow using it inside of COBs.
2023-09-10 20:44:38 +02:00
cloudhead 6695074479
cob: Don't require full `Entry` for update
Instead of taking the full entry type, we only need the id.
2023-09-10 20:44:38 +02:00
cloudhead c903a958e7
cob: Implement action authorization
Add `authorization` functions to `Patch` and `Issue` COBs.

Co-authored-by: Arastoo Bozorgi <ab1502@mun.ca>
2023-09-09 18:31:52 +02:00
cloudhead 5c256c8bfb
cob: Remove support for legacy issues 2023-09-07 11:48:18 +02:00
cloudhead 5fc6d96076
cob: Move action handling into its own function
This change is in preparation for further changes where action handling
is called from multiple places.
2023-09-06 20:40:15 +02:00
Fintan Halpenny bf8cee2446
cob: Remove EntryId and RevisionId conversions
Clippy warns that EntryId::from and RevisionId::from are not
required.

Remove the unecessary calls in radicle-cob and radicle.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-09-06 20:07:36 +02:00
cloudhead 287a920d12
cli: Add `rad patch set` command
Sets the current branch's upstream to point to the given patch.
This allows for pushes to update the patch.

When opening a patch, if the current branch already has an upstream,
it isn't updated automatically. This is why this command exists.
2023-09-06 20:03:58 +02:00
Arastoo Bozorgi 5466232dd4
cob: Work on missing functionality in patches
* Add resolve/unresolve functionalities to threads' comments
* Enable reactions on a revision and code locations
2023-09-06 10:56:14 +02:00
cloudhead 54cc92cc83
cob: Add missing `verdict` field to `review.edit`
This is generally useful, and required when constructing a review
without knowing the verdict ahead of time.
2023-09-05 11:17:59 +02:00
cloudhead 152be65053
cob: Wrap patch ID types with new types
To prevent using eg. a `RevisionId` when a `ReviewId` is required,
use compiler types.
2023-09-05 11:17:59 +02:00
cloudhead 9de10ade5a
cli: Add `--inventory` mode to `rad sync`
This mode allows to announce our inventory. It's not normally needed,
but may be suggested by the `rad publish` command in certain cases.
2023-09-05 11:16:09 +02:00
cloudhead bfc3c7164f
tools: Add `rad-merge` tool
It happened that a small bug in the remote helper caused a patch not to
merge correctly on push. Hence, I implemented a `rad-merge` tool for
these cases where we need to fix broken state.
2023-09-01 14:33:23 +02:00
cloudhead 9cf20ddef9
Introduce `RepositoryInfo` type 2023-09-01 12:13:45 +02:00
cloudhead 09a284f09c
node: Make fetch timeout configurable
When a fetch is user-requested, a timeout can be supplied that is passed
down to the worker.

When a fetch is service-requested, a default timeout is used.
2023-09-01 12:13:45 +02:00
cloudhead 9a54c8da7b
cli: Update commands for private repos
Several commands are updated to better support private repos.
2023-09-01 12:13:45 +02:00
cloudhead 257b950d6f
cli: Add `--private` option to `rad init`
Enables private repository initialization.
2023-09-01 12:13:45 +02:00
cloudhead 27f39514d4
node: Implement private repos
Private repos are implemented by extending the identity document with a
`visibility` attribute, that can either be `"public"` (default) or
`"private"`.

In case of `private` visibility, only the delegates are allowed to view
the repo, as well as any DIDs added to the allow list.

To implement repo visibility, we simply block fetches from and
announcements to peers for whom the repo should remain invisible.

Private repos are also not announced in the `inventory` message, since
the full list of peers that *may* have the repo is retrievable from the
repo identity. This could cause errors if eg. a peer who is allowed to
view the repo doesn't actually have it. However this is an ok trade-off
for now to keep the complexity low. For repos to truly be private, it's
important that the RIDs don't leak either.

Finally, we modify `radicle-httpd` for now to only list public repos.
Eventually, we would want to change this depending on whether an allowed
peer is authenticated with the service or not.

---

It's also worth mentioning why this approach was taken, vs. end-to-end
encryption. The reasons are as follows:

1. Nodes that do not have access to a private repo will generally not want to
   replicate encrypted data that they cannot examine or use.
2. The chosen solution is trivial, while encrypting git objects isn't.
3. Performance of the chosen solution is much better, there is no
   overhead.
4. Privacy of the chosen solution is better: RIDs are never leaked, and
   neither is the existence of a private repo, nor who has access to it.

There is one downside: Paying for storage of private repos is no better
in terms of privacy than what GitHub offers. Hosting providers will have
access to your private repos, if this solution is used.
2023-09-01 12:13:45 +02:00
cloudhead d8b5c1ad1c
cob: Remove support for legacy patches 2023-08-29 11:52:53 +02:00
cloudhead 205c15caa6
cob: Simplify `Entry` and `Change` types
Instead of having two types, we simply have one, called `Entry`, which
we rename from `Change`.

There was no real benefit to having two almost identical types, but lots
of added complexity. This patch simplifies the crate by removing one of
the types.
2023-08-24 12:57:30 +02:00
Sebastian Martinez 994f52d654
httpd: Add reactions and edits to patches
Signed-off-by: Sebastian Martinez <me@sebastinez.dev>
2023-08-22 14:42:09 +02:00
cloudhead 8cb8398be9
Introduce `cargo-deny`
Add `deny.toml` with a few exceptions, and update certain dependencies
to get rid of duplicates.

Running `cargo deny check` will yield some warnings still, but no
errors.
2023-08-22 14:33:00 +02:00
cloudhead 534503d002
cob: Add support for media embeds in COBs
This change allows for files (blobs) to be embedded into COB entries.
This allows for things like image attachments in issue comments for example.

For now, we only enable this in the `issue` COB.

The way it works is that relevant COB actions carry metadata about which
files are attached to them; and we store those files as blobs inside the
COB entry's *tree* object, under an `embeds/` folder.

The `Embed<T>` type is used for the above, and either carries actual
content, or carries a content-id. Retrieving the actual content is as
simple as asking the repository for a blob with that content-id.

As a possible future extension, MIME types could be stored alongside the
files in a "metadata" file. This could help clients display the content
appropriately.
2023-08-22 14:29:23 +02:00
cloudhead a922ecf3ce
cli: New `cob` command for debugging
Implement a `rad cob show` command for debugging COBs.
Prints out the full history without decoding the ops.
2023-08-18 10:54:27 +02:00
Sebastian Martinez 6abbd39926
httpd: Allow removing of reactions from issues
Signed-off-by: Sebastian Martinez <me@sebastinez.dev>
2023-08-14 16:41:04 +02:00
cloudhead 15b50ce6a5
radicle: Fix config deserialization
This is a fix for the breaking changes introduced with `PeerConfig`.
2023-08-14 10:31:35 +02:00
cloudhead 14c204a11f
radicle: Fix flaky test due to randomness 2023-08-13 21:20:27 +02:00
cloudhead d973fd42f2
cli: Auto-connect on sync
This adds functionality so that on `rad sync` and `rad clone`, we
connect to known seeds if necessary, before attempting to fetch.

In passing, some related changes were made:

* The `rad sync` command's arguments were reworked:
  `--replicas <count>` can be used to specify a replica count when
    fetching
* The `rad patch` command no longer has a `--fetch` option
* It's now possible to tell the node not to automatically connect to
  peers in the background
* `AddressBook` is being used underneath `Seeds` to allow for shuffled
  iteration
2023-08-11 11:45:27 +02:00
cloudhead efb8de2728
Rename our custom `HashMap` to `RandomMap`
This makes it easier to distinguish with the stdlib.

Also, we add an env var that can control the RNG seed, for testing
purposes.
2023-08-10 16:51:02 +02:00
cloudhead d13f47b2a4
Update toolchain to Rust 1.71 2023-08-09 18:49:13 +02:00
cloudhead fae518bca2
node: Improve `rad node connect` feedback
Only return once the connection is established or failed.

It's still not perfect, as several failure scenarios are not handled,
but it's an improvement.
2023-08-09 12:49:15 +02:00
cloudhead fb4a4e0079
node: Implement token-bucket rate limitter
Limit gossip messages and inbound connections.
For now, we don't limit git fetches.
2023-08-09 12:41:21 +02:00
cloudhead 260b1a428c
node: Move `address` field out of `State`
We move the session address to the top-level struct, since it's needed
in a bunch of places. This change is required for the rate-limiting
code that is coming next.
2023-08-09 12:41:21 +02:00
cloudhead dc7d67c1e2
node: Ignore non-routable addresses
We ignore addresses that are not globally routable when receiving them
in node announcement message, unless they are coming from a local peer.
2023-08-09 12:39:33 +02:00
cloudhead e888398519
node: Populate address book with bootstrap nodes
As a starting point, we populate the address book (if empty) with some
bootstrap nodes which are trusted.

To prevent these nodes being connected to during tests (including e2e
tests), we create a new constructor for `Config` that is used in tests.

Note that a `cfg(test)` check is not enough given that e2e tests don't
have that set.
2023-08-09 12:39:33 +02:00
cloudhead b456d3a401
Prepare crates for publishing to `crates.io` 2023-08-07 11:31:11 +02:00
Alexis Sellier 3bd5589b1a
cob: Stabilize formats
> I'm going to be making a set of breaking changes to COBs in order to
stabilize the data formats. This is hopefully a one-time change that
bundles various breaking changes.

All COBs have been reworked: issue, patch, id.

The changes included are:

* Revise the assign and tag actions to take a single list of
  assignees/tags to set, instead of an "add" and a "remove" list. This
  makes API usage simpler when editing issues, and simplifies the apply
  function
* Rename "tags" to "labels", and the tag action to label. This is
  because tag is confusing in the context of git, as it could mean a git
  tag. Using label removes that confusion.
* Use DIDs instead of PublicKeys for assignees -- this is more
  future-proof
* Modify the manifest file format in the COB tree. Mainly, remove the
  `history_type` key which is redundant, and use camelCase for keys
* Flatten the `Thread` actions into the parent action type
* Ensure that operations on redacted objects do not fail, since
  redactions could have happened concurrently
* Use a consistent naming scheme for actions, using `.` as separator
* Consolidate comment types and remove `CodeComment`, by adding an
  optional `location` field to `Comment`
* Add many placeholder actions that are not yet implemented

To preserve backwards compatibility, a `legacy` module is created with
the old `apply` function. When loading the manifest, we check whether it
is a legacy COB or a "stable" COB, and in the legacy case, use the
legacy code to materialize the state and then convert the object into
the stable type. Eventually, we'll delete the legacy code.
2023-08-07 11:05:14 +02:00
Alexis Sellier 885f4999da
remote-helper: Fix refs advertisement for patches
Some of the old patches had missing objects and that causes problems
during the ref advertisement, as the object is not found.

In this patch, we check that the object exists and also that the patch
is still open. Otherwise a remote tracking branch is created for every
single patch ref.
2023-08-06 17:29:21 +02:00
cloudhead 3898fece08
cli: `git pull` to update patches in working copy
We enable users to pull updates to a patch by calling `git pull`
on the patch branch.
2023-08-04 12:28:22 +02:00
Alexis Sellier b953e9f6b5
node: Use `camelCase` for control socket data
It's easier if we use camel-case everywhere, because some types may be
used on the web as well as on the control socket. This way, all
json-encoded data looks the same.
2023-08-01 15:48:44 +02:00
Alexis Sellier 8a3133e137
node: Include addresses in `Command::Seeds` result
This will allow the CLI to connect to seeds on the go.

We also do some refactoring of some of the code involved in fulfilling
the `Seeds` command.
2023-08-01 15:48:44 +02:00
Alexis Sellier 26e3c0fbff
Use numbers for `LocalTime` timestamps
Instead of using strings, use numbers.

*This is a breaking change and affects `config.json` parsing.*

To upgrade, simply unquote any number in your `config.json`.
2023-08-01 15:47:02 +02:00
Alexis Sellier 85b092872d
node: Support DNS address types at the wire level 2023-08-01 14:55:20 +02:00
Sebastian Martinez 7368e38942
radicle: Disable canonicalize_home tests on macos
We were doing a workaround stripping prefixes, but I think it makes more
sense to skip the test all together.

Signed-off-by: Sebastian Martinez <me@sebastinez.dev>
2023-07-26 13:23:15 +02:00
Alexis Sellier 41bff1b3bb
node: Make user connections persistent
While the node is running, connections made via the `Connect` command
should be retried.
2023-07-26 12:52:11 +02:00
Alexis Sellier 840fc496b8
node: Simplify control socket command parsing
Instead of using a command name plus arguments, we use a new `Command`
enum that is similar to the one in `radicle_node::service`. This way,
we're able to have more complex commands and parsing is simplified.
2023-07-26 12:35:55 +02:00
Alexis Sellier f639192dc6
cob: Remove CRDTs from COB state
It turns out that the CRDT formed by the union of Git DAGs
is enough to guarantee everything we need for COBs.

This changes the following things:
* COB operations no longer need to be commutative
* COB histories are traversed in the same deterministic order on all
  replicas
* It's now possible to implement RSMs on top of COBs, eg. scripting
* Lamport clocks have been removed
* `radicle-crdt` is no longer a dependency of `radicle`
* COBs are no longer instances of `Semilattice`
* The `Ops` type was removed in favor of having `Op` contain multiple
  actions
2023-07-25 15:29:27 +02:00
Vincenzo Palazzo a41f67d9ef
radicle: Import the new SQLite library with bundled binary
This commit addresses a workaround previously implemented to
include the SQLite library bundled inside the binary.

The upstream have successfully merged the patch that implements this
solution [1], allowing us to safely incorporate it
into our codebase.

[1] Link to the merged patch: https://github.com/stainless-steel/sqlite/pull/71/files

Fixes: 431a389944
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-07-25 12:30:03 +02:00
Alexis Sellier 3119294367
Add `--version` flag to binaries
Help users confirm all their radicle binary versions by providing the
'--version' flag.

Move the print_version function to the radicle crate for use by the node
and remote-rad binaries, and remove tests to confirm the output format
as it is deemed redundant.

Co-Developed-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-07-25 11:36:50 +02:00
Alexis Sellier f92308cf25
cli: Make key encryption optional
By leaving the passphrase prompt blank, users can create unencrypted
keys.
2023-07-20 10:40:15 +02:00
Alexis Sellier 4634913b66
radicle: Split out `SignRepository` trait
This new trait is used when only signing refs is needed, and not general
write access to a repository.

We also implement `ReadRepository` and `SignRepository` for
`DraftStore`.
2023-07-14 16:58:23 +02:00
Alexis Sellier ff7e9ded17
radicle: Introduce `cob::DraftStore`
This storage backend for COBs stores changes in a `draft/cobs/*` namespace,
which allows for some of the features needed for code review. For
example, users can draft comments and later decide to publish
them.
2023-07-14 16:57:55 +02:00
Alexis Sellier c9162121b0
cob: Parametrize `cob::Store` with repository type
The additional flexibility in choosing a backend for the COB store will
allow things like "draft" COBs that are stored in a different location,
without changing much of the logic.
2023-07-14 16:57:55 +02:00
Alexis Sellier 45ccbde226
Update `radicle-surf` and `git-ref-format` 2023-07-14 16:57:55 +02:00
Fintan Halpenny dfa7ba1df5
radicle: type safe tracking config
The tracking database can be opened in read-only or read-write
mode. Knowing which is being used is significant in concurrent
scenarios, e.g. it's safer to have a read-only handle on multiple
threads.

Provide static type information by signifying what kind of database
handle is being worked with, read or read-write.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2023-07-14 16:52:42 +02:00
Alexis Sellier 6b6f087c14
Use transaction to insert inventories
Instead of inserting inventory entries one by one, we use a transaction,
which is much faster.
2023-07-14 15:46:46 +02:00
Alexis Sellier b27313bc5c
Don't fail if we can't open the database
We make sure that even if there is no database file, we're still able
to return `None` for the alias lookup.
2023-07-12 11:37:20 +02:00