Previously, we would always ask nodes for messages from one hour ago.
This meant that we could miss messages if our node was offline for more
than an hour. This change uses a variable backlog by checking the
timestamp of the last gossip message we received.
Instead of storing received gossip messages in an in-memory `BTreeMap`,
we persist them to the database (`addresses.db`). There are two reasons
for this:
1. Node restarts do not wipe the state, meaning that it is much less
likely that some gossip message will get lost before reaching all
nodes.
2. This will allow us in the next commit to know how far behind we are
after a restart, to be able to request the correct time range of
gossip messages that we missed while offline.
Note that we now share the addresses.db file between two connections and
stores, which we don't do anywhere else. Having multiple connections to
the same database is fine, but the way we're having to do it is a bit
awkward. In a future patch, we could pull out the database schema from the
`address` module and rename the database to make things clearer.
This makes the divergence check on push slightly more lenient, by
allowing rolling back to a previous commit in the canonical branch
history.
Without this, it can become impossible to roll-back commits in certain
situations.
With the latest changes to fetching, `--scope all` will now clone all
remotes listed by the serving node.
Simplify by the `rad-clone-all` example by removing the extra fetch
and removing the corresponding TODO.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The git-daemon code is no longer needed since git-upload-pack is used
in its place.
Remove the git-daemon code from the runtime and worker.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Wire up the `radicle-fetch` crate to the fetch logic in the
`radicle-node` worker.
Any required traits for storage, identity, tracking, and transport
used by `radicle-fetch`.
The worker adds a new module to run the `git upload-pack` process and
pipes the `stdout`/`stdin` to the respective channels.
The `Worker` type required adding a `FetchConfig` to easily allow the
configuration of fetches and passing the signer and tracking store
location through to the fetch `Handle`.
The channels code is adapted to have a new writer that always flushes
when a call to `write_all` is made. This is necessary since the
Gitoxide code never calls `flush`, and so it will hang since no data
is sent until `flush` is called.
Since `UserInfo` is used to set up a newly cloned repository, this
changed some of the SHA1 hashes output in the CLI test examples. It
was also necessary to update the CLI's clone code to be deterministic
in its output for tests.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `radicle-fetch` crate introduces a method for fetching from a
server using Git protocol V2, which provides the implementation
control over performing a sequence of fetches and validation over
domain data.
The protocol is built on top of the suite of Gitoxide crates[[0]]. It
builds wrappers on top of Gitoxide's refdb, odb, and transport layers
-- as well as taking the ls-refs and fetch code straight from the code
base to simplify for the needs of this crate.
[0]: https://github.com/Byron/gitoxide
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Add the ability to remove a repository, given its RID.
This is useful in the case of attempting to clone a repository, and if
that process fails -- for example, if the repository is private --
then it's necessary to clean up the initially created repository.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `identity_doc_at` and `identity_doc` methods on `ReadRepository`
returned the unverified identity document. Every usage of these
methods would always verify the document after.
Simplify the code by calling verified in the methods and directly return the
verified identity document.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Add an enum to represent the special references, `rad/id` and
`rad/sigrefs`. These can be used for cases where it's more manageable
to work with the enum type over checking for reference string equality.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Make the validation logic customisable by extracting it out into a
separate trait that can be defined by other crates.
The Repository validation logic stays the same.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Instead of returning the unsigned references, the validation of a
repository's remotes returns the set of validation errors. The caller
is then expected to handle these validation errors appropriately, for
example, by logging them and re-raising an error.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Add a type to capture user information that can be used for Git
configuration and signatures.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `service::tracking` module has nothing specific to the
radicle-node code. This means that it can be moved to radicle under
the `node::tracking` module.
This makes it more resuable in other contexts, for example, for
fetching logic.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Similar to the `id` function, add a `sigrefs` function to construct
the reference name for a given remote's `rad/sigrefs`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Introduce a SignedRefsAt type that combines a SignedRefs along with the Oid
it was loaded at.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
This change allows users to track/untrack peers and repos without a
running node. This fixes a bug when running `rad init` without the node
running, where the repo would be tracked with a different scope than
what was specified on the CLI.
We were using a breadth-first search which doesn't work as-is for
topological ordering in all cases. Instead of using Kahn's algorithm
which is a little complex, we switch to a depth-first search.
The `stable-commit-ids` feature flag was being included in non-dev,
non-test builds because of how cargo works (or doesn't work).
For now, we ignore the tests that this affects, and they can be run with
the `RAD_COMMIT_TIME=1514817556` environment.
This is very much not up to the standards of an official Debian
package. To make a proper package of Radicle will require packaging
all the dependencies first, and the Radicle crates need to be
published to crates.io, among other things. This is just a lazy first
cut to see if there's any interest in a .deb.
Currently we only handle data Uris, but when a user sends a comment that
has a existing git Oid, due to editing a comment we also need to handle
this case.
Signed-off-by: Sebastian Martinez <me@sebastinez.dev>
The `AliasStore` mechanism was added but not used in the `rad node`
command. Since the launched node itself can display aliases based on
the address book, it would make sense if the `rad node` command could
do this as well.
Use the `Profile`'s `AliasStore` implementation to fallback to an
alias if the tracking store does not contain an alias -- essentially
using the address book instead.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
If a `rad` command that requires an RID is executed within a
subdirectory of a Radicle project, then it will fail saying:
✗ Error: Current directory is not a radicle project
Improve this flow by searching parent directories and/or using the
GIT_DIR environment variable.
This is safe as long as the operations are read-only when calling
`cwd`. Currently they are:
- `Repository::open_ext` -- will not create a repository, only open
one.
- `Repository::find_remote` -- this will only read the remote from the
config and get the URL to parse the RID.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
The `rad patch checkout` command would blindly create a branch when
being executed. If there was a branch already existing and user had
committed to that branch, switched without updating the patch, and
then switched back, then they would lose the commit on that branch --
the original commit of the patch would be taken instead.
Fix this by first checking if the branch existed beforehand, and
setting the workspace to use that branch's commit.
Add a test to ensure that this works as expected.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
When specifying a `patch.base` to point to for stacking patches, it's
useful to accept any kind of revspec, e.g. `HEAD^`.
Change the argument parsing to use the `Rev` type instead, and change
the `rad-patch-ahead-behind` to use `HEAD^` to test the example usage.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
`cargo clippy --all --all-targets` is warning that there are multiple
redundant clones.
Apply the suggestions by removing the calls to `clone`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
When testing the CLI, "since"-calculations will result in different
values over time:
now
1 month ago
2 months ago
10 months ago
1 year ago
10 years ago
Initially, the solution was to use the wildcard match from `snapbox`,
`[..]`, to match on these values. However, since these values change
in length it means that they will affect the width of the tables
produced by the CLI.
To ensure the width is stable across runs over time it's possible to
use a time that is the same as the commit time used at testing.
To do this the `Timestamp` type is now a newtype which has a
constructor `now` which will use the `RAD_COMMIT_TIME` env variable
for construction, if it's available and in debug mode. This ensure
that all "since"-calculations will result in `now`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
Introduces a new COB type to store repository identity documents.
The reason for this change is to:
1. Simplify the code, as the identity document logic resembled a COB,
yet it had custom logic. This allows existing COB code to be used for
identities.
2. Make identity document update logic more flexible, since COB actions
can be added in the future.
3. Re-purpose existing tools around COBs to work on identities, eg. `rad
cob`.
4. Unify the concept of an identity change proposal, with regular identity
changes. This means we can remove the `id.proposal` COB in favor of
using the `id` COB itself.
Notes
-----
* Each repository has one Identity COB.
* The `Proposal` COB has been repurposed into the `Identity` COB.
* Identity documents are stored as *embeds* inside the Identity COB
actions.
* The action that contains new document versions is called `revision`,
just like the Patches COB.
* The namespaced `rad/id` ref is a symbolic reference to that Identity
COB.
* The canonical `rad/id` ref is a direct reference to the commit in the
Identity COB that contains the latest *accepted* revision of the
document.
* All commands for managing identities have been folded into `rad id`.
Hence `rad delegate` and `rad edit` are removed.
* The concept of "rebasing" an identity document is gone.
* The `rad id` output has been updated to match the style of other
commands.
* When a revision has enough signatures, it is automatically adopted as
the current identity, there is no longer the need for a "commit"
action.
* When an identity revision is proposed, and the current identity has a
threshold of `1`, that identity is automatically accepted due to the
above point.
* The idea of "verifying a peer's identity branch" no longer applies, as
COBs cannot be verified one history at a time.
* Since the root commit of the Identity COB does not have a "Resource"
to point to (it would have to point to itself, which is impossible),
we've made the resource id optional for COBs.
The mock transport was not thread-safe when multiple threads used the
same node ids. This is fixed by including the thread id as part of the
storage key.
We employ various tricks to ensure that tests are reproducible.
In this case, having stable commit ids allows us to test specific
traversal orders of a COB history in cases of concurrent updates.
This commit introduces the `FileStats` struct, that adds the insertions
and deletions for each file to `DiffContent`.
Signed-off-by: Sebastian Martinez <me@sebastinez.dev>
The `graph_descendant_of` method is strict in that it will return
false if the two `Oid`s are equal -- they are strictly not
descendants.
If the `head` is the same as the `canonical_oid`, no error should be
returned and the `push` can happen without any issues.
Add a check for `head != canonical_oid` before the
`graph_descendant_check`.
Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
We were being too strict by requiring that no routes existed that
weren't passed into the function. This isn't really useful for testing,
since we're more interested in checking that certain routes are there,
rather than testing that *only* those routes are there.
cloudhead
Sebastian Martinez
Fintan Halpenny
Rūdolfs Ošiņš
Michalis Zampetakis
Lars Wirzenius
dietrich ayala