Commit Graph

184 Commits

Author SHA1 Message Date
Alexis Sellier cdc3c8a265
node: Try to cleanup socket file on exit
Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-25 17:51:49 +01:00
Alexis Sellier b4f44f39f2
Update `netservices` dependency 2023-01-25 15:35:24 +01:00
Alexis Sellier 963ad3c3d7
Get `rad clone` working correctly 2023-01-25 11:35:20 +01:00
Alexis Sellier c888f40660
node: Respond to `announce-refs` control 2023-01-25 10:21:57 +01:00
Alexis Sellier d324eab377
node: Remove control socket at the end 2023-01-25 10:21:57 +01:00
Alexis Sellier fea14ff7b3
Update `clone` functionality for new replication 2023-01-25 10:21:57 +01:00
Alexis Sellier ef950d515a
node: Fully integrate worker fetch into service 2023-01-25 10:21:57 +01:00
Alexis Sellier ec2e7d7ee1
node: Refactor fetch response code
Only user-requested fetches require a response.

Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-25 10:21:57 +01:00
Alexis Sellier e6f56ff3bc
Update `io-reactor` dependency 2023-01-25 10:21:57 +01:00
Alexis Sellier 3aff91f9b7
Correctly initialize home in `Profile::init` 2023-01-22 13:39:48 +01:00
Alexis Sellier 8318b07d89
node: Fix spurious test failures
In some circumstances, the nodes are already offline when the objects
are dropped.

Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-19 10:20:23 +01:00
Alexis Sellier 3bc07195f5
Migrate to `localtime` from `nakamoto`
The only thing we needed nakamoto for was `LocalTime`.

Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-18 18:03:11 +01:00
Alexis Sellier d25c47ad0e
node: Get rid of worker response channel 2023-01-18 17:45:46 +01:00
Alexis Sellier 7e55ed346a
node: Update log target to match type name
Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-18 17:45:46 +01:00
Alexis Sellier 067b532154
node: Get replication test to pass
Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-18 17:45:46 +01:00
Alexis Sellier 723cdcf233
node: Update `io-reactor`
Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-18 17:45:46 +01:00
Alexis Sellier b9484b6832
node: Implement graceful shutdown
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2023-01-18 17:44:16 +01:00
Alexis Sellier f33e79f14b
node: Fix redundant announcements
This change makes sure that nodes don't gossip announcements back to
the announcers.

Since the tests were relying on this mechanism to populate the routing
table, we had to change a few more things, such as adding the inventory
to the node's routing table.

Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-18 08:48:33 +01:00
Alexis Sellier 3d04849b7b
node: Properly parse replication data
Fix all of the `FIXME` in the worker.

Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-17 12:59:00 +01:00
Alexis Sellier df9346fefb
node: Fix E2E tests
The tests were flaky because the routes are sampled at a moment
when gossip could still be happening. Instead, we check the
local inventory directly to build the expected routing table.

Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-17 12:59:00 +01:00
Alexis Sellier 3bedb4438e
node: Fix a few logging-related things
Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-17 11:01:36 +01:00
Alexis Sellier 97d4729fa4
Switch to crates.io for `cyphernet` 2023-01-17 11:01:36 +01:00
Alexis Sellier be9def8b11
Rename `profile::Paths` to `profile::Home`
This makes the intent clearer.

Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-17 11:01:36 +01:00
Alexis Sellier d31968509d
Speed up node E2E tests
By allowing the node to run using a `MockSigner`, we speed up tests
significantly, since PBKDF2 doesn't need to run.

Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-17 11:01:36 +01:00
Alexis Sellier 741f7c9651
node: Rename `Transport` to `Wire`
Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-17 11:01:36 +01:00
Alexis Sellier 63e7248847
node: Complete Noise handshake integration
Co-authored-by: Dr. Maxim Orlovsky <dr.orlovsky@gmail.com>
Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-17 11:01:36 +01:00
Alexis Sellier 1aba9812c7
node: Request recent messages on connection
Instead of only being interested in messages from "now", we request
messages from one hour in the past up to now. This lets nodes catch
up on state more quickly, without having to wait for new announcements.

Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-17 11:01:36 +01:00
Alexis Sellier 6d420813f5
node: Fix attempted session state
We shouldn't be marking attempted peers as "connected". We should wait
for the connection to be properly established.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2023-01-13 15:44:01 +01:00
Alexis Sellier 438ae84304
node: Implement E2E node test
There are few unrelated changes in this commit that I wasn't able to
split out, mostly due to laziness, so I'll list them here:

* The node startup logic is moved out of the main function, and into a
  dedicated functionin `client.rs`.
* There is a tentative worker implementation that is untested, and
  handles git replication both ways. It's likely lacking in various ways
  but I wanted to get it in there anyway.
* `Address` was moved from `radicle-node` to `radicle`, because it's
  needed to send `Connect` commands from the node handle.
* The `netservices` dependency was updated.
* Finally, there is a first end-to-end node test, which currently
  doesn't use NoiseXK, but uses the new reactor in `netservices`.

Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-12 20:34:31 +01:00
Dr. Maxim Orlovsky 08334f790b
node: Get rid of `nakamoto::DisconnectReason`
Signed-off-by: Alexis Sellier <self@cloudhead.io>
2023-01-10 16:35:40 +01:00
Dr Maxim Orlovsky cf0d55fb11
node: Update io-reactor
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2023-01-06 11:27:07 +01:00
Dr. Maxim Orlovsky 947c89a398
node: Implement simple worker thread pool 2023-01-05 15:56:08 +01:00
Dr. Maxim Orlovsky a7b1c19746
node: Implement worker thread handover
Also implements listener behavior.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2023-01-05 15:16:38 +01:00
Alexis Sellier f26674a5a9
Upgrade Rust to 1.66
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-12-15 20:55:26 +01:00
Adam Szkoda ac3ee5f709
Build container images for deployment
Signed-off-by: Adam Szkoda <adaszko@gmail.com>
2022-12-14 17:15:40 +01:00
Adam Szkoda 9186d1f30f
Do not install OpenSSL as it's statically linked 2022-12-14 17:15:40 +01:00
Adam Szkoda 294ae8fd24
Put fully static binaries into containers 2022-12-14 17:15:40 +01:00
Adam Szkoda 4abb24feca
Dockerize services
Signed-off-by: Adam Szkoda <adaszko@gmail.com>
2022-12-14 17:15:40 +01:00
Dr Maxim Orlovsky aa02c52a2f
node: Integrate new Noise_XK transport
This is a large change that introduces transport encryption via
a Noise_XK handshake. The inner `Service` now works directly
with `NodeId` and doesn't need to interact with socket addresses.

The transport logic takes place in `wire::transport`, and the
established sessions are passed down to `Service`.

Note that the i/o implementation in the `poll-reactor` crate still
needs testing, and there is still work to be done to integrate the
Git transport as part of this architecture.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-12-13 14:59:06 +01:00
Slack Coder e7f2637126
node: Limit number of Refs in RefAnnouncements
Use the BoundedVec to limit the supported number of refs in
RefAnnouncement messages.  The size (235) is the maximum supported by
message wire size.

To reduce the complexity of the changes, BoundedVec is used again
instead of using a new bounded Refs or BTreeMap.  This requires
defining new Decode/Encode logic for Ref's element types, and
workarounds for arbitrary instances.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-12-09 21:54:10 +01:00
Slack Coder c676d8b99a node: Set test peer's time on creation
Allow setting a test peer's local time to avoid slight race conditions in
tests.  The race condition's happen because peers created within a test may
unexpectedly have slightly different local times.

To support this, a struct is used as argument to peer::config(..).  The
linter 'cargo clippy' will complain if instead of using a struct, the local
time is passed as an additional parameter in config()'s function call.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-12-09 13:14:41 -05:00
Alexis Sellier 7b8bb08b08
Consolidate `node::Handle` traits into one
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-12-09 10:52:18 +01:00
Alexis Sellier a100f1c683
cli: Implement tracking correctly
The tracking command was tracking a project when it should
have been tracking peers.

We also add the ability to pass an alias when tracking from
the CLI.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-12-09 10:52:18 +01:00
Fintan Halpenny 5f80ee9671
cobs: allow for batched operations
Sometimes it is more efficient to perform multiple operations at once
and store those in the collaborative object graph together.

Change the contents to be a non-empty sequence of byte sequences,
i.e. NonEmpty<Vec<u8>>. These are stored as blobs in a git tree in
order by their index in the non-empty sequence. They are, in turn,
retrieved in order of their original index.

Change the conversion of an `EntryWithClock` to return a non-empty
sequence of `Op`s, which are in turn used to apply many changes in
each `from_history` implementation.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2022-12-07 17:11:40 +01:00
Dr Maxim Orlovsky 7c9ee0f911
node: Remove `RefClock`
It wasn't used anymore.
2022-12-07 16:06:35 +01:00
Alexis Sellier 1007756238
node: Add tracking configuration store
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-12-07 15:41:56 +01:00
Alexis Sellier db3568db2f
Update `sqlite` dependency to 0.30.3
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-12-07 15:41:56 +01:00
Alexis Sellier f9c81ab382
Replace `quickcheck` with `qcheck`
The latter is a fork of the former, which adds const generics
and a bunch of other quality of life improvements that aren't
in the original crate due to it being unmaintained.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-12-04 20:40:54 +01:00
Slack Coder 86beae2ebb
node: Document and test `BoundedVec`
Borrow documentation from Rusts standard library to document BoundedVec's
functions, and add documentation tests where appropriate.

And make it's deref trait return a slice instead of a Vec, which is more
common in rust.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-12-01 21:04:50 +01:00
Slack Coder fe60305455
node: Define naive message vector limits
Restrict message size decoding by limiting Message's vector size.
Introduce the BoundedVec type to do this on the type level for
convenience and clarity.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-12-01 14:54:40 +01:00
Alexis Sellier be789a487f
Move `hash` module to `radicle-crypto`
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-12-01 12:05:44 +01:00
Dr Maxim Orlovsky da55e521a1
Use socket addr instead of Ip for the address book 2022-11-29 17:58:38 +01:00
Slack Coder f5d06f6138 node: Fix addresses sent to peers on conn.
Send the node's configured external addresses instead of its listenning
addresses.  External addresses are ones which the service is reachable
from the outside world, the listening addresses can be the wrong context
(local area network, or 0.0.0.0).

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-11-23 13:13:56 -05:00
Alexis Sellier 308112618f
Get around circular-dependency
This was causing an issue in rust-analyzer, and probably would fail
in other ways via cargo. In any case it wasn't a good idea.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-11-22 13:41:32 +01:00
Dr. Maxim Orlovsky 3e48548745
Integrate framing and muxer into wire protocol 2022-11-21 16:05:53 +01:00
Dr. Maxim Orlovsky 13c9491e49
Implement framer and multiplexer 2022-11-21 16:05:53 +01:00
Dr. Maxim Orlovsky 983b2dc534
Two-stage connectivity with handshake 2022-11-21 16:05:53 +01:00
Dr. Maxim Orlovsky 008681a2ca
Complete transcoder documentation 2022-11-21 16:05:53 +01:00
Dr. Maxim Orlovsky 4d77f95c21
Disconnect peers on failed handshakes 2022-11-21 16:05:53 +01:00
Dr. Maxim Orlovsky e9ae5897f7
Complete transcoder
* Encode/encrypt data sent to the remote peer
* Remove unused Transport
* Rename Decoder -> Deserializer
2022-11-21 16:05:53 +01:00
Dr. Maxim Orlovsky 56fb4d67a9
Introduce transcoder parameter to Wire & Transport 2022-11-21 16:05:53 +01:00
Dr. Maxim Orlovsky 6b87eed670
Identify remote peers by full `SocketAddr`
Wire inboxes were kept per IP address. However, there could be multiple peers
connecting from the same IP (for instance, if it is an IP of a NAT).
2022-11-21 16:05:53 +01:00
Slack Coder 5472931ff0 node: prune project routing table
Avoid wasting persistent memory by cleaning up stale routing entries
when the routing table is considered constrained.  Limit pruning to
improve resiliency in the face of long intervals between service
downtimes.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-11-15 12:26:34 -05:00
Slack Coder a74cb416c1 node: sync project routing for known nodes
Keep the node's understanding of project hosting location current by
updating how Inventory Announcement messages are processed.

The Inventory Announcements contain a node's entire listing of projects.
Replace the routing table information for the announcer with those from the
message.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-11-15 12:26:08 -05:00
Slack Coder 186eb42145 node: use announcer's timestamp in project inventory
Address a FIXME in service.process_inventory() by using the announcer's
time instead of the service's.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-11-15 12:26:08 -05:00
Slack Coder 507bc2237a node: make tests in tests mod run
Fix a problem from a recent commit which caused tests in the tests
module not to run.

The list of modules in 'lib.rs' should use the 'tests' module's new
location.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-11-11 09:24:15 -05:00
Slack Coder 503a618743 node: announce node only when accessible
Avoid redundantly emitting Node Announcements about it's publicly
available addresses when it has none.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-11-09 09:49:21 -05:00
Slack Coder a9b38dc1ae node: let others know their address
Include the node's IP address in the Node's announcements.  Source it
from the service's configuration which can assumed to be populated with
default values when available.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-11-09 09:49:07 -05:00
Slack Coder f3d867af4d node: keep a healthy connection to the network
Try having a number of outbound connections to other Radicle nodes at any
given time.  The number has been defined earlier as TARGET_OUTBOUND_PEERS.
All network nodes are treated equally, there is no preference.

Attempts to keep connected happen at set intervals (IDLE_INTERVAL) or when
a node disconnects.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-11-09 09:49:07 -05:00
Slack Coder ff045e0813 node: use term 'session' over 'peer'
Keep up to date with recent changes of renaming service.peers to
service.sessions.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-11-09 09:49:07 -05:00
Slack Coder a91dcfb100 node: sync known sessions on disconnect
Make sure the service is current by removing disconnected peers.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-11-09 09:49:07 -05:00
Alexis Sellier f05a040be6
Make `track` functions return correctly
Previously we always returned `true`. Now we return
what the node actually returns.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-11-09 13:29:54 +01:00
Alexis Sellier a8dc065dce
Use database transactions where appropriate
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-11-08 09:39:23 +01:00
Alexis Sellier 609d05fff2
Update `sqlite` dependency
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-11-08 09:39:23 +01:00
Alexis Sellier 32cd94dd23
Add `rad-init` command
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-11-07 22:01:49 +01:00
Alexis Sellier 223d36086e
Add `radicle-cli` crate's `terminal` module
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
Co-authored-by: xphoniex <dj.2dixx@gmail.com>
2022-11-07 14:48:01 +01:00
Alexis Sellier 45c7f3049b
Update to Rust 1.65
* Use new let-else syntax
* Fix new clippy lints

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-11-05 12:47:26 +01:00
Alexis Sellier dc79130988
Zeroize passphrases
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-11-03 16:52:29 +01:00
Slack Coder c7706156c6
node: fix test dependency error
Enable Radicle Crypto's test feature.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-11-03 16:02:50 +01:00
Alexis Sellier 092ccfdcbc
Add `profile::env` module
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-11-03 13:06:08 +01:00
Alexis Sellier 577b632056
Use `Signer::try_sign` in interactive contexts
In interactive (user) contexts, use `try_sign` instead of `sign`.

We also revisit the `Profile` by adding a `signer` method that can
fail, eg. if we couldn't connect to ssh-agent.

Finally, we replace `UnsafeSigner` with `MemorySigner`, which makes
use of `ssh::Keystore`.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-11-03 13:06:00 +01:00
Alexis Sellier a79ef67b2a
Add `Signer::try_sign` method and cleanup usage
To support ssh-agent, add a signing method that can fail.
Also cleanup usage so that we always accept signer references.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-11-03 10:12:22 +01:00
Slack Coder 25c982cd8c
node: use method to access session's Node ID
Make accessing an important session value, the Node ID, as a method to
Session.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-10-31 18:17:27 +01:00
Alexis Sellier 6f3a598cc1
Remove `git-url` dependency
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-10-31 18:14:54 +01:00
Alexis Sellier 7b7e83fbc7
node: implement node announcement sybil resistance
We implement a small proof-of-work on the node announcement message
to prevent the routing table from being filled by fake entries.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-10-31 18:13:14 +01:00
Slack Coder 3c900f106f radicle-node: distinguish announcment nodes
Its easy to confuse what/who 'node' or 'session' is.

Rename session to 'relayer' to convey its role as message propagator.
Rename the announcement's node to 'announcer', clarifying its as the
source.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-10-31 11:40:50 -05:00
Slack Coder a8895aee80 node: follow up on session renaming
Rename the module to session from peer to be consistent with recent
renaming of the type from Peer to Session.  Remove stutter by renaming
session::SessionState to session::State, and session::SessionError to
session::Error.

Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-10-28 09:57:56 -05:00
Fintan Halpenny c0b0d29e19
radicle-crypto: split out from radicle
The crypto types are useful to use in other components that are
separate from the radicle crate, e.g. the incoming radicle-cobs
crate.

Separate out the types into a radicle-crypto crate. This crate defines
the usual PublicKey, SecretKey, and Signature types. It also provides
the ssh functionality, under the `ssh` feature. It is also necessary
to provide the `From` impl for `Component` here, which again is
guarded by the `git-ref-format` feature.

The `Arbitrary` implementation are moved. The arbitrary decision of
redefining `ByteArray` in both radicle-crypto and radicle was made.

Signed-off-by: Fintan Halpenny <fintan.halpenny@gmail.com>
X-Clacks-Overhead: GNU Terry Pratchett
2022-10-27 20:40:39 +01:00
Alexis Sellier 2a1154ef87
Re-work git transports
The changes in this commit are all intertwined and it was hard to
split them into smaller commits:

* Use custom transports everywhere

Instead of `file://` and instead of using a child process for fetching
and pushing; we go through our custom transports, which use the `rad://`
and `heartwood://` schemes.

* Introduce a 'mock' remote transport

To test the remote transport without needing to spawn TCP streams,
we add a mock transport that works between storages on the file
system.

* Get rid of node's `git-url`

Instead of git urls, nodes simply use their node-ids to replicate.
2022-10-26 21:40:19 +02:00
Alexis Sellier 2ec320acec
Use 'primary key' constraint instead of index
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-10-19 22:39:31 +02:00
Alexis Sellier 706fbc6550
Get the node running
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-10-19 22:39:24 +02:00
Slack Coder 10d8519101
node: improve ping/pong tests
Signed-off-by: Slack Coder <slackcoder@server.ky>
2022-10-18 17:23:31 +02:00
Alexis Sellier 09c438f8fb
Remove `Envelope` type
This type that was wrapping `Message` isn't very useful here.
If we decide that we do need something like that, it's best handled one
layer below.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-10-18 16:45:10 +02:00
Alexis Sellier 156a3a576a
Enforce max size for `Message` type on wire
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-10-18 15:44:06 +02:00
Alexis Sellier f5e881b5a9
Move message size constant to `Message` type
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-10-18 13:06:45 +02:00
Alexis Sellier c306392d13
Improve wire module
* Set `Size` correctly to `u16`
* Remove `usize` support, since it's OS-dependent

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-10-18 13:04:12 +02:00
Alexis Sellier 874386540a
Document service constants
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-10-18 13:04:12 +02:00
Alexis Sellier ae95b229f5
Activate lint for dead-code checking
For test code, we need it in a couple of places.

Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-10-18 13:03:34 +02:00
Alexis Sellier 1df5cf0c45
Remove redundant docs from trait impl
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-10-18 13:00:55 +02:00
Alexis Sellier 2499317589
Implement `Handle::listening`
Signed-off-by: Alexis Sellier <alexis@radicle.xyz>
2022-10-18 13:00:50 +02:00