Building a lightning node server and command-line interface (CLI)

I spent the two weeks working on LN-Node — a CLI and server built on top of the Lightning Developer Kit (LDK) sample lightning node. I wanted to understand some of the inner workings of a lightning node and thought the best way to learn was to dive head-first into the LDK sample node. In this time frame, I forked and re-architected the LDK node application into one that has a separate CLI binary and server, embedded within the node, listening for requests from different client types. In this article, I try to explain my thought process while learning and building. It also contains information on how to test the application and upgrades I would love to implement sometime in the near future.

Architecture

This section covers some of the architectural modifications I made. It describes the state of the ldk-sample, the server, and the CLI binary.

Lightning Node: ldk-sample

This binary has remained largely unchanged from what I forked. A lightning node is a complex application with several parts coupled across the lightning protocol suite. The thought and decision-making that went into the network, messaging, p2p, routing, and payment layers were things that I didn’t want to change, given my scant but growing knowledge about the lightning network architecture. The fact that certain features are built across multiple layers meant that any changes I made, with the expectation that they would be constrained to one, would trickle across other layers.

The ldk-sample node has a coupled CLI as the only way to interact with the node. This CLI process commands that allow peer connection, channel opening, invoice generation, sending payment, signing messages, and a handful of other commands to glean information about the state of the node and to act in a channel with channel partner/peer.

Figure 1: ldk-sample node

My goal for this project was to decouple the CLI and run the node in ways akin to how LND runs, i.e. lnd node running separately with lncli sending commands and retrieving node state responses. Within the LDK node, I decided to build a server that would listen to requests from the CLI, as well as other clients capable of sending requests to the server. The diagram below shows a depiction of the architectural changes I decided to make.

Figure 2: LN-Node

Server

In the entry point to the ldk-sample project is a call that polls start_ldk() to start the node. Within this start function, the CLI for the node is started too. This interface polls for users’ input, matching them against defined lightning commands to, for example, get a node's information, or list connected peers, among others.

Listing 1: ldk-sample entry point

I decided to replace the CLI’s loop within the node’s start-up function with a running instance of an actix-web server as shown below.

Listing 2: Replacing the CLI loop with a server

To pass the node’s variables to the server’s application state, I defined a struct NodeVar<EventHandler>, generic for any field that implements EventHandle. This event handler takes in the node's handle_ldk_events function and polls in another tokio runtime, awaiting the various event outcomes (e.g. funding generated, payment received, etc) possible within a payment channel.

Listing 3: Implementing an event handler

With this implementation, I could then create handlers for each command/path as shown below.

Listing 4: server handlers

For demonstration purposes, the node variable struct node_var is passed to the list_peers request handler within which the peers connected to the LN-Node can be retrieved and sent back to the calling client (CLI or HTTP).

Listing 5: Retrieving connected peers

Beyond just the CLI making requests to this server, I wanted external HTTP clients to have the same capabilities, i.e. getting information about the node’s state, and acting to transact with other nodes in the network. I thought this would be a nice-to-have because of its “perceived” ease of use when compared with CLIs. Imagine having a dashboard with a great user interface to interact with your lightning node.

CLI binary

For the CLI to run separately, I created a different binary lnnode-cli within the project to take in command inputs, extract matching request body, send a request to the node's server with the reqwest crate, process server responses, and print to the caller's terminal.

Listing 6: Collecting user command line input

To demonstrate the construction of a request body for the connectpeer command, the corresponding match arm parses the command’s arguments, i.e. public key, host, and port, to generate a hash map.

Listing 7: Constructing hash map for request body

With the map constructed, the reqwest client can send the request to the server with the right body, i.e. json(&command). To further explain what right means here, a user would type out a command on the terminal to connect with a new peer.

The request body constructed from this terminal input is a hash map that is saved to the variable command .

Listing 8: Sending request to server with request body

With the returned response, the processing can be done and printed to the terminal within a match arm for the path visited.

Listing 8: processing server response to view a listing of connected peers

Testing

If you do not have bitcoin core and either LND, eclair, or c-lightning installed, testing the application will be difficult because of all the setup required. However, with the installation of Lightning Polar , you can create a node network with as many as 3 lightning nodes and 1 block source (i.e. bitcoin core running in regtest). LN-Node can be connected to polar’s bitcoind and establish direct channels with any of the other nodes in the network. With the pre-requisites covered, clone the LN-Node repository and change to its root directory.

Within the root directory is a startup script lnnode.sh I wrote to make the process easier. With polar running and bitcoin node selected, copy the connection details from the Connect tab and populate the matching fields in the start-up script.

Figure 3: Polar’s bitcoind connection information highlighted in red

Having done that, open a terminal and run the script.

This starts up the node and the embedded server. Switch to another terminal to test the CLI and get a list of the command and arguments that can be passed.

With a HTTP client like Postman or Insomnia, the same requests to the node server can be made just like the CLI. Below is a screenshot of the request to open a channel with a peer with public key as shown, listening on host 127.0.0.1 and port 9736, with a channel funding amount of 100,000 sats.

This node information was gleaned from another node (carol) on polar. The repo also has a JSON file (insomnia_rest_api.json) importable into a client like Insomnia to test the endpoints.

Upgrades

As my knowledge of the lightning network and Rust grows, I expect to periodically revisit and upgrade this project primarily to achieve the following:

1. Remove reliance on the ldk-sample node by building a lightning node from scratch
2. Capture the channel events in server responses
3. Use a CLI-dedicated crate like clap to build a better client

Conclusion

The building experience was more challenging than I anticipated. I struggled with Rust more than I expected to, given that I had spent over a month learning the language. However, some positives to take from this experience is my increasing comfort reading and writing the language, as well as my growing knowledge of the lightning network.

I am always open to helpful feedback. Reach out here or on Twitter (@engb_os) if you have some or just want to chat.

References

1. LDK-sample: https://github.com/lightningdevkit/ldk-sample