How To Create A Monero Service in Linux : Part 3 – Security

Introduction

In Part 1 we turned our Monero daemon into a service.

Then in Part 2 we outlined how to troubleshoot a service that is failing to start.

Here in Part 3 we will look in more depth at security matters. Specifically firewalling with firewalld, SELinux and securing your daemon’s API.

Firewalld

What is Firewalld?

Firewalld is the ‘new’ front-end to the firewall features of the Linux kernel.

It’s definitely a good idea to use a firewall on Linux as it’s one of the most simple and effective elements of securing your system. Further, firewalld operates at the host level, whereas a network firewall only operates on traffic flowing between networks. So unlike a host firewall, it can’t protected from other systems on the same network – such as if the unlucky circumstance that another host on your network is compromised.

When to use Firewalld?

If your monero service is only listening on the internal loopback interface (localhost or 127.0.0.1), then often there is no firewall policy on that interface. That’s because only processes on the local system can talk to the loopback interface. By definition, the address 127.0.0.1 is not usable over a network.

But if your service is listening on any externally-facing interface – ethernet or wireless – then it is a good idea to make sure your firewall is on, and you are making an exception to the inbound filters for connections to monerod. This is the interface we’ll use to accept connections from wallets on other systems.

Is Firewalld Running?

You can check this with:

systemctl status firewalld

If you do not have firewalld installed or running, we won’t cover how to remedy that here, but it is definitely recommended.

Firewalld Policies

A default firewall policy should already be installed on your Linux distribution. Usually these do two things:

• Allow all outbound connections – this system to other systems
• Block most inbound connections except ssh– other systems to this one

There may be few other connections allowed in my default, but it is very likely your rpc-port is blocked in the default firewall policy. So we are going to create a new rule for it below.

First let’s confirm that by taking a look at the current rules:

> firewall-cmd --list-all
public (active)
  target: default
  icmp-block-inversion: no
  interfaces: eth0
  sources:
  services: dhcpv6-client ssh
  ports:
  protocols:
  masquerade: no
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:

This output tells us that the default zone is public and the interface eth0 is attached to it. We can also see that two services are allowed to talk into the system, dhcpv6-client and ssh. Note that the loopback interface (often lo0) is not part of the default zone.

Determining the Firewall Rule to add

In Part 1 we configured monerod to listen for wallet connections on a particular TCP port using the rpc-bind-ip option. In this guide we actually used the default port for monero’s mainnet, which is 18081.

We now have two choices. We can allow any connection from other systems to TCP/18081 on this system, or, we could only connections from certain IP address ranges to TCP/18081. Everyone’s needs will vary, so I’m going to show you how to do either.

Allowing Wallet/RPC Connections From Any Source

Here we just allow the port via a port rule:

firewall-cmd --add-port 18081/tcp
firewall-cmd --add-port 18081/tcp --permanent

To confirm the change is applied:

> firewall-cmd --list-all
public (active)
  <..>
  services: dhcpv6-client ssh
  ports: 18081/tcp
  <..>

Allowing Wallet/RPC Connections From Specific Sources

To do this, we need to add what firewalld calls a rich rule.

First let’s suppose that you only want to only allow from other systems on your home network, and your home network uses the IP network 192.168.0.0/16. Add the rich rules below:

 firewall-cmd --add-rich-rule='rule family="ipv4" source address="192.168.0.0/16" port port="18081" protocol="tcp" accept'
 firewall-cmd --add-rich-rule='rule family="ipv4" source address="192.168.0.0/16" port port="18081" protocol="tcp" accept' --permanent

To confirm the change:

> firewall-cmd --list-all
public (active)
  <..>
  rich rules:
      rule family="ipv4" source address="192.168.0.0/16" port port="18081" protocol="tcp" accept

Firewalld Summary

That’s pretty much it. Even though Firewalld has been around for years, its logging capability is not great. Fortunately it’s quite reliable and you can be fairly sure the policy is doing what is says.

TCPDump

If you’re still not confident the right traffic is at least getting to your system’s network interface, tcpdump is an ideal utility for this.

dnf install tcpdump
tcpdump -i <interface-name> -n -nn -v 'tcp port 18081'

This will display a line with IP and port details for each packed that matches the filter tcp port 18081.

SELinux

For this section we are assuming you want to run SELinux in Enforcing mode (you should want that!), but, SELinux is blocking monerod from running or starting.

What is SELinux?

SELinux is a suite of tools and kernel features that provides security policy and enforcement over much of the activity happening internally on a Linux operating system. Whereas a firewall controls network traffic coming in and out of your system, SELinux acts like a firewall for internal system activity. For example, every time a process wants to read or write to a file, SELinux checks and enforces the policy for that specific combination of process and file.

SELinux used to be something most people turned off, but it’s become a lot more usable and no less essential in recent years. Nevertheless, anytime you modify your system – for example by adding new files to it – you are potentially going to have to contend with SELinux. That’s the price of good security.

How does SELinux work?

Very simply, SELinux uses the concept of labels in it’s policy definition. Labels can apply to files and processes and determine the right policy to apply. Further there are several types of labels.

We don’t need to know too many of the details, but let’s give a quick example. When a process wants to open a file, it makes a system call to the kernel. If SELinux is loaded the kernel triggers a policy check. The SELinux kernel module will look at the source labels (in this case of the process) and the destination labels (iin this case a file) and look for a policy rule matching that specific combination. If it doesn’t find one, the action will be blocked.

Why is SELinux Confusing?

The problem usually starts because a process that has an action blocked by SELinux doesn’t know that’s going on. The process will know something has gone wrong, because the system call will fail, but they may report an error that is misleading or unclear to the user. This is because many applications are not written with SELinux policy violations in mind.

Fortunately I am going to show you how to easily check for SELinux policy violations and also how to solve them to some extent. In short, the fix is usually quite simple – we just have to label the new files appropriately. The challenge is in detecting the problem and then working out the correct labels to use.

Checking Status of SELinux

We covered this briefly in Part 2, where we turned SELinux from Enforcing to Permissive just in case it was causing any problems.

Before we proceed let’s just confirm we are back in in Enforcing mode. In this mode SELinux is still going to both log and enforce policy violations:

> getenforce
Enforcing

If you are Permissive, switch to Enforcing as follows:

> setenforce 1

Installing SELinux Tools

This package can help with solving any SELinux policy violations:

> dnf install setroubleshoot-server
> dnf info setroubleshoot-server
<..>
Description  : Provides tools to help diagnose SELinux problems. When AVC messages
             : are generated an alert can be generated that will give information
             : about the problem and help track its resolution. Alerts can be configured
             : to user preference. The same tools can be run on existing log files.

setroubleshoot basically goes through logs generated by the auditd component of SELinux and packages them in a more human-readable way. It also makes a recommendation about how to fix the problem with an associated confidence level.

Inspecting Labels

We definitely made some change to our filesystem in Part 1, so let’s see what those labels currently are:

ls -ZR /opt/crypto
.:
    unconfined_u:object_r:unlabelled_t:s0 monero
    unconfined_u:object_r:unlabelled_t:s0 monero-data
    unconfined_u:object_r:unlabelled_t:s0 monero-linux-x64-v0.17.1.9.tar.bz2
    unconfined_u:object_r:unlabelled_t:s0 monero-x86_64-linux-gnu-v0.17.1.9

./monero-data:

./monero-x86_64-linux-gnu-v0.17.1.9:
	unconfined_u:object_r:unlabelled_t:s0 LICENSE
	unconfined_u:object_r:unlabelled_t:s0 monero-blockchain-ancestry
	unconfined_u:object_r:unlabelled_t:s0 monero-blockchain-depth
	unconfined_u:object_r:unlabelled_t:s0 monero-blockchain-export
	unconfined_u:object_r:unlabelled_t:s0 monero-blockchain-import
	unconfined_u:object_r:unlabelled_t:s0 monero-blockchain-mark-spent-outputs
	unconfined_u:object_r:unlabelled_t:s0 monero-blockchain-prune
	unconfined_u:object_r:unlabelled_t:s0 monero-blockchain-prune-known-spent-data
	unconfined_u:object_r:unlabelled_t:s0 monero-blockchain-stats
	unconfined_u:object_r:unlabelled_t:s0 monero-blockchain-usage
	unconfined_u:object_r:unlabelled_t:s0 monerod
	unconfined_u:object_r:unlabelled_t:s0 monero-gen-ssl-cert
	unconfined_u:object_r:unlabelled_t:s0 monero-gen-trusted-multisig
	unconfined_u:object_r:unlabelled_t:s0 monero-wallet-cli
	unconfined_u:object_r:unlabelled_t:s0 monero-wallet-rpc

As you can see the -Z option to ls shows us the current SELinux labels. On my system all files under /opt/crypto happen to be labelled as follows:

• SELinux User: unconfined_u
• SELinux Role: object_r
• SELinux Type: unlabelled_t

These labels could be very different on your system. Defaults are very distribution dependant and when you copy files from other locations, the labels from those locations are often preserved.

Viewing SELinux Logs

Now let’s attempt to restart the service to generate some fresh logs.

systemctl restart monerod

Let’s go back to our monitoring terminal and type the following. This will display the last 100 logs generated by the setroubleshoot package we installed earlier:

journalctl -t setroubleshoot -n 100

There can be many errors but I’m going to give just one example. Here is one of the errors original generated on my system before applying any fixes:

Jan 17 15:08:45 crypto.upaya.net.au setroubleshoot[152886]: SELinux is preventing /usr/lib/systemd/systemd from read access on the lnk_file /opt/crypto/monero.

                                                             *****  Plugin catchall (100. confidence) suggests   **************************

                                                             If you believe that systemd should be allowed read access on the monero lnk_file by default.
                                                             Then you should report this as a bug.
                                                             You can generate a local policy module to allow this access.
                                                             Do
                                                             allow this access for now by executing:
                                                             # ausearch -c '(monerod)' --raw | audit2allow -M my-monerod
                                                             # semodule -X 300 -i my-monerod.pp

Notice there is a recommended fix provided by setroubleshoot. Now this fix results in a policy that I would not describe as ideal. In fact we don’t need a new policy rule, we just need to label our filesystem correctly. But the recommendations will usually work in a pinch, so it’s good to know they’re there, in case you fail to figure out a more appropriate fix.

Changing Filesystem Labels

So in my view the problem here is that the default policy supplied with my distribution (Centos 8) is not expecting systemd to need access to files in the /opt directory. That’s because /opt is really outside the normal filesystem hierarchy for a base Linux install. So in principle we just need to label these new files and directories similar to directories that systemd can access.

So here are the changes needed on my system to make the service to run successfully.

These commands add default labels, for the paths we created in Part 1, to the current policy:

semanage fcontext -a -s system_u -t default_t "/opt/crypto(/.*)?"
semanage fcontext -a -s system_u -t bin_t "/opt/crypto/monero/monerod"

Essentially this allows all the matching paths to be read by systemd, and the monerod daemon to be executed. It’s a fairly safe change to make and is not too lax or permissive unlike the recommended fix from setroubleshoot.

This command restores that policy to the actual filesystem, and will update the labels accordingly:

restorecon -vR /opt/crypto/

Test and Test Again

At this point you can try restarting the monerod service and checking the status and logs again.

From here, this is very much a game of whack-a-mole. Each time you update the policy and restart the service, you may get a brand new error from SELinux. So repeat the process until you see no more errors when starting monerod.

SELinux Summary

In summary, if you are still getting SELinux errors after applying my recommended policy change, you can probably resolve the problem by using the recommendations from setroubleshoot. This is still preferable to keeping SELinux in Permissive mode.

Securing the Monero Daemon API

Wallets talk to monero daemons using an API, which monero calls RPC. The monero API allows you to do all sorts of things, but wallets only need very basic access and this is more secure.

Restricted RPC

We already restricted the type of calls that the monerod would accept with this option:

restricted-rpc=1

RPC over TLS (SSL)

Another important security feature is to make sure your RPC communication channel is using TLS. This is the default configuration for monerod so it should already be working. The daemon will generate its own certificate the first time, and wallets will start up TLS if requested (or demanded) by the server. This ensures that private information is not transmitted over your network in clear text – a good idea even for local area network communication.

To force TLS you can add this option to your monerod.conf:

rpc-ssl=enabled

Of course there are also options to replace the self-generated certificate and key with your own.

RPC Login

You can also require wallets to authenticate themselves to the server. Otherwise any wallet that can reach the RPC port could connect to your server.

rpc-login <username>[:password]

This username and optional password are stored in plain text in your monerod.conf or passed to monerod on the command-line. And the username at least is transmitted in clear text unless TLS is securing the communications. So this is not a particularly secure authentication mechanism, but it does add one more layer of protection and one more obstacle to any would-be attackers.

To use a login from a CLI wallet, add the --daemon-login option to the command-line, for example:

/opt/crypto/monero/monero-wallet-cli --daemon-address <rpc-bind-ip>:<rpc-bind-port> --wallet-file <path-to-wallet> --daemon-login <username>

You will be prompted to enter your password afterwards. If you didn’t configure one, just hit enter at the prompt:

This is the command line monero wallet. It needs to connect to a monero daemon to work correctly.
WARNING: Do not reuse your Monero keys on another fork, UNLESS this fork has key reuse mitigations built in. Doing so will harm your privacy.
Monero 'Oxygen Orion' (v0.17.1.9-release)
Logging to /opt/crypto/wallets/monero-wallet-cli.log
Wallet password: XXXXXXXX
Daemon client password:

P2P Protocol

In addition to listneing on a port for RPC communication, your daemon will also be listening on another port for accepting connections from other nodes/daemons. This is called Peer-to-peer (P2P) communication. The default port for monero mainnet is 18080.

Unlike the RPC port, monerod will be listening on the P2P port on all interfaces.

Displaying Open Ports & Connections

You can verify this as follows:

> ss -tapn | grep 1808
LISTEN   0        128               0.0.0.0:18080               0.0.0.0:*        users:(("monerod",pid=1278290,fd=10))
LISTEN   0        128               0.0.0.0:18081               0.0.0.0:*        users:(("monerod",pid=1278290,fd=14))
LISTEN   0        100             127.0.0.1:18082               0.0.0.0:*        users:(("monerod",pid=1278290,fd=21))
ESTAB    0        0               10.0.33.7:58238          1.216.189.9:18080    users:(("monerod",pid=1278290,fd=31))
ESTAB    0        0               10.0.33.7:49816        1.234.190.246:18080    users:(("monerod",pid=1278290,fd=29))
ESTAB    0        0               10.0.33.7:34326           1.10.211.9:18080    users:(("monerod",pid=1278290,fd=34))
ESTAB    0        0               10.0.33.7:45206       1.201.219.175:18080    users:(("monerod",pid=1278290,fd=27))
ESTAB    0        0               10.0.33.7:40124        1.76.163.187:18080    users:(("monerod",pid=1278290,fd=22))
ESTAB    0        0               10.0.33.7:45540          1.47.222.82:18080    users:(("monerod",pid=1278290,fd=33))
ESTAB    0        0               10.0.33.7:54586        1.105.90.107:18080    users:(("monerod",pid=1278290,fd=30))
ESTAB    0        0               10.0.33.7:37150        1.109.206.150:18080    users:(("monerod",pid=1278290,fd=32))
ESTAB    0        0               10.0.33.7:37474          1.247.61.57:18080    users:(("monerod",pid=1278290,fd=37))
ESTAB    0        0               10.0.33.7:46602        1.69.245.223:18080    users:(("monerod",pid=1278290,fd=28))
ESTAB    0        0               10.0.33.7:52392           1.36.20.21:18080    users:(("monerod",pid=1278290,fd=35))
ESTAB    0        0               10.0.33.7:48160        1.119.153.32:18080    users:(("monerod",pid=1278290,fd=36))
ESTAB    0        0               10.0.33.7:18081             10.0.33.5:59656    users:(("monerod",pid=1278290,fd=38))

This command is listing any open ports or connections to or from the server on TCP ports containing ‘1808’. Let’s go through them. I’ve added line numbering for reference and IP addresses have been changed to protect the innocent.

• Lines 2-3 show the server is listening for new inbound connections over all interfaces on ports 18080 and 18081 as we expected (mainnet RPC and P2P respectively).
• Line 4 shows it’s also listening for local connections on a port we haven’t discussed yet – 18082. We’ll cover this in the next section.
• Lines 5-16 lists twelve outbound connections on port 18080. These were initiated by my daemon to other monero daemons, also referred to as peers or nodes. This is how my node is communicating with the monero network, getting new blocks etc. Each of these nodes will connect to twelve or more other nodes and so on.
• Finally, line 17 is an inbound connection on port 18081. This happens to be is from a wallet on another system in my network.

Notice there are no inbound connection to port 18080, even though my server is listening. That’s because my host and network firewalls don’t allow that. Essentially that makes my node a child node of the monero network. I am relying on other peers for my information, but I’m not providing the same service back to other peers that would like to get synced.

Allowing Connections from Other Peers

Is this a problem? Well, ideally I would be contributing more fully to the network by allowing inbound peers as well, but for security reasons I have chosen not to in this case. If you do want to allow peering connections, you can follow similar steps to enable this port in firewalld (as well as any network firewall/router you have). There is no authentication for P2P connections because of course these could be from anywhere on the internet. But the number is limited for each node so that connections do not become concentrated on only a few nodes, so the overhead is not large. And no private information is available over the P2P channel so it’s actually not as essential to lock it down, as long as you trust the monero code itself.

Now since my firewalld is blocking the connection on port 18080, I don’t really care that the server is still listening on that port – nothing will get in either way. But if you do want to lock it down further, you can tell your daemon to only listen on localhost as follows:

p2p-bind-ip=127.0.0.1

ZMQ Protocol

Remember that line 4 was showing our system listening on the loopback interface on port 18082. Well, apparently this is an experimental or unfinished feature in monero. It’s not causing too much harm listening on localhost, but it can be disabled altogether with the following option:

no-zmq

Verify Your Binaries

For the very security conscious you may want to verify the monero binaries you’ve download. There’s a good guide on how to do this on the official Monero Site.

Monero Security Summary

That’s about it for securing monerod. There is a very interesting but still experimental feature in the works called I2P, so we won’t cover that here. I may add another section on that as it develops. In summary:

• Secure your network with a firewall
• Secure your Linux hosts with firewalld (or iptables)
• Do use SELinux
• Restrict access to your daemon, in particular the RPC channel
• Only enable features that are actually needed or that contribute to the community!