Skip to content

Security and auditing

Wondering what's encrypted, what gets logged, and whether an attacker can erase their tracks?

This article describes what Core encrypts at rest, what a node records when someone logs in over SSH, what Core records about every change, and how to limit API access to specific IP addresses.

Encryption at rest

"At rest" means the data as it sits on disk, when nobody is reading it.

Databases

MariaDB databases can be encrypted at rest. This is a cluster-level setting called 'MariaDB encryption keys'. Once it's on, every InnoDB table on the cluster is encrypted on disk. Existing tables are encrypted in a background pass, and new ones are encrypted automatically.

Nothing changes in how you use the database. Queries, mysqldump dumps, restores and phpMyAdmin all see the same content as before. See All about databases for the performance impact and how to check progress.

PostgreSQL databases are not encrypted at rest.

Files

Files on the filesystem, so your website files, uploads, logs and mailboxes, are not encrypted at rest. The nodes don't use full-disk encryption.

Every node is a virtual machine, and its files live on physical disks in a data centre. Full-disk encryption protects against one scenario: someone gets hold of those disks, or of the virtual machine's disk image, and reads the files directly instead of logging in to the node. To do that they'd have to break into the data centre and physically carry out the disks, or get access to the virtualisation platform the node runs on. Data is spread across multiple disks, so taking one disk gets you fragments rather than a usable copy of anyone's files. You'd need all of them. Only a small number of people can get into the data centre at all.

What full-disk encryption doesn't protect against is much more common. While a node is running, its disks are unlocked and the filesystem is mounted, because that's the only way the web server can read the files it serves. Anyone who logs in to the node reads exactly the same files whether the disk is encrypted or not. That includes someone using a leaked SSH key or password, and someone who got in through a vulnerability in a website. Those are the realistic ways files get stolen, and encryption at rest does nothing about them.

Encryption also costs performance. Encrypting and decrypting happens on every read and write, so every file operation uses extra CPU and sites get slower across the board. You pay that continuously, in exchange for protection against a data-centre break-in. Shared hosting nodes aren't usually encrypted for this reason.

If you have data that has to be encrypted on disk, for example because a compliance rule says so, encrypt it in your application before you write it, or keep it in MariaDB on a cluster that has encryption enabled.

SSH logins on a node

What's recorded

The node records every SSH login to it: the time, the UNIX user that was logged in to, the IP address the connection came from, and, for logins with an SSH key, the fingerprint of the key that was used.

The fingerprint is what lets you trace a login back to a person. If everyone has their own SSH key, the fingerprint in the log says exactly whose key was used. If several people share one key, or log in with a password, the log only shows that someone logged in as that UNIX user.

Give every person their own SSH key and never share one. See All about UNIX users for adding keys.

What a normal user can and can't erase

Commands typed in an interactive shell go into that user's shell history file, usually ~/.bash_history. The file belongs to the UNIX user, so the user can empty it, and an attacker who gets into a UNIX user account usually does.

Clearing the shell history doesn't remove the login record. The system writes that record outside the UNIX user's own files, so a normal UNIX user can't change it. Even after a history wipe, you can still see that the account was logged in to, from which IP address, and with which key.

The limit: root access

Anyone with root access on the node can edit or delete any file on it, including the system log files that hold the login records. So the node's own logs are reliable evidence about what a normal UNIX user did, but not about someone who managed to become root.

What we're working on

We're looking into central log aggregation. The node would send its log lines to a separate system as it writes them, so a copy exists somewhere the node has no control over. Someone with root on the node could still wipe the local copy, but the copy on the log system would survive. This isn't available yet.

What changed in Core, and who changed it

Core keeps two separate records. Both are written on Cyberfusion's own infrastructure, not on your cluster.

Object logs: what changed

Core records every create, update and delete of an object as an object log. Each entry holds:

  • Which object changed, and what kind of object it is.
  • Whether it was a create, an update or a delete.
  • For updates, the old and the new value of every field that changed.
  • Which API user made the change.
  • Which request it belonged to.

Before storing an entry, Core replaces the values of sensitive fields with ********. That covers passwords, private keys, tokens and passphrases, so the log never holds a usable secret.

Core records changes the same way whether you made them in the interface or by calling the API directly.

Request logs: who called what

Core records every call to the Core API as a request log: the time, the API user, the HTTP method and path, the source IP address, and the request and correlation IDs that tie related calls together.

Neither can be deleted

Nobody can delete an object log or a request log. There's no way to do it in the interface and no way to do it through the API, for any user, including you. Someone who takes over your Core account can make changes, but can't remove the record of having made them.

Core keeps both for 45 days and then removes them automatically.

Getting at them

Object logs are available through the Core API at /api/v1/object-logs, filtered to your own objects.

Request logs aren't shown in the interface for customer accounts. Ask Cyberfusion support if you need them, for example while investigating an incident.

Restricting Core API access to specific IP networks

An API user signs in with a username and password and gets back an access token that's valid for 30 minutes.

You can also restrict an API user to one or more IP networks. Core then refuses any sign-in attempt from another address with a 403, so a correct password on its own isn't enough. This is useful for API users that your own automation signs in as, since those normally connect from a fixed set of addresses.

This isn't a self-service setting. Ask Cyberfusion support to set the allowed IP networks for an API user.