Migrating nextcloud to a new server

Nextcloud?

I’ve been using a self-hosted file sharing service for the past three years in order to move away from large commercial parties in this space. I started using owncloud in the summer of 2016, but switched to nextcloud later that year as Frank Karlitschek, the founder, left owncloud. The open source, large community and the self-hosted aspects of nextcloud are the biggest draws to the project for me. Hosting my own instance means I get to indulge in some sysops, one of my pastimes. You can find more details on the self-hosted aspect in the ‘Self-hosted’ section at bottom of this post. In short, I decided to switch to a new offering of my VPS provider which meant migrating nextcloud to a new server.

Migrating nextcloud 15

I neglected to detail the installation process of my owncloud instance back in 2016, which meant figuring some stuff out during the migration. Wanting to avoid repeating this mistake, I decided to detail the migration process for my future self in this post. Environment-wise both servers aren’t too different: the OS has been updated from Ubuntu 16.04 to 18.04 which means newer versions for most packages. Additionally, I decided to move away from mysql to its open source fork mariadb. Given mariadb is a drop-in replacement for mysql, the process went smoothly with the exception on a pesky apparmor bug (which only occurs when you install mariadb after installing and removing mysql) detailed here and here.

For the actual migration I followed the NC documentation. Make sure to read it thoroughly in order to get a sense of all the steps involved. I’ve added my notes to each of the steps below.

Step1: setup new machine

Using the Hetzner cloud console, I setup a Ubuntu 18.04 machine at their data center in Nuremberg. The server was up and running within 10 seconds after creation. If you forgot to add an SSH key, you can SSH via the credentials that you received via email. When you login for the first time, you will be forced to change the root password. On this new server, I setup SSH/UFW/NGINX/MariaDB and PHP via the following steps:

OpenSSH

• Ensure that SSH key is present in authorized_keys:
  • vim /root/.ssh/authorized_keys
• Disable password auth, only allowing key-based aut: Important: don’t disable this if you want to login via username/password!
  • vim /etc/ssh/sshd_config
  • Set PasswordAuthenication no
• Restart SSH:
  • systemctl restart sshd.service
• Allow SSH and enable firewall:
  • ufw allow OpenSSH
  • ufw enable

Nginx

• Install Nginx:
  • apt install nginx
  • ufw allow 'Nginx Full'

MariaDB

• Install, setup and test MariaDB:
  • apt install mariadb-server
  • mysql_secure_installation
  • mysqladmin -p -u root version

PHP-fpm

• Install PHP-fpm
  • apt install php-fpm php-mysql
  • vim /etc/nginx/sites-available/default
    • add index.php to index
    • Enable PHP: search for fastcgi_pass line in location ~ \.php {} Line should state fastcgi_pass unix:/var/run/php/php7.2-fpm.sock;
• Ensure there are no errors in the config: nginx -t
• systemctl reload nginx.service

Step2: stop nextcloud

• Set nextcloud to maintenance mode on the old server:
  • cd /var/www/nextcloud
  • sudo -u www-data php occ maintenance:mode --on
• Stop serving nextcloud on the old server:
  • rm /etc/nginx/sites-enabled/nc.vdna.be
  • systemctl reload nginx.service

Step3: copy database

• Create a database dump on the old server and copy it to the new server:
  • mysqldump --add-drop-table -h localhost -u nextcloud -p nextcloud > nextcloud.sql
  • scp nextcloud.sql root@116.203.98.100:/root/mysql-exports
• Create a database and a user (replace xxx with password) on the new server:
  • mysql
  • CREATE DATABASE nextcloud;
  • CREATE USER 'nextcloud' IDENTIFIED BY 'xxx';
  • GRANT USAGE ON *.* TO 'nextcloud'@localhost IDENTIFIED BY 'xxx';
  • GRANT ALL privileges ON 'nextcloud'.* TO 'nextcloud'@localhost;
  • FLUSH PRIVILEGES;
  • SHOW GRANTS FOR 'nextcloud'@'localhost';
• Import the nextcloud database on the new server:
  • mysql -u nextcloud -p -h localhost nextcloud < /root/mysql-exports/nextcloud.sql

Step4: copy files

• On the old server, run rsync with the a flag in order to preserve meta data while copying files (the z flag adds compression). Note this can take a while:
  • rsync -az /var/www/nextcloud/ root@116.203.98.100:/var/www/nextcloud
• Copy the nginx config file from the old server to the new server:
  • scp /etc/nginx/sites-available/nc.vdna.be root@116.201.98.100:/etc/nginx/sites-available/

Step5: configure nextcloud and nginx

Nextcloud

• Install any missing dependencies of NC on the new server. This will depend on your setup, but I had to install the following dependencies:
  • apt install php7.2-gd php7.2-json php7.2-mysql php7.2-curl php7.2-mbstring
  • apt install php7.2-intl php-imagick php7.2-xml php7.2-zip
• On the new server, update the NC config file with the new database user:
  • vim /var/www/nextcloud/config/config.php
  • set dbname, dbhost, dbuser and dbpassword

Nginx

• Update nginx config file on the new server:
  • Update root path (if necessary)
  • Temporarily change server_name for testing
  • Set the correct php-fpm path
  • Set the correct ssl certificate paths. I first copied the TLS certs from the old server and installed letsencrypt after NC was up and running.
• Activate nginx config on the new server:
  • ln -s /etc/nginx/sites-available/nc.vdna.be /etc/nginx/sites-enabled/nc.vdna.be
  • nginx -t
  • systemctl reload nginx.service

Testing

Test nextcloud operation by pointing your browser to the server_name of the new nextcloud instance. You should see the maintenance page. If you see an internal server error, then something is likely amiss with your configuration. Check nextcloud and nginx log files to fix the issue. My internal server errors were solved by installing missing dependencies.

Once you are sure that NC is working properly you can disable maintenance mode on the new server:
sudo -u www-data php occ maintenance:mode --off
Now, login as admin and confirm normal function of Nextcloud.

At this point, you should check the security warnings in the NC admin panel. I had to fix the following issues on the new server:

• Environment variables for php-fpm
• Enable PHP Opcode cache
• Install cron job for www-data user

It is advisable to skim through the Nextcloud install docs and fix anything you missed.

Step6: Update DNS entry

Now that NC is working properly, point the DNS entry of nextcloud to your new server. Don’t forget to undo the change to server_name in the nginx config file. Clients should pickup on the new server once their DNS caches are expired.

Note that you might want to setup letsencrypt at this point.

Self-hosted

There are many commercial nextcloud providers, but I decided to host my own instance. While it is possible to host at home, I’ve always ran my instance on an inexpensive CX10 instance at Hetzner.de. At € 4,72 per month Hetzner offered a VPS with 1 virtual CPU core, 1GiB of ram and 25GB of disk space. At just 2.7GB of disk space for my NC instance (I am a light user), there is still plenty of free disk space for other services.

However, I noticed that 1GiB of RAM can be a limiting factor when sharing a photo album with large JPEG images (24MP). Specifically, when NC generates image thumbnails for the album then PHP spool ups five fpm worker processes which quickly exhausts RAM causing the machine to swap and become unresponsive. While this can be alleviated by downsizing the php fpm worker pool, I read that Hetzner offers a new cloud VPS service with double the amount of RAM at a lower price.

Hetzner Cloud

For a monthly fee of € 3,01 the CX11 offers 1vCPU, 2GiB RAM, 20GB disk space and 20TB of traffic. Additionally, Hetzner now offers flexible Block Storage Volumes at € 0.0484/GB per month. For my usage the 5GB reduction in disk space isn’t a downside at the moment. Given the low cost of block storage, the €1,7 price difference actually buys you 35GB of storage.

	vCPU	RAM	Disk space	Extra storage	Traffic	Price
CX10	1	1 GiB	25 GB	Unknown	2 TB	€4,72
CX11	1	2 GiB	20 G	€ 0.0484/GB	20 TB	€3,01

Hetzner’s new CX11 doubles the amount of RAM at a lower price.