Technology

OKD Docker private Registry on NFS mount

If you use OKD/OpenShift then most probably you also run internal and private Docker registry for your builds. Cluster uses this to lookup for containers images for further deployment. For basic, default installation your Docker Registry is located in a project called default. It also uses quasi permanent storage which lasts until next redeployment of registry container (pod). There is however a possiblity to mount a NFS volume in the registry deployment configuration so your images which have been pushed onto the registry will not go away in case you need to redeploy registry itself. This need might come if you run certificates redeploy Ansible playbook. If you review this playbook you are going to see a step in which there is a registry redeploment so you need to have permenent storage in your registry in such a scenario.

First install NFS server on separate machine and configurate directory and /etc/exports. After that restart NFS server (service nfs-server restart).

/opt/PVdirectory		*(rw,root_squash,no_wdelay)

Next you need to create PV (which stands for persistant volume) configuration in OKD master:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: PVname
spec:
  capacity:
    storage: 10Gi
  accessModes:
  - ReadWriteOnce
  nfs:
    path: /opt/PVdirectory
    server: 192.168.1.2
  persistentVolumeReclaimPolicy: Recycle

Apply this configuration:

oc create -f filename.yaml

You just created PV definiton which tell OKD to look for NFS volume at 192.168.1.2 at /opt/PVdirectory which 10GiB of space which will be recycled if unbound. Next you need to copy you current registry contents, which is Docker images. There is no scp to copy files, but first pack them with tar:

cd /registry
tar -cf docker.tar docker

Now go to the master, locate your docker-registry container name (replace abcdefg with proper ID):

oc rsync docker-registry-abcdefg:/registry/docker.tar .

Move archive file to your NFS server and unpack it there. Main folder you have owner as nfsnobody but internal contents same as original:

sudo chown -R 1000000000:1000000000 docker

Now go to OKD webconsole bring registry down (scale to 0 pods). Go to deployment configuration and remove default storage and add in place your passing /registry as path for it. Bring registry online and test it. Now it should use NFS mount and you are free to redeploy your registry if you need.