/ FreeNAS

Setting up RancherOS in Freenas 11 RC4 with Rancher

This will be a very simple write-up to setup RancherOS with networking/ mounting of NFS storage and installing the rancher-server container + agent.

Setting up the VM using iohyve:
  • open a ssh connection to your FreeNAS and acquire sudo permissions with sudo su

  • setting up the iohyve enviroment in FreeNAS 9.10 RC4
    iohyve setup pool=ssd kmod=1 net=igb0
    ssd refers to the zpool and igb0 to the NIC of your choice. FreeNAS will create datasets on your zpool for me it created /mnt/ssd/iohyve

  • fetch the RancherOS ISO file with
    iohyve fetch https://github.com/rancher/os/releases/download/v0.6.1/rancheros.iso i use 0.6.1 due to the familiarity of grub also RancherOS will automatically update

  • type:
    iohyve create RancherOS 26G which creates a new 26GiB zpool under /mnt/ssd/iohyve/RancherOS
    iohyve set RancherOS loader=grub-bhyve ram=16G cpu=8 con=nmdm0 os=debian
    iohyve install RancherOS rancheros.iso

  • open a new shell
    sudo iohyve list
    sudo iohyve start RancherOS
    sudo iohyve console RancherOS

  • once you connected to the OS you'll be greeted by the bootloader grub. type
    grub> set root=(cd0,msdos1)
    grub> linux /boot/vmlinuz* ro rancher.password=rancher
    grub> initrd /boot/initrd*
    grub> boot

  • RancherOS will boot and you can login with username rancher and password rancher. [rancher@rancher ~]$ should appear.

  • creating a cloud config. Type:
    vi cloud-config.yml and press i for insert mode and paste the following.

#cloud-config
ssh_authorized_keys:
  - ssh-rsa AAAAB----ZZZ generated-openssh-key-use-putty-or-equivalent  

#/var/lib/rancher/conf/cloud-config.d/netconfig-config.yml
rancher:
  network:
    interfaces:
      eth0:
        addresses:
          - 192.168.0.16/24
        gateway: 192.168.0.1
        dhcp: false
    dns:
      nameservers:
        - 8.8.8.8
        - 8.8.4.4
  • hit ESC and type :x! to save your cloud-config.yml file
  • validate your cloud-config.yml file sudo ros config validate -i cloud-config.yml you should not receive any error messages, if you do, fix them.
  • finalize and install RancherOS with sudo ros install -c cloud-config.yml -d /dev/sda
  • RancherOS will ask you to restart/shutdown. But it will not reboot.
  • Open a new shell and navigate to /mnt/iohyve/RancherOS
  • create a boot file with sudo nano grub.cfg and paste
set root=(hd0,1) 
linux /boot/vmlinuz* printk.devkmsg=on rancher.state.dev=LABEL=RANCHER_STATE rancher.state.wait console=tty0 ro rancher.password=rancher
initrd /boot/initrd*
boot
  • leave the part ro rancher.password=rancher in for the time in case your ssh-rsa key will be rejected, if you can successfully logon to rancher using your SSH key just remove this part from the grub.cfg.

  • after you saved your grub.cfg enter the following into your shell.
    sudo iohyve set RancherOS os=custom
    sudo iohyve set RancherOS boot=1

  • add tuneables throught your FreeNAS GUI

  • start RancherOS with iohyve start RancherOS and connect to it.

  • connect via ssh and ssh-rsa key to RancherOS.

NFS shares and persistent storage.

I used the FreeNAS GUI to create 2 shares of data-sets (mapall->root and mapallgroup -> wheel), namely nfs://192.168.0.2/mnt/zpool/dbs and nfs://192.168.0.2/mnt/zpool/docker

  • in rancher shell acquire sudo permissions and navigate to **/var/lib/rancher/conf/cloud-config.d/ **
  • create nfs.yml and paste:
#/var/lib/rancher/conf/cloud-config.d/nfs.yml
write_files:
  - path: /etc/rc.local
    permissions: "0755"
    content: |
      #!/bin/bash
      [ ! -e /usr/bin/docker ] && ln -s /usr/bin/docker.dist /usr/bin/docker

rancher:
  services:
    nfs:
      image: d3fk/nfs-client
      labels:
        io.rancher.os.after: console, preload-user-images
        io.rancher.os.scope: system
      net: host
      privileged: true
      restart: always
      volumes:
        - /usr/bin/iptables:/sbin/iptables:ro
        - /mnt/docker:/mnt/docker:shared
        - /mnt/dbs:/mnt/dbs:shared
      environment:
        SERVER: 192.168.0.2
        SHARE: /mnt/zpool/docker
        MOUNTPOINT: /mnt/docker

#cloud-config
mounts:
  - ["192.168.0.2:/mnt/zpool/dbs", "/mnt/dbs", "nfs", ""]

The nfs.yml will install the nfs-client container will create mountpoints /mnt/docker and /mnt/dbs and mount them to the nfs shares before user-images are loaded. Reboot and ssh into rancher navigate to /mnt/docker and check if its correctly mounted. (I used Mapall:root for both folders so docker containers can utilize chown, if needed. This is of course a security risk)

Setting up Rancher-Server
  • install the rancher-server docker container with a docker run command.
    sudo docker run -d --restart=always --network=rancherlocal --name=rancherserver -v /mnt/docker/rancher/rancher-server-mysql:/var/lib/mysql --restart=always -p 8080:8080 rancher/server:v1.0.2
    or in with a rancher-compose file:
#/var/lib/rancher/conf/cloud-config.d/rancher-server.yml
rancher:
  services:
    rancherserver:
      image: rancher/server
      container_name: "rancher-server"
      environment:
        PGID: '1101'
        PUID: '1101'
        TZ: Europe/Berlin
      labels:
        io.rancher.os.after: console, preload-user-images
      privileged: true
      restart: always
      ports:
        - "8080:8080"
      volumes:
        - /var/run/docker.sock:/var/run/docker.sock
        - /mnt/docker/rancher/rancher-server-mysql:/var/lib/mysql
  • use a webbrowser and navigate to your http://rancherosip:8080 and click on admin and set up your prefered authentication method.

Navigate to Infrastructure -> Hosts -> Add Hosts and copy the docker run command.

Run the docker run command that you copied on the same machine and a new host will apear in the menu.
Done. Have Fun managing the Docker host via the Rancher Webtools