Kubernetes: Fixed?

Learning from my mistakes last time, I decided not to experiment in production and instead spun up some VMs running Alpine Linux on my desktop to figure out Calico/Canal. I really need this to work as I have some experiments to run for work stuff, so god damn it this weekend it’s happening!

After some grief, I wrote myself an Ansible playbook that installs K8s onto an Alpine VM, spun up three of them and fired for effect. I then created a single master, two nodes, and had to write myself a service which I call kubefix which does the three things necessary to make Calico/Canal work on Alpine:

cat > /etc/init.d/kubefix 
#!/sbin/openrc-run

command="/dev/null"

depend() {
        before kubelet
}

start() {
        mount --make-shared /
        mount --make-shared /sys
        [ -e /usr/libexec/cni ] || ln -s /opt/cni/bin /usr/libexec/cni
        sleep 5
}

Then enable + start the service:

rc-update add kubefix default
service kubefix start

This makes / and /sys shared mounts, puts a symlink in place so that the CNI plugins can be found, then pauses for five seconds because for some reason if I don’t have this then kubelet starts early, dies, and has to manually be restarted.

The symlink seems to only be required once, but it’s easy enough to put in place so fuck it, rough enough is good enough. I’m not sure if this is the correct way to make one-shot services for Alpine, but again… rough enough.

And it seems to do the trick! I rebooted the machines, took the entire cluster down and started it again, and Canal “just worked”. Canal was important to me, because making Calico (which uses BIRD internally) work with MetalLB (which also wants BIRD) didn’t look like fun, and I never had a problem with Flannel’s VXLAN stuff doing what I wanted, I just wanted NetworkPolicy support!

So now I have that, I can move on with life… it would be nice to figure out if there’s a permanent way to fix the mounts, but if there is I’m too dumb to figure it out.

Interestingly, I don’t appear to have needed any changes for Canal to work on Ubuntu? I’m not sure if it will stay that way or not when I do a fresh install one of these days.

Horsham, VIC, Australia

Author:

fwaggle

Published:


Modified:


Filed under:


Location:

Horsham, VIC, Australia

Navigation: Older Entry Newer Entry