home-server

This is taking a bit of a turn.

History

Incase it matters

1. Started as VMs
2. Ended up becoming docker containers
3. Ended up becoming locally defined services
4. Back to containers
5. Kubernetes, flux, from gitlab
6. Kubernetes, argocd, from repobase
7. Attempts to make more public

We're up to this final step. Prior to now, the initialization was to define a bunch of initial secrets and namespaces, run the argocd install, wait for some deployments and then run some more custom manifests.

I'm trying to:

1. Remove manually defined secrets, allowing for repeatable deployments incase it helps people
2. Make the system initialization more streamlined

Installing

You need to install K3S. I do this using nixos

You then need to run the following:

curl https://repobase.net/j/home-server/archive/main:kustomize.tar.gz | tar zxvfs -
kubectl apply -f home-server/manual/
kubectl apply -k home-server/

Manual currently has two namespaces: argocd for installing argo into and secrets for applying manual secrets to.

Currently, I'm only thinking of using the 'secrets' namespace to store OpenBaos token so that ESO can access it.

The kustomize will install argocd into the argocd namespace. It will then bootstrap to the deploy directory in this repository, and argocd will take the wheel from there.

There's room to improve this process. I've just spent too much time getting to this point.

Change ArgoCD Admin Password

The node port seems to be random? It's fine you can find it via the argocd-server service.

To change the password, you can use the following:

% HTPASSWORD=$(htpasswd -bnBC 10 "" "somepass" | tr -d ':\n')
% kubectl -n argocd patch secret argocd-secret -p "{\"stringData\":{\"admin.password\":\"$HTPASSWORD\"}}"

You can just use the initial admin secret I guess. I've done that at home. I just find it easier to use a generic, memorable password for this.

General Notes

Installing more complex helm charts - such as external secrets - that require cluster scoped CRDs is a massive pain in the ass. ArgoCD will throw up errors during subsequent syncs such as:

one or more objects failed to apply, reason: roles.rbac.authorization.k8s.io "secrets-external-secrets-leaderelection" already exists ...

The most annoying part is that I knew this would happen. I've been through this before, but I still have that dropkick from Xork that kept saying "just use a dependency". Just make it a dependency. It just works. Just make it a depedency.

Fuck that guy. Gas lighting drop kick.

I can't seem to make cluster scoped CRDs install cleanly. I've already spent too many days fucking around with this deployment so I'm noting this for the future.

When using something like ArgoCD, do not use Helm Charts to define your clusters configuration.

Helm charts define application deployments. Cluster configuration is NOT an application deployment.

Fuck me that cunt will read this and act like he knows shit. Joy.

Anyway.

OpenBao

I tried to isntall OpenBao but I couldn't get it to become reliable. Any major changed nuked it out and it would rebuild and require re-init. Very annoying.

I tried to use HA but it required consul for some reason.

Yeeted.

External Secrets Store

This is a huge issue. I had issues at work and I never got to fix it, just hacked around it. I tried to rewrite the crds but one of them is just way too large.

one or more objects failed to apply, reason: error when patching "/dev/shm/1495094583": CustomResourceDefinition.apiextensions.k8s.io "clustersecretstores.external-secrets.io" is invalid: metadata.annotations: Too long: may not be more than 262144 bytes,error when patching "/dev/shm/60857711": CustomResourceDefinition.apiextensions.k8s.io "secretstores.external-secrets.io" is invalid: metadata.annotations: Too long: may not be more than 262144 bytes,error when patching "/dev/shm/1929213987": Internal error occurred: failed calling webhook "validate.clustersecretstore.external-secrets.io": failed to call webhook: Post "https://external-secrets-webhook.secrets.svc:443/validate-external-secrets-io-v1-clustersecretstore?timeout=5s": no endpoints available for service "external-secrets-webhook"

Server side or client side it doesn't matter. Nothing seems to fix this. The CRD is just way too large and we can't get around it.

So... fuck it. I'm installing with helm.