I'm working on a custom Kubernetes Operator in Golang, as a shift away from an Operator generated based on existing Helm Charts. The way you can create a Pod (or Job in the example I will share) is like this, as I understand it:
func returnJob(cr *myApi.MyCRObject) *batchv1.Job {
return &batchv1.Job{
ObjectMeta: metav1.ObjectMeta{
Name: cr.Name,
Namespace: cr.Namespace,
},
Spec: batchv1.JobSpec{
BackOffLimit: 0,
Template: apiv1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Name: cr.Name
},
Spec: apiv1.PodSpec{
RestartPolicy: "Never",
ServiceAccountName: "",
SecurityContext: apiv1.SecurityContext{
RunAsUser: "",
FsGroup: ""
},
ImagePullSecrets: apiv1.ImagePullSecrets{
Name: ""
},
Volumes: []apiv1.Volumes{
{
Name: "config",
ConfigMap: apiv1.ConfigMap{
Name: cr.Name + "-config"
}
}
},
Containers: []apiv1.Container{
{
Name: "container",
Image: "prefix/image:tag",
ImagePullPolicy: "Always",
WorkingDir: "installdir",
SecurityContext: apiv1.SecurityContext{
RunAsUser: "",
FsGroup: ""
},
Env: []apiv1.Env{
{
Name: "KEY",
Value: "VAL"
},
{
Name: "NAME",
Value: cr.Name
}
}
Command: []string{
"mycommand",
"-x"
},
Resources: apiv1.Resources{
Requests: apiv1.Requests{
Cpu: 1,
Memory: "1G"
},
Limits: apiv1.Limits{
Cpu: 1,
Memory: "1G"
}
}
}
}
}
}
}
}
}
Then elsewhere in the Reconcile call, the job is actually created on Kubernetes:
job := returnJob(instance)
// Set instance as the owner and controller
if err := controllerutil.SetControllerReference(instance, job, r.scheme); err != nil {
return reconcile.Result{}, err
}
// Check if this Job already exists
found := &batchv1.Job{}
err = r.client.Get(context.TODO(), types.NamespacedName{Name: job.Name, Namespace: job.Namespace}, found)
if err != nil && errors.IsNotFound(err) {
reqLogger.Info("Creating a new Job", "Job.Namespace", job.Namespace, "Job.Name", job.Name)
err = r.client.Create(context.TODO(), job)
if err != nil {
return reconcile.Result{}, err
}
// Job created successfully - don't requeue
return reconcile.Result{}, nil
} else if err != nil {
return reconcile.Result{}, err
}
This is... very clearly not optimal. I can't see a great way to fill in things such as the "Image" or to allow overrides for "Resources" etc., and that's all second to it being a massive hardcoded blob (I removed identifying info and about 120 lines from the specification above, things like additional volumes and other containers).
I'd like to just be able to pull from an existing YAML file in the manifest (what was previously used with Helm Charts and Helm Operators) and return that in the format Go needs from a method such as this.
I've been scouring the internet for something like this, but there just aren't a lot of Go or k8s Operators resources out there yet, so I figured I'd ask here.
Any ideas or solutions you guys have found?
