# Issues with okteto-api reading wildcard SSL cert **URL:** \ **Category:** Okteto Self-Hosted\ **Created:** [November 8, 2022, 2:10pm UTC](https://community.okteto.com/t/issues-with-okteto-api-reading-wildcard-ssl-cert/325 "2022-11-08T14:10:58Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![pmarks](https://avatars.discourse-cdn.com/v4/letter/p/ecccb3/32.png) [@pmarks](https://community.okteto.com/u/pmarks)\ **Post date:** [November 8, 2022, 2:10pm UTC](https://community.okteto.com/t/issues-with-okteto-api-reading-wildcard-ssl-cert/325/1 "2022-11-08T14:10:58Z")
Hello everyone! I have been playing with Okteto for a few days along with a coworker since meeting the team at KubeCon, and so far we like what we see, but are trying to pass a roadblock on setting up our product inside Okteto. While attempting to enable SSL, we are unable to get the okteto-api deployment to recognize the cert. I have been following the instructions found [here](https://www.okteto.com/docs/self-hosted/administration/certificates/) and am attempting to deploy a letsencrypt cert that has already been validated and is ready to use. I have attempted deploying the cert and the Helm chart a few times to recheck my steps and here is where I am so far: Starting with a predeployed Okteto deployment with a self-signed cert (the okteto-api won’t start otherwise), I validate the cert locally with openssl and create the standard tls secret: ```auto $ ls cert1.pem chain1.pem fullchain1.pem privkey1.pem $ openssl x509 -noout -subject -dates -ext subjectAltName -in ./cert1.pem subject=CN = *.dev-prism. notBefore=Nov 3 17:20:04 2022 GMT notAfter=Feb 1 17:20:03 2023 GMT X509v3 Subject Alternative Name: DNS:*.dev-prism., DNS:dev-prism. $ k create secret tls dev-prism-cert --key ./privkey1.pem --cert ./cert1.pem --namespace okteto secret/dev-prism-cert created $ k describe secret dev-prism-cert Name: dev-prism-cert Namespace: okteto Labels: Annotations: Type: kubernetes.io/tls Data ==== tls.key: 1704 bytes tls.crt: 1911 bytes ``` I then updated the Helm chart values files to include the following: ```auto wildcardCertificate: create: false name: dev-prism-cert ingress-nginx: controller: extraArgs: default-ssl-certificate: "okteto/dev-prism-cert" ``` Redeploy was successful: ```auto $ helm upgrade okteto okteto/okteto -f dev-utils/helm/postoffice/dev-prism/okteto-values.yml -n okteto Release "okteto" has been upgraded. Happy Helming! NAME: okteto LAST DEPLOYED: Mon Nov 7 12:38:35 2022 NAMESPACE: okteto STATUS: deployed REVISION: 2 TEST SUITE: None NOTES: ``` Inspecting the cluster shows the api rolling release failing: ```auto $ k get po NAME READY STATUS RESTARTS AGE okteto-api-596c98c444-46586 0/1 CrashLoopBackOff 3 (38s ago) 88s okteto-api-767f6c5f74-bnnb9 1/1 Running 0 2d21h okteto-autoscaler-5b5f786749-7bfjm 1/1 Running 0 88s okteto-buildkit-5b21767586-0 1/1 Running 0 57s okteto-daemon-ltgmn 1/1 Running 0 89s okteto-daemon-wxtf6 1/1 Running 0 89s okteto-daemon-zsqrp 1/1 Running 0 89s okteto-destroy-all-checker-27797373-zg6gx 0/1 Completed 0 7m11s okteto-destroy-all-checker-27797376-rk67f 0/1 Completed 0 4m11s okteto-destroy-all-checker-27797379-hhp5t 0/1 Completed 0 71s okteto-frontend-67cfff89cf-4qmzw 1/1 Running 0 2d21h okteto-frontend-67cfff89cf-mw99l 1/1 Running 0 2d21h okteto-ingress-nginx-controller-797cfb4b66-h7ls6 1/1 Running 0 68s okteto-ingress-nginx-controller-797cfb4b66-m2m7x 1/1 Running 0 89s okteto-ingress-nginx-defaultbackend-c54f6c679-6p74m 1/1 Running 0 2d21h okteto-ingress-nginx-defaultbackend-c54f6c679-6txbm 1/1 Running 0 2d21h okteto-installer-checker-27797370-fc6jw 0/1 Completed 0 10m okteto-installer-checker-27797375-zrjcx 0/1 Completed 0 5m11s okteto-installer-checker-27797380-6vghq 0/1 Completed 0 11s okteto-migration-a2643dd5b759268e-ljg6s 0/1 Error 0 88s okteto-migration-a2643dd5b759268e-mjhtf 0/1 Error 0 83s okteto-migration-a2643dd5b759268e-t2g2v 0/1 Completed 0 73s okteto-mutation-webhook-74cfb97cbf-8j2m2 1/1 Running 0 88s okteto-mutation-webhook-74cfb97cbf-c74sw 1/1 Running 0 77s okteto-registry-5b4c9d89bb-x5cjc 1/1 Running 0 88s okteto-reloader-754dd844b6-82lm9 1/1 Running 1 2d21h okteto-telemetry-27793440-8w5sw 0/1 Completed 0 2d17h okteto-telemetry-27794880-g9r59 0/1 Completed 0 41h okteto-telemetry-27796320-n7kdc 0/1 Completed 0 17h ``` The logs show that it is unable to read the cert as a valid: ```auto $ k logs okteto-api-596c98c444-46586 {"timestamp":"2022-11-07T17:40:25Z","message":"Starting api service...","severity":"INFO","context":{"data":{"instance.id":"okteto-api-596c98c444-46586"}}} {"timestamp":"2022-11-07T17:40:25Z","message":"using no-op tracer","severity":"INFO","context":{"data":{"instance.id":"okteto-api-596c98c444-46586"}}} {"timestamp":"2022-11-07T17:40:25Z","message":"loading buildkit certificate","severity":"INFO","context":{"data":{"instance.id":"okteto-api-596c98c444-46586"}}} {"timestamp":"2022-11-07T17:40:25Z","serviceContext":{"service":"okteto-enterprise-backend"},"message":"couldn't find a valid cert: unable to decode private key PEM: unable to get PrivateKey from PEM type: PRIVATE KEY","severity":"CRITICAL","context":{"data":{"instance.id":"okteto-api-596c98c444-46586"},"reportLocation":{"filePath":"github.com/okteto/app/backend/pkg/registry/registry.go","lineNumber":52,"functionName":"Init"}}} ``` However, the `ingress-nginx` deployment is able to read the cert successfully: ```auto $ k describe ingress okteto Name: okteto Labels: app.kubernetes.io/component=api app.kubernetes.io/instance=okteto app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=okteto app.kubernetes.io/part-of=okteto helm.sh/chart=okteto-1.0.0 Namespace: okteto Address: .us-east-1.elb.amazonaws.com Default backend: default-http-backend:80 () TLS: dev-prism-cert terminates okteto.dev-prism. ``` The configuration shows it is using the cert with the `default-ssl-certificate` arg: ```auto $ k describe po okteto-ingress-nginx-controller-797cfb4b66-h7ls6 Name: okteto-ingress-nginx-controller-797cfb4b66-h7ls6 Namespace: okteto Priority: 0 Node: ip-10-0-1-173.ec2.internal/10.0.1.173 Start Time: Mon, 07 Nov 2022 12:39:03 -0500 Labels: app.kubernetes.io/component=controller app.kubernetes.io/instance=okteto app.kubernetes.io/name=ingress-nginx pod-template-hash=797cfb4b66 Annotations: kubernetes.io/psp: eks.privileged Status: Running IP: 10.0.1.210 IPs: IP: 10.0.1.210 Controlled By: ReplicaSet/okteto-ingress-nginx-controller-797cfb4b66 Containers: controller: Container ID: docker://87e64bff3947e4661bba09d1ad6924375d6f58190363fd423cef64dd6cb6d0aa Image: registry.k8s.io/ingress-nginx/controller:v1.3.0@sha256:d1707ca76d3b044ab8a28277a2466a02100ee9f58a86af1535a3edf9323ea1b5 Image ID: docker-pullable://registry.k8s.io/ingress-nginx/controller@sha256:d1707ca76d3b044ab8a28277a2466a02100ee9f58a86af1535a3edf9323ea1b5 Ports: 80/TCP, 443/TCP, 1234/TCP Host Ports: 0/TCP, 0/TCP, 0/TCP Args: /nginx-ingress-controller --default-backend-service=$(POD_NAMESPACE)/okteto-ingress-nginx-defaultbackend --publish-service=$(POD_NAMESPACE)/okteto-ingress-nginx-controller --election-id=ingress-controller-leader --controller-class=k8s.io/ingress-nginx --ingress-class=nginx --configmap=$(POD_NAMESPACE)/okteto-ingress-nginx-controller --tcp-services-configmap=$(POD_NAMESPACE)/okteto-ingress-nginx-tcp --default-ssl-certificate=okteto/dev-prism-cert --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services-configmap State: Running Started: Mon, 07 Nov 2022 12:39:04 -0500 Ready: True ``` And when I view the site, the insecure https splash page no longer appears and shows a valid cert: ![okteto-letsencrypt](https://us1.discourse-cdn.com/flex019/uploads/okteto/original/1X/81b2df96416ab64830cc12a18433ab917efe6035.png) I’m not sure what I am missing, as I think I followed all the steps correctly, but it’s possible I missed something in the documentation, or simply typo’d something. Any assistance would be greatly appreciated! Peter ---
**Author:** ![ramiro](https://sea1.discourse-cdn.com/flex019/user_avatar/community.okteto.com/ramiro/32/6_2.png) [@ramiro](https://community.okteto.com/u/ramiro)\ **Post date:** [November 8, 2022, 6:29pm UTC](https://community.okteto.com/t/issues-with-okteto-api-reading-wildcard-ssl-cert/325/2 "2022-11-08T18:29:49Z")
@pmarks how did you create the initial certificates? ---
**Author:** ![pmarks](https://avatars.discourse-cdn.com/v4/letter/p/ecccb3/32.png) [@pmarks](https://community.okteto.com/u/pmarks)\ **Post date:** [November 8, 2022, 6:54pm UTC](https://community.okteto.com/t/issues-with-okteto-api-reading-wildcard-ssl-cert/325/3 "2022-11-08T18:54:03Z")
Hi @ramiro! Since using cert-manager would require setting up cross-account IAM roles that I currently don’t have permissions to do, I opted to create the cert by hand using `certbot` and performed the HTTP-01 and DNS-01 challenge by hand. We will look to set up cert-manager for a more permanent solution with this to get us through our POC. ---
**Author:** ![ramiro](https://sea1.discourse-cdn.com/flex019/user_avatar/community.okteto.com/ramiro/32/6_2.png) [@ramiro](https://community.okteto.com/u/ramiro)\ **Post date:** [November 8, 2022, 7:02pm UTC](https://community.okteto.com/t/issues-with-okteto-api-reading-wildcard-ssl-cert/325/4 "2022-11-08T19:02:34Z")
That makes sense, that’s a great way to get around that issue! Looking at the instructions for certbot on [this repo](https://github.com/nabsul/k8s-letsencrypt) (I’m assuming you followed a similar path, please let me know if you didn’t), could you try re-importing the certificate but using `fullchain1.pem ` as the value of `--key`? ---
**Author:** ![pmarks](https://avatars.discourse-cdn.com/v4/letter/p/ecccb3/32.png) [@pmarks](https://community.okteto.com/u/pmarks)\ **Post date:** [November 8, 2022, 7:23pm UTC](https://community.okteto.com/t/issues-with-okteto-api-reading-wildcard-ssl-cert/325/5 "2022-11-08T19:23:17Z")
Hello again, Yes, I followed a similar path, except I ran certbot locally and used a readily available webserver (instead of containers as per the docs) to perform the HTTP-01 challenge. I also didn’t specify the challenge I wanted to do, so I had to perform both an HTTP and a DNS challenge for the cert to be valid. I did attempt to add the cert with the full chain pemfile as you recommended, but received the same error: ```auto $ k create secret tls dev-prism-cert --key=privkey1.pem --cert=fullchain1.pem --namespace okteto secret/dev-prism-cert created ``` After a fresh `helm install`: ```auto $ k logs okteto-api-596c98c444-9fr2m {"timestamp":"2022-11-08T19:17:06Z","message":"Starting api service...","severity":"INFO","context":{"data":{"instance.id":"okteto-api-596c98c444-9fr2m"}}} {"timestamp":"2022-11-08T19:17:06Z","message":"using no-op tracer","severity":"INFO","context":{"data":{"instance.id":"okteto-api-596c98c444-9fr2m"}}} {"timestamp":"2022-11-08T19:17:06Z","message":"loading buildkit certificate","severity":"INFO","context":{"data":{"instance.id":"okteto-api-596c98c444-9fr2m"}}} {"timestamp":"2022-11-08T19:17:06Z","serviceContext":{"service":"okteto-enterprise-backend"},"message":"couldn't find a valid cert: unable to decode private key PEM: unable to get PrivateKey from PEM type: PRIVATE KEY","severity":"CRITICAL","context":{"data":{"instance.id":"okteto-api-596c98c444-9fr2m"},"reportLocation":{"filePath":"github.com/okteto/app/backend/pkg/registry/registry.go","lineNumber":52,"functionName":"Init"}}} ``` I originally tried an update but saw no change of behavior, so I attempted a fresh install to ensure it was trying the latest version of the cert. ---
**Author:** ![provecho](https://sea1.discourse-cdn.com/flex019/user_avatar/community.okteto.com/provecho/32/45_2.png) [@provecho](https://community.okteto.com/u/provecho)\ **Post date:** [November 8, 2022, 7:39pm UTC](https://community.okteto.com/t/issues-with-okteto-api-reading-wildcard-ssl-cert/325/6 "2022-11-08T19:39:13Z")
Hi @pmarks, I bet your private key is in PKCS8 format, which is not currently supported by Okteto components. The following PEM header matches PKCS8: ```auto -----BEGIN PRIVATE KEY----- ``` The following PEM header matches PKCS1: ```auto -----BEGIN RSA PRIVATE KEY----- ``` To convert your current PKCS8 private key into PKCS1, use the following command. Replace `cert-pkcs8.pem` with the filename of your current private key. Replace `cert-pkcs1.pem` with the desired filename for your PKCS1 private key. ```auto openssl rsa -in cert-pkcs8.pem -out cert-pkcs1.pem ``` ---
**Author:** ![pmarks](https://avatars.discourse-cdn.com/v4/letter/p/ecccb3/32.png) [@pmarks](https://community.okteto.com/u/pmarks)\ **Post date:** [November 8, 2022, 9:53pm UTC](https://community.okteto.com/t/issues-with-okteto-api-reading-wildcard-ssl-cert/325/7 "2022-11-08T21:53:51Z")
@provecho That did the trick! The command that worked for me was slightly different than the one you gave: `$ openssl pkcs8 -in cert-pkcs8.pem -traditional -nocrypt -out cert-pkcs1.pem` Otherwise, it was not changing the format as expected. Once I updated the cert and reinstalled Okteto, everything worked as expected! I can now verify the api is happy and `okteto context` works without the insecure flag. Thanks very much to you and to @ramiro for your assistance, as we are now unblocked and can work on our proof-of-concept further! Once we get a bit of time to play around and gather our questions, we will be reaching back out to talk further! Thanks again to you both! Peter ---
**Author:** ![ramiro](https://sea1.discourse-cdn.com/flex019/user_avatar/community.okteto.com/ramiro/32/6_2.png) [@ramiro](https://community.okteto.com/u/ramiro)\ **Post date:** [November 8, 2022, 10:13pm UTC](https://community.okteto.com/t/issues-with-okteto-api-reading-wildcard-ssl-cert/325/8 "2022-11-08T22:13:15Z")
Great! We’ll update our docs to be clearer about the supported certificate formats.