allow setting different certificates for kube-controller-managed CSR signers #90822

deads2k · 2020-05-06T20:14:41Z

builds on #90731

The kube-controller-manager managed signers can now have distinct signing certificates and keys. See the help about --cluster-signing-[signer-name]-{cert,key}-file. --cluster-signing-{cert,key}-file is still the default.

/kind feature
/priority important-soon
@kubernetes/sig-auth-pr-reviews

The kube-controller-manager managed signers can now have distinct signing certificates and keys.  See the help about `--cluster-signing-[signer-name]-{cert,key}-file`.  `--cluster-signing-{cert,key}-file` is still the default.

pkg/controller/certificates/signer/config/types.go

fejta-bot · 2020-05-06T22:48:26Z

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

api/api-rules/violation_exceptions.list

deads2k · 2020-05-11T14:42:42Z

/retest

deads2k · 2020-05-12T12:23:26Z

/retest

micahhausler · 2020-06-08T19:24:13Z

Maybe a dumb question, but could I use this to turn off specific signers? If I only wanted to sign node-serving-cert but not node-client cert, could I do that with this change?

enj · 2020-06-11T04:25:34Z

Maybe a dumb question, but could I use this to turn off specific signers? If I only wanted to sign node-serving-cert but not node-client cert, could I do that with this change?

No, but you could set the node-client-cert to point to a private key that is not trusted anywhere, which would make it inert. Not really a solution though ...

enj

Overall LGTM.

I feel like we should add something like a --cluster-signing-explicit-signers flag which is a bool (default false) that, when set to true, prevents KCM from starting when each signer does not have an explicit key/cert specified. This would allow you to know that a new signer has been added and that you need to address that in some way. Otherwise a new signer addition would implicitly use the default cert/key file, which may not be desired.

enj · 2020-06-11T04:33:25Z

cmd/kube-controller-manager/app/certificates.go

-	if ctx.ComponentConfig.CSRSigningController.ClusterSigningCertFile == "" || ctx.ComponentConfig.CSRSigningController.ClusterSigningKeyFile == "" {
+	hasSingleSigningFile := ctx.ComponentConfig.CSRSigningController.ClusterSigningCertFile == "" || ctx.ComponentConfig.CSRSigningController.ClusterSigningKeyFile == ""
+	hasKubeletServingFile := ctx.ComponentConfig.CSRSigningController.KubeletServingSignerConfiguration.CertFile == "" || ctx.ComponentConfig.CSRSigningController.KubeletServingSignerConfiguration.KeyFile == ""
+	hasKubeletClientFile := ctx.ComponentConfig.CSRSigningController.KubeletClientSignerConfiguration.CertFile == "" || ctx.ComponentConfig.CSRSigningController.KubeletClientSignerConfiguration.KeyFile == ""
+	hasKubeAPIServerClientFile := ctx.ComponentConfig.CSRSigningController.KubeAPIServerClientSignerConfiguration.CertFile == "" || ctx.ComponentConfig.CSRSigningController.KubeAPIServerClientSignerConfiguration.KeyFile == ""
+	hasLegacyUnknownFile := ctx.ComponentConfig.CSRSigningController.LegacyUnknownSignerConfiguration.CertFile == "" || ctx.ComponentConfig.CSRSigningController.LegacyUnknownSignerConfiguration.KeyFile == ""
+	if !hasSingleSigningFile && !hasKubeletServingFile && !hasKubeletClientFile && !hasKubeAPIServerClientFile && !hasLegacyUnknownFile {


Er, what happens if you specify one of the signers but not the default signing file?

Can we add some tests to exercise this wiring?

Er, what happens if you specify one of the signers but not the default signing file?

You'll get an error about an unspecified file. Comes from the code doing the initial load. What is your preferred behavior?

Can we add some tests to exercise this wiring?

Do you have an example of flag wiring behavior tests you like? I don't object, I just haven't seen many.

You'll get an error about an unspecified file. Comes from the code doing the initial load. What is your preferred behavior?

That seems fine to me.

Do you have an example of flag wiring behavior tests you like?

There are some tests in kube-controller-manager/app/options/options_test.go that test the input flags to CSRSigningControllerOptions bits. Maybe combine that with startCSRSigningController and some permutations of the flags?

deads2k · 2020-07-13T17:08:46Z

/retest

fejta-bot · 2020-07-13T19:46:34Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

deads2k · 2020-07-13T20:38:03Z

/retest

fejta-bot · 2020-07-13T23:16:33Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-14T02:46:35Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-14T06:16:40Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-14T11:10:34Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-14T15:22:34Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-14T20:58:47Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-15T01:32:49Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-15T04:40:20Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-15T08:31:23Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

deads2k · 2020-07-15T13:04:11Z

/retest

deads2k · 2020-07-15T17:30:30Z

/retest

fejta-bot · 2020-07-15T20:46:30Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-16T03:04:12Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-16T07:16:13Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-16T12:52:25Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

deads2k · 2020-07-16T18:05:45Z

/retest

fejta-bot · 2020-07-16T23:42:02Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-17T03:54:01Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-17T08:48:00Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-17T13:21:00Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

liggitt · 2020-07-17T17:04:17Z

/retest

fejta-bot · 2020-07-17T22:48:13Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

k8s-ci-robot · 2020-07-18T04:45:40Z

@deads2k: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
pull-kubernetes-e2e-kind-ipv6	5fa1580efb9a1bb3dc68efc40dfc2518950d536a	link	`/test pull-kubernetes-e2e-kind-ipv6`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

fejta-bot · 2020-07-18T08:36:28Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

k8s-ci-robot requested review from jayunit100 and tallclair May 6, 2020 20:18

munnerz reviewed May 6, 2020

View reviewed changes

pkg/controller/certificates/signer/config/types.go Show resolved Hide resolved

pkg/controller/certificates/signer/config/types.go Show resolved Hide resolved

deads2k force-pushed the csr-separate-signer-flags-02 branch 2 times, most recently from 77857ca to 4a889bc Compare May 11, 2020 13:32

deads2k commented May 11, 2020

View reviewed changes

api/api-rules/violation_exceptions.list Show resolved Hide resolved

deads2k force-pushed the csr-separate-signer-flags-02 branch from 4a889bc to 950bedd Compare May 11, 2020 18:15

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 5, 2020

deads2k force-pushed the csr-separate-signer-flags-02 branch from 950bedd to d60a117 Compare June 8, 2020 14:41

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 8, 2020

enj reviewed Jun 11, 2020

View reviewed changes

deads2k force-pushed the csr-separate-signer-flags-02 branch 2 times, most recently from 5fa1580 to e89bd83 Compare July 2, 2020 19:00

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 2, 2020

deads2k force-pushed the csr-separate-signer-flags-02 branch from e89bd83 to 94f2ac5 Compare July 6, 2020 18:08

k8s-ci-robot merged commit 05f6812 into kubernetes:master Jul 18, 2020

github-actions bot mentioned this pull request Jul 22, 2020

Week Ending July 19, 2020 dev-obs/actus#196

Open

ialidzhikov mentioned this pull request Oct 27, 2022

kube-controller-manager: unknown flag: --cluster-signing-kube-apiserver-client-cert-file gardener/gardener#6906

Closed

Jan	FEB	Mar
	10
2024	2025	2026

allow setting different certificates for kube-controller-managed CSR signers #90822

allow setting different certificates for kube-controller-managed CSR signers #90822

Conversation

deads2k commented May 6, 2020

fejta-bot commented May 6, 2020

deads2k commented May 11, 2020

deads2k commented May 12, 2020

micahhausler commented Jun 8, 2020

enj commented Jun 11, 2020

enj left a comment

Choose a reason for hiding this comment

enj Jun 11, 2020

Choose a reason for hiding this comment

deads2k Jun 30, 2020

Choose a reason for hiding this comment

deads2k Jun 30, 2020

Choose a reason for hiding this comment

enj Jun 30, 2020

Choose a reason for hiding this comment

deads2k commented Jul 13, 2020

fejta-bot commented Jul 13, 2020

deads2k commented Jul 13, 2020

fejta-bot commented Jul 13, 2020

fejta-bot commented Jul 14, 2020

fejta-bot commented Jul 14, 2020

fejta-bot commented Jul 14, 2020

fejta-bot commented Jul 14, 2020

fejta-bot commented Jul 14, 2020

fejta-bot commented Jul 15, 2020

fejta-bot commented Jul 15, 2020

fejta-bot commented Jul 15, 2020

deads2k commented Jul 15, 2020

deads2k commented Jul 15, 2020

fejta-bot commented Jul 15, 2020

fejta-bot commented Jul 16, 2020

fejta-bot commented Jul 16, 2020

fejta-bot commented Jul 16, 2020

deads2k commented Jul 16, 2020

fejta-bot commented Jul 16, 2020

fejta-bot commented Jul 17, 2020

fejta-bot commented Jul 17, 2020

fejta-bot commented Jul 17, 2020

liggitt commented Jul 17, 2020

fejta-bot commented Jul 17, 2020

k8s-ci-robot commented Jul 18, 2020 • edited Loading

fejta-bot commented Jul 18, 2020

k8s-ci-robot commented Jul 18, 2020 •

edited

Loading