Add Helm E2E tests

[ArangoGutierrez]

What would you like to be added:
Now that we are running the E2E with Kind and we have more flexibility over our E2E infra we should include a small test suite for running via Helm deployment.

Why is this needed:
Currently, E2E deploys objects from code, leaving Helm charts untested. E2E could pass but Helm charts be broken and we won't tell until a manual test.

/help
/good-first-issue

[k8s-ci-robot]

@ArangoGutierrez:
This request has been marked as suitable for new contributors.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-good-first-issue command.

Details

In response to this:

What would you like to be added:
Now that we are running the E2E with Kind and we have more flexibility over our E2E infra we should include a small test suite for running via Helm deployment.

Why is this needed:
Currently, E2E deploys objects from code, leaving Helm charts untested. E2E could pass but Helm charts be broken and we won't tell until a manual test.

/help
/good-first-issue

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[THUzxj]

Hi, I would like to work on this. I might need to spend some time understanding this project first. Thanks!

[TessaIO]

Hi, I would like to work on this. I might need to spend some time understanding this project first. Thanks!

lmk if you need any guidance with that!

[THUzxj]

Hi @TessaIO , thank you for offering help. I am struggling to run the e2e test locally. Since the Makefile doesn't provide scripts to create the testing environment, I'd like to use Kind. Here is my practice to run the e2e test:

kind create cluster
# in source dir
make image-all
kind load docker-image  registry.k8s.io/nfd/node-feature-discovery:xxxx
make test-e2e

However, I am faced with the problem that Kind still complains "Failed to pull image" error and I am investigating this.

Thus I would appreciate it if you could answer these questions: Do you run the e2e tests locally when developing? If so, do you have some recommended ways?

[TessaIO]

Hi @THUzxj,
Your setup is correct. Few params need to be tweaked here. When u run make test-e2e you can see this code snippet in the Makefile file:

e2e-test:
	@if [ -z ${KUBECONFIG} ]; then echo "[ERR] KUBECONFIG missing, must be defined"; exit 1; fi
	$(GO_CMD) test -timeout=1h -v ./test/e2e/ -args \
	    -nfd.repo=$(IMAGE_REPO) -nfd.tag=$(IMAGE_TAG_NAME) \
	    -kubeconfig=$(KUBECONFIG) \
	    -nfd.e2e-config=$(E2E_TEST_CONFIG) \
	    -nfd.pull-if-not-present=$(E2E_PULL_IF_NOT_PRESENT) \
	    -ginkgo.focus="\[k8s-sigs\/node-feature-discovery\]" \
	    -ginkgo.label-filter=$(E2E_GINKGO_LABEL_FILTER) \
	    -ginkgo.v \
	    $(if $(OPENSHIFT),-nfd.openshift,)
	if [ "$(E2E_TEST_FULL_IMAGE)" = "true" ]; then \
	    $(GO_CMD) test -timeout=1h -v ./test/e2e/ -args \
	        -nfd.repo=$(IMAGE_REPO) -nfd.tag=$(IMAGE_TAG_NAME)-full \
	        -kubeconfig=$(KUBECONFIG) \
	        -nfd.e2e-config=$(E2E_TEST_CONFIG) \
	        -nfd.pull-if-not-present=$(E2E_PULL_IF_NOT_PRESENT) \
	        -ginkgo.focus="\[k8s-sigs\/node-feature-discovery\]" \
	        -ginkgo.label-filter=$(E2E_GINKGO_LABEL_FILTER) \
	        -ginkgo.v \
	        $(if $(OPENSHIFT),-nfd.openshift,); \
	fi

As you can see that nfd.tag is equal to the IMAGE_TAG_NAME which is equal to the value of this command:

$(shell git describe --tags --dirty --always)

So in a nutshell, you're e2e-test command would try to pull node-feature-discovery:smth-random-here
You have two ways here:
1- If you want to run tests against the latest version you need to change version in you Makefile to point 0.16.1 or whatever u want
2- Build and push your own image using:

make image
make push

and then run your e2e-tests

/assign @THUzxj

[marquiz]

However, I am faced with the problem that Kind still complains "Failed to pull image" error and I am investigating this.

You need to use the E2E_PULL_IF_NOT_PRESENT=true so that your test cluster doesn't try to pull that (local) image:

E2E_PULL_IF_NOT_PRESENT=true make e2e-test