Skip to content

test: add e2e framework and test runner#67

Open
kaffie-1517 wants to merge 5 commits into
volcano-sh:mainfrom
kaffie-1517:e2e-ci-workflow
Open

test: add e2e framework and test runner#67
kaffie-1517 wants to merge 5 commits into
volcano-sh:mainfrom
kaffie-1517:e2e-ci-workflow

Conversation

@kaffie-1517
Copy link
Copy Markdown

Summary

This PR adds the Ginkgo e2e test framework, runner scripts, and Go module dependencies needed to write and execute end-to-end tests for volcano-global. It provides shared multi-cluster clients, CRUD + wait helpers for every volcano-global resource type, and a make e2e-test entrypoint.

Changes in this PR

  • test/e2e/framework/clients.go : shared Clients struct holding Karmada, host, member Kube + Volcano clientsets
  • test/e2e/framework/framework.go : BeforeSuite init, namespace helpers, polling constants, CLI flags for kubeconfig paths
  • test/e2e/framework/vcjob.go : Create / Delete / Get / WaitForStatus helpers for VCJob
  • test/e2e/framework/queue.go : Create / Delete / Get / WaitForOpen helpers for Queue
  • test/e2e/framework/propagationpolicy.go : Create / Delete helpers for PropagationPolicy
  • test/e2e/framework/resourcebinding.go : WaitForSuspended / WaitForUnsuspended / WaitForExists helpers for ResourceBinding
  • test/e2e/framework/hyperjob.go : Create / Delete / Get / WaitForChildJobs helpers for HyperJob
  • test/e2e/e2e_suite_test.go : Ginkgo bootstrap that registers the framework and runs all suites
  • hack/e2e.sh : top-level entrypoint with SKIP_SETUP support (calls setup-e2e-env.sh then run-e2e.sh)
  • hack/run-e2e.sh : installs ginkgo, ensures PATH, overrides GOOS/CGO_ENABLED for local runs, runs tests with --race --fail-fast -p, collects Kind + pod logs
  • go.mod / go.sum : add ginkgo/v2, gomega, and transitive deps
  • Makefile : add e2e-test target

Follow-up changes

  • Test cases for all 4 scenarios
    • Add suites for: resource quota & priority, cross-cluster VCJob scheduling, data-dependency aware scheduling, and HyperJob scheduling
  • CI workflow and documentation
    • Add GitHub Actions workflow to run e2e in CI + docs for running/extending tests

Test plan

  • Run the e2e tests locally against an existing Karmada + Volcano environment:

    SKIP_SETUP=true make e2e-test
  • Verify Ginkgo discovers and executes suites:

    cd hack && ./run-e2e.sh
  • Confirm all framework helpers compile:

    go build ./test/e2e/...

Liked issue:

Signed-off-by: kaffie-1517 <jasmeetsingh171002@gmail.com>
Signed-off-by: kaffie-1517 <jasmeetsingh171002@gmail.com>
@volcano-sh-bot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign jessestutler for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a comprehensive E2E testing suite for Volcano-Global, including deployment automation, test execution scripts, and a multi-cluster testing framework. Feedback focuses on improving the robustness and portability of the environment by pinning tool versions and disabling CGO, while also suggesting the parameterization of hardcoded cluster names and dependency versions to enhance framework flexibility.

Comment thread hack/run-e2e.sh

# Install ginkgo
echo "Installing ginkgo..."
GO111MODULE=on go install github.com/onsi/ginkgo/v2/ginkgo@latest
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Installing ginkgo with @latest can lead to non-deterministic test environments if a new version is released with breaking changes. It is better to pin the version to match the one defined in go.mod.

Suggested change
GO111MODULE=on go install github.com/onsi/ginkgo/v2/ginkgo@latest
GO111MODULE=on go install github.com/onsi/ginkgo/v2/ginkgo@v2.28.1

Comment thread hack/run-e2e.sh
kubectl --context karmada-apiserver get ns default >/dev/null

set +e
GOOS=$(go env GOHOSTOS) GOARCH=$(go env GOHOSTARCH) CGO_ENABLED=1 \
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Setting CGO_ENABLED=1 might cause build failures in environments without a C compiler (like some CI runners). Since E2E tests typically only perform API calls, CGO is likely unnecessary. Consider setting it to 0 for better portability.

Suggested change
GOOS=$(go env GOHOSTOS) GOARCH=$(go env GOHOSTARCH) CGO_ENABLED=1 \
GOOS=$(go env GOHOSTOS) GOARCH=$(go env GOHOSTARCH) CGO_ENABLED=0 \

Comment thread hack/run-e2e.sh Outdated
Comment thread test/e2e/framework/clients.go Outdated
IMAGE_PREFIX=${IMAGE_PREFIX:-"volcanosh"}
KARMADA_KUBECONFIG=${KARMADA_KUBECONFIG:-"${HOME}/.kube/karmada.config"}
KARMADA_HOST_CLUSTER=${KARMADA_HOST_CLUSTER:-"karmada-host"}
VOLCANO_VERSION=${VOLCANO_VERSION:-"release-1.10"}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The default VOLCANO_VERSION is set to release-1.10, but the project appears to be using a much newer version (v1.14.0-alpha.0 according to go.mod). Using an outdated version for CRDs might lead to compatibility issues.

Suggested change
VOLCANO_VERSION=${VOLCANO_VERSION:-"release-1.10"}
VOLCANO_VERSION=${VOLCANO_VERSION:-"master"}

Comment thread hack/deploy-volcano-global.sh Outdated
Signed-off-by: kaffie-1517 <jasmeetsingh171002@gmail.com>
Signed-off-by: kaffie-1517 <jasmeetsingh171002@gmail.com>
Signed-off-by: kaffie-1517 <jasmeetsingh171002@gmail.com>
@JesseStutler
Copy link
Copy Markdown
Member

Thanks @kaffie-1517, but I didn't see any github workflow has been added in this PR, did you miss it? With github workflow, we can know that whether a PR will have bugs or have influence before the merge, and it's better for me to judge your PR, no need to build the clusters from the scratch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants