[AntonTyb]

Summary

Replaces the legacy external https://actuator.api.<env>.<domain>/actuate URI in the Helm ConfigMaps with a shared xmanager-cloud.actuatorAudience helper (https://actuator.xmc.svc.cluster.local/actuate) and removes the unused legacy Ingress BackendConfig (grpc-backend-config.yaml) and xmc_iap_secret blueprint module.

Motivation & Context

Following the migration from Cloud Tasks HTTP push to in-cluster Redis Streams (use_redis_queue: true) and from legacy GKE v1 Ingress to the GKE Gateway API (HTTPRoute), the Actuator service only runs on a cluster-local ClusterIP (actuator-service on HTTP :8080) and no longer exposes /actuate externally. However, the Actuator (audience) and ESS (ess_actuator_url) Helm ConfigMaps still constructed an external https://actuator.api.<env>.<domain>/actuate JWT/OIDC audience identifier, and the Helm chart and Cluster Toolkit v2 blueprint still provisioned an unused v1 Ingress BackendConfig (shared-backend-config) and dummy xmc-iap-secret that GKE Gateway API ignores.

Key Changes

• Defined xmanager-cloud.actuatorAudience once in deploy/helm/templates/_helpers.tpl and referenced it in both deploy/helm/templates/actuator/configmap.yaml (audience) and deploy/helm/templates/ess/configmap.yaml (ess_actuator_url).
• Removed deploy/helm/templates/ess/grpc-backend-config.yaml and its entry in test/e2e/lib/helm_deployer.py.
• Removed the unused xmc_iap_secret module from deploy/ctk_v2/xmanager-cloud.yaml.
• Updated test/helm/actuator_dev_oidc_config_test.py to parse and assert equality of actuator-config audience and ess-config ess_actuator_url against https://actuator.xmc.svc.cluster.local/actuate, and added test_chart_omits_legacy_ingress_backend_config to test/helm/httproute_config_test.py.

Assumptions

• In-cluster logical audience identifier https://actuator.xmc.svc.cluster.local/actuate satisfies all active OIDC token and per-task Tink JWT signature validation paths — verified against internal/task_queue/client.go:304-307 and cmd/actuator/workunittaskhandler/workunit_task_handler.go:346,365 where ESS signs tasks with aud = ess_actuator_url and Actuator validates against audience — if either side drifted from the shared xmanager-cloud.actuatorAudience template helper, signature validation would fail for every task.
• Operators drain the xmc-tasks-stream Redis stream (quiescing work unit launches until the stream backlog is 0) before running the one-time helm upgrade from the legacy https://actuator.api.<env>.<domain>/actuate audience to xmanager-cloud.actuatorAudience — verified against cmd/actuator/workunittaskhandler/workunit_task_handler.go:265-266,365 and internal/kubernetes/common/redis/consumer.go:234-237, where actuator and ess Deployments roll independently on their own checksum/config and Actuator treats JWT signature/audience mismatches as non-retryable (moving messages to xmc-dlq-stream without marking the Spanner Operation done) — if tasks signed with the old audience are backlogged or enqueued by old ESS pods while new Actuator pods consume during rollout, those tasks will land in xmc-dlq-stream.
• No active deployment consumes BackendConfig/shared-backend-config or Secret/xmc-iap-secret — verified by searching all deploy/helm/templates/** and deploy/ctk_v2/** manifests, where cloud.google.com/backend-config annotations are absent and GKE Gateway API routes (HTTPRoute) do not read v1 Ingress BackendConfig CRDs — if an out-of-tree v1 Ingress manifest referenced shared-backend-config, it would fail to find the BackendConfig.

Verification & Testing

• Commands Executed:
  • bazel test --test_env=PATH --test_env=XMC_REQUIRE_HELM=1 //test/helm:actuator_dev_oidc_config_test //test/helm:httproute_config_test
  • ./.githooks/pre-commit (including python3 ./tools/run_affected_tests.py --skip-emulator-check)
• Coverage & Test Status: All Helm unit tests in test/helm and all 23 affected Bazel test targets (//test/e2e/...) passed (0 Trivy misconfigurations, 0 lint issues).

Related Issues / Bugs

N/A

TAG=agy
CONV=c5f694f5-b286-42df-821a-8d5c1a9350de

[gemini-code-assist]

Code Review

This pull request removes legacy GKE Ingress BackendConfig resources and updates the actuator and ESS configmaps to use a cluster-local actuator URI instead of an external one. However, the proposed cluster-local URI (https://actuator.xmanager.xmc.svc.cluster.local/actuate) contains an extra .xmanager label, which violates standard Kubernetes DNS formatting (<service-name>.<namespace>.svc.cluster.local) and will lead to DNS resolution failures. Please update the URI to https://actuator.xmc.svc.cluster.local/actuate across the configmaps and test files.

[janzak-dev]

[Priority: Actionable] deploy/helm/templates/_helpers.tpl:122-127
Observation: Thanks for documenting the drain step. Looking closer, the wording from my earlier suggestion turns out to be too loose to prevent task loss in practice:

1. More than launches enqueue signed tasks. Stop/delete (batchstopworkunits.go:148), restart (batchrestartworkunits.go:182), update (updateworkunit.go:101) and the ESS resumption worker (resumption/worker.go:335) all do. A global scheduling pause doesn't help either: signature validation (workunit_task_handler.go:265) runs before the pause check (:214), and DELETE is never paused (pause_handler.go:102).
2. The window is the rollout itself, not just the backlog before it. The actuator (replicas: 1, default RollingUpdate) keeps its old pod until the new one is Ready, while ESS rolls separately. So new ESS pods can sign with the new aud for the old actuator, and old ESS pods with the old aud for the new actuator. Producers must stay stopped until both rollouts finish, not only until the backlog drains.
3. "Backlog is 0" can't be checked with XLEN. The stream is only trimmed by approximate MAXLEN ~10000 (publisher.go:49-50), and XACK doesn't delete entries, so XLEN never reaches 0. The real signal is XINFO GROUPS xmc-tasks-stream → actuator-group with pending=0 and lag=0 (the same thing backlog.go:42-81 computes).

Risk / Rule: An operator following this comment can still send tasks to xmc-dlq-stream, where nothing replays them and their Operation is never marked done. Operators also don't read _helpers.tpl: the upgrade paths (INSTALL.md "Upgrading an Existing Installation" Step 4, dev_env.py gke-upgrade) never mention the drain. Documentation Alignment (AGENTS.md principle 5); accurate operational contracts (architectural_contracts.md).
Suggestion: Keep only the lasting rule in this comment, and move the one-time procedure into INSTALL.md Step 4 (or a docs/migrations/ runbook, like Step 6). For example: scale experiment-state-server-deployment to 0 → wait for actuator-group pending=0/lag=0 → helm upgrade → kubectl rollout status for the actuator → scale ESS back. The alternative is to have the actuator accept both audiences for one release.

Suggested change

	{{/*
	Logical JWT/OIDC audience shared by the ESS task signer and the actuator validator.
	Never dialed; must be identical on both sides. Upgrading from the legacy
	https://actuator.api.<env>.<domain>/actuate value changes the JWT aud contract
	while ess and actuator Deployments roll independently; quiesce launches until the
	xmc-tasks-stream Redis backlog is 0 before running helm upgrade.
	*/}}
	{{/*
	Logical JWT/OIDC audience shared by the ESS task signer (ess_actuator_url) and the actuator
	validator (audience). Never dialed; must be identical on both sides. See INSTALL.md
	"Upgrading an Existing Installation" for the one-time cutover from the legacy
	https://actuator.api.<env>.<domain>/actuate audience.
	*/}}