Repository navigation
chore(tools): classify build and test-helper diffs and warn on missing doc sync - #1625
VLukiantsev wants to merge 2 commits into
Conversation
…g doc sync analyze_diff_size.py counted Bazel BUILD/.bzl files and Go-style test-helper packages (prototest/, experimenttest/) as production code, inflating prod_code_lines toward the 400-line split threshold. - Classify `<name>test/` directories as tests, excluding English words such as latest/ and attest/. - Add a `build` category (BUILD, BUILD.bazel, *.bzl), reported as build_lines and excluded from prod and total code. - Emit a non-blocking `warnings` entry when production code under api/, cmd/, deploy/, internal/, or pkg/ changes with zero doc lines. - List the concrete doc locations under Documentation Alignment and reference the warning from the feature-development pre-PR step and the PR review checklist. TAG=agy CONV=cfb1c6d7-b7c5-44e1-88a9-635a7588d712
There was a problem hiding this comment.
Code Review
This pull request updates agent rules, skills, and the analyze_diff_size.py tool to support a new 'Build Definitions' category (excluding Bazel build files from code totals) and introduces a 'doc-sync' warning when production code changes without accompanying documentation updates. The review feedback correctly identifies a critical runtime error in tools/analyze_diff_size.py due to the use of the unsupported inline flag subtraction syntax (?-i:...) in Python's standard re module. To resolve this, the reviewer suggests refactoring _ORDERED_CATEGORIES and classify_file to handle category-specific regex flags, ensuring case-sensitive matching for build files while keeping other categories case-insensitive.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request updates agent rules, skills, and the analyze_diff_size.py tool to exclude build definitions (such as BUILD and *.bzl files) from code totals, categorize Go-style test-helper packages as test code, and introduce advisory warnings for production changes that lack corresponding documentation updates. The review feedback correctly identifies a critical runtime error in tools/analyze_diff_size.py caused by using the unsupported (?-i:...) inline regex modifier in Python's standard re module. The reviewer provided excellent suggestions to resolve this crash and optimize performance by pre-compiling the regular expressions at the module level with appropriate case-sensitivity flags.
Compile the classification patterns once at import time and compile the Bazel build patterns case-sensitively instead of relying on an inline `(?-i:...)` group. Behavior is unchanged; the existing build and lowercase-build-script tests cover it. TAG=agy CONV=cfb1c6d7-b7c5-44e1-88a9-635a7588d712
Summary
tools/analyze_diff_size.pynow counts Go-style test-helper packages as tests. It also reports Bazel build files as their ownbuild_linesmetric, so neither inflates the production-code split threshold anymore. The tool also adds a non-blocking doc-sync warning, and the agent rules now list the specific docs to check when behavior changes.Motivation & Context
internal/prototest/and*/experimenttest/as production code, along withBUILD.bazeland*.bzledits. That pushed focused PRs toward the 400-line production threshold and caused split-verification audits that weren't needed.AGENTS.mdalready has a Documentation Alignment principle, but recent PRs still changed behavior and APIs without updating flow docs or READMEs. Following the rule-authoring control ladder, the strongest cheap control is a tool-level check (rung 3). It is backed by named doc locations in the existing Documentation Alignment bullet (an amendment, not a new rule file) and cross-linked from the pre-PR and review checklists.Key Changes
tools/analyze_diff_size.py:<name>test/directory segments as tests. A small exclusion list keeps English words such aslatest/,attest/andcontest/in prod.buildcategory (BUILD,BUILD.bazel,*.bzl), reported asmetrics.build_linesand left out of both prod and total code.BUILDmatches case-sensitively, so lowercasebuildscripts still count as prod.warningslist. It fires when production lines underapi/,cmd/,deploy/,internal/orpkg/change and the diff has zero documentation lines. It is advisory and does not affectshould_trigger_split_verifier.tally_lines,exceeded_threshold_reasonsanddoc_sync_warningsto keep functions small.tools/analyze_diff_size_test.py: covers the helper-dir positives and false positives, build classification, exclusion of build files from totals, and the warning's edge cases..agents/rules/core_engineering.md: the Documentation Alignment bullet now listsdocs/flows/*.md, binary READMEs, component and profile READMEs, anddocs/migrations/*.md, and points to the new warning. The sizing list also documents build definitions..agents/skills/xmc_feature_development/SKILL.mdand.agents/rules/pr_review_guidelines.md: the existing pre-commit verification step and review checklist item now point to that bullet and to the warning..agents/skills/xmc_pr_split_verification/SKILL.mdand.agents/rules/github_authoring_and_prs.md: the sizing tables gain the test-helper and build-definition categories.Assumptions
analyze_diff_size.[a-z0-9]+test/with an English-word exclusion list is precise enough —git ls-filesonmainmatches only real helper packages (prototest,securitytest,helmtest,healthprobetest,statusdetectortest,crreadertest) — a future production directory ending intestwould be under-counted until added to the exclusion list.Verification & Testing
bazel test //tools:analyze_diff_size_test,./.githooks/pre-commit(Ruff, mypy, agent doc path check,run_affected_tests.py),./tools/analyze_diff_size.py --base-ref origin/main...//tools:analyze_diff_size_testand//tools:run_affected_tests_testpass, and all pre-commit checks pass. Every new branch (helper dir, false-positive exclusion, build category, each warning edge case) has a dedicated test.Related Issues / Bugs
N/A
TAG=agy
CONV=cfb1c6d7-b7c5-44e1-88a9-635a7588d712