Repository navigation
GROOVY-12725: Preserve default-import package precedence and thread-safety in ResolveVisitor - #3260
daniellansun wants to merge 5 commits into
Conversation
…afety in ResolveVisitor
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #3260 +/- ##
==================================================
+ Coverage 74.4010% 74.4711% +0.0701%
- Complexity 44345 44874 +529
==================================================
Files 1699 1703 +4
Lines 150998 152063 +1065
Branches 28770 29144 +374
==================================================
+ Hits 112344 113243 +899
- Misses 28380 28409 +29
- Partials 10274 10411 +137
🚀 New features to boost your workflow:
|
JMH summary — classic (commit
|
| Group | Speedup | Calibrated | n |
|---|---|---|---|
| bench | 1.275 × | 0.980 × | 134 |
| core | 1.170 × | 1.177 × | 108 |
| grails | 1.303 × | 1.245 × | 80 |
⚠️ 3 benchmarks at least 1.5× slower than the 90-day baseline:
org.apache.groovy.bench.AryBench.groovy ( {"n":"1000000"} )— 2.73× slower (calibrated)org.apache.groovy.bench.AryBench.groovyCS ( {"n":"1000000"} )— 2.73× slower (calibrated)org.apache.groovy.bench.StaticMethodCallIndyBench.staticChain_groovyCS— 1.70× slower (calibrated)
⚠️ Runner speed differs ≥15% from the historical baseline hardware for: bench, core-hz, grails-ad. Raw speedups are not meaningful for those parts — use the calibrated column.
Runner calibration (this run vs baseline hardware): bench 1.29× (27 rulers) · core-ag 0.87× (3 rulers) · core-hz 1.25× (3 rulers) · grails-ad 1.23× (3 rulers) · grails-ez 0.92× (3 rulers)
Baseline: dev/bench/jmh/<part>/classic/data.js on gh-pages, trailing 90 days. Daily dashboard · Per-suite raw data
… cached first Order default-import packages in one place, and check a higher-precedence prefix before a partial cache entry is accepted. The resolver cache is a ConcurrentHashMap, so the memoize wrapper no longer forwards Map methods.
JMH summary — indy (commit
|
| Group | Speedup | Calibrated | n |
|---|---|---|---|
| bench | 1.115 × | 0.952 × | 134 |
| core | 2.180 × | 2.176 × | 108 |
| grails | 1.407 × | 1.313 × | 80 |
⚠️ 3 benchmarks at least 1.5× slower than the 90-day baseline:
org.apache.groovy.bench.AryBench.groovyCS ( {"n":"1000000"} )— 2.17× slower (calibrated)org.apache.groovy.bench.StaticMethodCallIndyBench.staticChain_groovyCS— 1.57× slower (calibrated)org.apache.groovy.bench.StaticMethodCallIndyBench.instanceChain_groovy— 1.55× slower (calibrated)
⚠️ Runner speed differs ≥15% from the historical baseline hardware for: bench, core-hz, grails-ez. Raw speedups are not meaningful for those parts — use the calibrated column.
Runner calibration (this run vs baseline hardware): bench 1.16× (27 rulers) · core-ag 0.89× (3 rulers) · core-hz 1.23× (3 rulers) · grails-ad 0.93× (3 rulers) · grails-ez 1.20× (3 rulers)
Baseline: dev/bench/jmh/<part>/indy/data.js on gh-pages, trailing 90 days. Daily dashboard · Per-suite raw data
| if (matches.size() == 1) { | ||
| return Collections.singleton(matches.get(0)); | ||
| } | ||
| return Collections.unmodifiableSet(new LinkedHashSet<>(matches)); |
There was a problem hiding this comment.
why not work with a LinkedHashSet from the beginning? And why transform it to a unmodifiableSet?
There was a problem hiding this comment.
ordered() now inserts into a LinkedHashSet while it walks precedence. Match order is insertion order, and there is no ArrayList to copy afterwards.
The unmodifiable wrapper stays. Collections.unmodifiableSet does not copy; it only wraps that set. Callers publish the result: ResolveVisitor stores it in a ConcurrentHashMap, and until a later hit replaces the entry the same instance is the startup index (Groovy12725 checks that with assertSame). A reader on another thread must not observe a set being edited, and the tests expect add to throw. An update replaces the map entry. An empty result is still Collections.emptySet(), and a single match is still Collections.singleton, which is already immutable.
Package precedence and the replace-rather-than-mutate cache behaviour are unchanged.
There was a problem hiding this comment.
The unmodifiable wrapper stays.
Collections.unmodifiableSetdoes not copy; it only wraps that set. Callers publish the result:ResolveVisitorstores it in aConcurrentHashMap, and until a later hit replaces the entry the same instance is the startup index (Groovy12725checks that withassertSame). A reader on another thread must not observe a set being edited, and the tests expectaddto throw. An update replaces the map entry. An empty result is stillCollections.emptySet(), and a single match is stillCollections.singleton, which is already immutable.
I assume you mean src/test/groovy/org/apache/groovy/vmplugin/internal/DefaultImportPackagesTest.groovy. This test does not establish multi threading behaviour. It tests an API and can maybe be used as regression test to ensure the ordering logic is kept. But it does not explain why this is needed. What is the goal? To catch the mistake during development? It most likely is not to enable multithreading behaviour. Also unmodifiable is not the same as immutable. A unmodifiableSet may still have publication issues based on the underlying collection. Again... I am not saying this is wrong. Just want a more clear picture why here exactly like this and not different. Just pointing to the test with the argument of thread safety and then having a test that does not test that if not convincing.
There was a problem hiding this comment.
You are right on both counts. DefaultImportPackagesTest (and the add throws checks in Groovy12725) are API / regression tests: they lock the iteration order and the fact that a caller cannot add to a returned set. They do not establish concurrent behaviour, and I should not have pointed at them as if they did.
The concurrency design is not the wrapper itself. It is that an update replaces the ConcurrentHashMap entry instead of mutating the set a reader may already be iterating. compute in ResolveVisitor.resolveOneDefaultImport calls including(), which returns either the same instance or a new set; it never adds to a set that has already been published. The map provides the safe publication of that reference. unmodifiableSet does not invent that race freedom, and it is also not the same as an immutable set: if the backing LinkedHashSet were still written after the wrap, the view would change and you could have a publication problem.
What the wrapper is for: the same set instance is shared. Until a later hit replaces a cache entry, the cache holds the startup index sets (assertSame in Groovy12725). A caller who could mutate a cache value would also mutate that snapshot. The wrapper is a defensive contract against that, and the add tests are there to catch a later change that starts returning a mutable set or updating in place. They are not a thread-safety proof.
I used Collections.unmodifiableSet rather than Set.copyOf / Set.of because those do not keep the LinkedHashSet insertion order, which is the precedence order. The mutable set is a local in ordered(); it is not retained or written after the wrap, so the published view does not change. Empty and single-element results are already Collections.emptySet() and Collections.singleton.
ordered() inserts matches while walking the precedence array and still wraps the set, because callers publish it to concurrent readers. including() asks for capacity 1 when existing is null, since only the new package prefix is added.
|
There are two aspects I would like to see addressed before merging: 1. Verify cache behavior across different classloaders
The new partial-cache algorithm appears to account for this by checking higher-precedence packages before accepting a previously discovered lower-precedence package. However, the regression tests currently do not establish that resolution remains independent of compilation history across different classloaders. I suggest adding a test along these lines:
The expected resolutions are respectively This would verify that the globally shared cache does not cause one compilation's classloader visibility to influence another compilation's resolution. 2. Make the cache-state invariant explicit The distinction between complete and partial cache entries currently relies on object identity: known == indexedThis works with the current implementation because startup index entries are shared with the cache, while subsequent updates replace cache values rather than modifying them. However, object identity now carries semantic meaning: it determines which resolution algorithm is used. This is an important invariant for anyone modifying this code in the future. I suggest documenting explicitly that startup entries represent the complete index, dynamically discovered entries are only partial information, and cache updates must preserve this distinction. I would also consider a concurrency regression test for simultaneous resolutions of the same name. The replacement-based update strategy using Overall, I agree with the implementation direction. These suggestions are primarily about establishing and preserving the correctness properties on which the optimization depends. |
Document that a startup cache entry is the complete index and that a replaced entry is only partial, distinguished by object identity. Add regression tests for classloader-independent resolution and for concurrent resolutions of the same simple name.
|
Thanks for the suggestions. I added two regression tests in
The new comments describe a startup cache entry as the complete index, and a replaced entry as only partial information. The distinction is object identity, because updates replace the map entry rather than mutating a shared startup set. I also added The resolution algorithm itself is unchanged. |
|
✅ All tests passed ✅🏷️ Commit: 2df506b Learn more about TestLens at testlens.app/docs. |



https://issues.apache.org/jira/browse/GROOVY-12725