Cache ContentModuleScanner results to fix Junie perf issue - #1567
Cache ContentModuleScanner results to fix Junie perf issue#1567Flórián Garaba (fgaraba) wants to merge 1 commit into
Conversation
| ) : ModuleDescriptorResolver<FileBasedModule>() { | ||
|
|
||
| private val contentModuleScanner = ContentModuleScanner(fileSystemProvider) | ||
| private val moduleArtifactCache = ConcurrentHashMap<Path, Map<String, Path>>() |
There was a problem hiding this comment.
Scope this cache to a single resolveContentModules pass rather than the lifetime of IdePluginManager. ContentModuleLoader is retained by IdePluginManager, and the manager can be reused to parse many plugin artifacts, so this map otherwise retains every scanned artifact and its module paths indefinitely.
|
|
||
| private fun resolveModuleArtifact(pluginArtifactPath: Path, moduleName: String): Path? { | ||
| val moduleMap = moduleArtifactCache.computeIfAbsent(pluginArtifactPath) { path -> | ||
| contentModuleScanner.getContentModules(path).modules.associate { it.id to it.artifactPath } |
There was a problem hiding this comment.
associate silently collapses duplicate module IDs. If the same root-level module descriptor exists in two JARs under lib/, we now pick one artifact and bypass LibDirectoryPluginLoader, which previously reported MultiplePluginDescriptors.
Please preserve all candidates, e.g. as Map<String, List<Path>>, and use the direct-JAR fast path only when singleOrNull() returns a unique match. For duplicates, fall back to the existing loader or report the ambiguity explicitly.
2 participants
Cache
lib/scan results inFileBasedModuleDescriptorResolverto fix content-module resolution bottleneckJunie (plugin 1072850) declares 64 content modules, none of which use the fast path (lib/modules/.jar) -- they all land in a single JAR under
lib/. This causedFileBasedModuleDescriptorResolver.getModuleCreatorto fall through toLibDirectoryPluginLoaderfor every module, which sequentially opens all JARs in lib/ to find the one containing the descriptor. With 243 JARs in Junie'slib/, that's 15,552 JAR operations per plugin parse - adding ~2 minutes to structure parsing in the Marketplace's PluginStructureParser.ContentModuleScanneralready scans all JARs inlib/andlib/modules/and builds a moduleId -> artifactPath map; it was previously used only for classpath setup. This change reuses it insideFileBasedModuleDescriptorResolver: on the first miss of thelib/modules/<name>.jarfast path, the resolver callsContentModuleScanneronce and caches the result in a map keyed by plugin artifact path. Subsequent modules resolve via O(1) map lookup.