← All tasks
javaapache/hudi #19469Not a task: already works

hudi-presto-bundle depends on hudi-hadoop-mr-bundle, a fat jar whose classes it already shades

envgap__apache__hudi-19469

01 / FAILURE SIGNATURE

As reported upstream

No identifying execution failure has been captured.
Not a benchmark task.
  • The project already builds and runs before the fix, so there is nothing to repair.

02 / ENVIRONMENT RECIPE

Base commit
acbe341c9c741eff55bd779d219f3b14fa868335
Manifest
packaging/hudi-presto-bundle/pom.xml
Reproduce
Awaiting issue-specific recipe
Run under trace
Awaiting a meaningful runtime command

03 / ORIGINAL ISSUE TEXT

apache/hudi #19469 · read the original issue
## Task Description

**What needs to be done:**

Review whether `packaging/hudi-presto-bundle` should keep depending on `hudi-hadoop-mr-bundle`, and drop it if not.

**Why this task is needed:**

`hudi-hadoop-mr-bundle` is itself a shaded fat jar. `hudi-presto-bundle` depends on it and then shades a set of classes that jar already contains, so the same classes are pulled in from a bundle instead of from the library modules that produce them.

After #19433 the dependency block declares all three:

```xml
<dependency>
  <groupId>org.apache.hudi</groupId>
  <artifactId>hudi-hadoop-mr-bundle</artifactId>
  <version>${project.version}</version>
</dependency>
<dependency>
  <groupId>org.apache.hudi</groupId>
  <artifactId>hudi-hadoop-mr</artifactId>
  <version>${project.version}</version>
</dependency>
<dependency>
  <groupId>org.apache.hudi</groupId>
  <artifactId>hudi-hadoop-common</artifactId>
  <version>${project.version}</version>
</dependency>
```

`hudi-hadoop-mr` and `hudi-hadoop-common` were added by #19433 because the bundle shades both but reached them only transitively through `hudi-hadoop-mr-bundle`. Once that bundle started publishing a reduced POM the includes matched nothing and the presto jar lost 109 `org/apache/hudi/hadoop/**` entries. Declaring them directly fixed it and decoupled this bundle's shade inputs from another bundle's published dependency list.

That fix leaves `hudi-hadoop-mr-bundle` looking redundant, but #19433 deliberately did not remove it -- that PR was scoped to published POM metadata, and dropping a dependency changes shade inputs, which needs its own verification.

**What to check:**

1. Whether anything in the presto bundle's `artifactSet` still resolves only through `hudi-hadoop-mr-bundle` after the direct declarations were added.
2. Whether removing it changes jar contents at all. The bar is a byte-identical listing, or a difference that is understood and intended.
3. Whether the presto bundle should depend on library modules only, as a general rule -- one fat jar depending on another is worth avoiding regardless of whether it currently causes a problem.

**Verification:** the install-then-resolve recipe from #19433, since an in-reactor build cannot see published reduced POMs:

```
mvn install -pl packaging/hudi-hadoop-mr-bundle -DskipTests
mvn package -pl packaging/hudi-presto-bundle -DskipTests   # no -am
unzip -l packaging/hudi-presto-bundle/target/hudi-presto-bundle-*.jar | grep -c org/apache/hudi/hadoop/
```

Expect 109 before and after. Note that CI cannot currently catch a regression here -- see #19468.

## Task Type

Code improvement/refactoring

## Related Issues

**Parent feature issue:** #16407
**Related issues:** #19433, #19466, #19468
Continue on GitHub ↗

04 / LABELS

Labels from the report text only; not yet run

No supported category has been assigned.

Label rules and the text that matched
[]