Can't compile android+jackson: duplicate module-info.class

Hello,
in our android project we updated jackson library to the latest version GitHub - FasterXML/jackson-databind: General data-binding package for Jackson (2.x): works on streaming API (core) implementation(s)
and now it includes module-info.class file in each jar of the library.


We import 3 dependencies like this:

dependencies {
    implementation com.fasterxml.jackson.core:jackson-core:2.12.1
    implementation com.fasterxml.jackson.core:jackson-annotations:2.12.1
    implementation com.fasterxml.jackson.core:jackson-databind:2.12.1
}

The problem is proguard think this is a valid (or invalid) classes and build fails:
* What went wrong:
Execution failed for task ‘:minifyMobileReleaseWithProguard’.

java.io.IOException: Can’t write [/home/s/develop/projects/ivi4/zengalt-android-ivi/ivi/build/intermediates/shrunk_jar/mobileRelease/minified.jar] (Can’t read [/home/s/.gradle/caches/transforms-3/c79e01a738f9da01ca5a61c74a478473/transformed/jetified-jackson-databind-2.12.1.jar(;;;;;;;;**.class)] (Duplicate jar entry [module-info.class]))

I google similar problems and none of the solutions are for gradle-implementation-imported libraries.
For example,

-injars my.jar(!META-INF/versions/**)

can’t be used, because we don’t import file named my.jar, instead we import library android-gradle-way by implementation com.fasterxml.jackson.core:ja...
Another solution

task ('proguard', type: ProGuardTask) {
    injars 'my.jar', filter: '!META-INF/versions/**'
}

can’t be used too for the same reason.

We use proguard like this:

configurations.all {
	resolutionStrategy {
		dependencySubstitution {
			substitute module('net.sf.proguard:proguard-gradle') with module('com.guardsquare:proguard-gradle:7.0.1')
		}
	}
}
	dependencies {
		classpath("com.android.tools.build:gradle:4.1.2") {
			exclude module: 'proguard-gradle'
		}
      }

Project successfully compiles without proguard or without jackson.
Please, help, I’m stuck.

Hi @samoylenkodmitry !

I was able to reproduce this issue with the jackson dependencies. The problem is, as you point out, the module-info.class files that are in the same package in different dependencies. When ProGuard processes the dependencies it merges them all together into the final jar resulting in duplicates.

However, since you’re building an Android app I don’t think these module-info.class are required.

Do you have a keep rule for them? I can only produce the error if I add a keep rule (or use -dontshrink).

If I remove the keep rule, ProGuard will shrink these since they are unused and therefore there will not be duplicates in the output.

1 Like

Thank you for taking a time to reproduce this. I don’t have keep rule for it, but have dontwarn rule. I will try to remove it and will reply later.

Removed -dontwarn rule and nothing changes.
However I think this is a partly project problem because I failed to reproduce the bug in simple new project. Now I will try to reproduce it in simple small project to know what precisely a reason why those files are not filtered out before proguard task. Unfortunately main project is too big (1m loc)

Thanks for the update!

The -dontwarn won’t affect the duplicate error during the output writing - as soon as a second file with the same name is encountered, an exception is thrown: https://github.com/Guardsquare/proguard-core/blob/master/src/proguard/io/ZipOutput.java#L276

If you can reproduce in a sample project that would be very useful. Do you have any keep rule like -keep class *?

For -keep part: no, and also I removed any mentions of module-info in my proguard-rules file.

Today I have tried to create a simple project using the same proguard and gradle configuration and failed to reproduce this error. Now it is clear to me that something wrong with my big project configuration. The next idea is to remove parts from it until error goes away. I will reply when I find out something new. It seems like my problem is unique. But some developers have similar problem related to module-info&proguard in other cases. It would be a good feature to support excluding filter in proguard config file.

Hi @samoylenkodmitry

It’s possible there is a consumer -keep rule that keeps this file - these are rules that are distributed in dependencies in proguard.txt or META-INF/proguard/* files.

You could print the complete configuration by adding -printconfiguration fullconfig.pro to your ProGuard config.

Unfortunately there isn’t an easy way to filter out these consumer rules with the AGP ProGuard plugin but at least this would point to the cause of the issue.

I followed your recommendation to print out all config and there is no -keep rule for module-info.class.
(fullconfig.pro - Pastebin.com)
However, I managed to catch this error in a smallish project. I will remove some NDA dependencies and will publish it later.

I think I’ve tracked down the problem rule:

# Jackson
-keep class * {
    @com.fasterxml.jackson.annotation.JsonCreator
    <fields>;
    @com.fasterxml.jackson.annotation.JsonProperty
    <fields>;
    @com.fasterxml.jackson.annotation.JsonCreator
    <methods>;
    @com.fasterxml.jackson.annotation.JsonProperty
    <methods>;
}

Is this your own rule or one from a consumer rules file?

This will keep all classes, and instead it should use -keepclassmembers

There’s some common Jackson rules here: JacksonOnAndroid · FasterXML/jackson-docs Wiki · GitHub

1 Like

You can see the effect of this rule on a Jackson jar using the ProGuard Playground: https://playground.proguard.com/p/6BxoZ8

Indeed, you are correct. That rule was in one of my dependencies aar.
I create a simple project reproducing this error including this aar - GitHub - samoylenkodmitry/proguard-module-info-error: Simple little project that reproduce proguard error when importing a 3rd party library that have explicit rule to keep some other dependencies

I will ask our dependency provider to recompile this lib.

However, how can we resolve this by ourselves? Can we explicitly omit this rule?
Also, I noticed that when I switch to R8 project compiles successfully. Does it resolve this conflict somehow?

Thanks for creating the sample project! I see in tricky_lib.aar the offending rule:

# Jackson
-keep class * {
     @com.fasterxml.jackson.annotation.JsonCreator *;
     @com.fasterxml.jackson.annotation.JsonProperty *;
}

This is definitely something that the library developer should fix, since shrinking is basically turned off for all classes (since it matches all classes, all of them are kept) for anyone who uses this library + ProGuard/R8. Another rule that should not be in the library is -printmapping - this should be up to the end-user of the library if they want to print a mapping file or not.

Some of their other rules are also broad and will keep too much of an app:

-keep public class * {
    public protected *;
}

This will keep all public/protected members + public classes in your app unnecessarily. This rule could be applied to the library itself but it shouldn’t be in the consumer rules (proguard.txt inside the aar).

The -keepattributes rule may also keep too many attributes. The library should keep a minimum that are required, but the rest should be up to the app developer.

As for R8, I’m not sure how they handle module-info.class, maybe they handle it specifically since for Android it’s not required. So far, since ProGuard is used for Java and Android it hasn’t tried to handle this automatically and just treats it like any other class. I’ll try to check what R8 does.

2 Likes

Thank you, James!

Now it’s up to our lib provider to fix this.

1 Like