Duplicate jar entry

Hi,

I fall on this error after been waiting a long time, trying to obfuscate my project with proguard through sbt plugin .

Herre are informations about my build.
Scala 2.12.12
sbt 1.3.13

Dependencies :
“org.scalacheck” %% “scalacheck” % “1.14.1” % “test”,
“org.scalactic” %% “scalactic” % “3.2.7”,
“org.scalatest” %% “scalatest” % “3.2.7” % “test”,
“org.scalanlp” %% “breeze-natives” % “1.0”,
“org.scalanlp” %% “breeze” % “1.0”,
“org.typelevel” %% “spire” % “0.16.2”,
“org.apache.commons” % “commons-math3” % “3.4.1”,

sbt settings :
Proguard / proguardOptions ++= Seq(
“-keep interface org.** { ; }",
"-keep class org.
* { ; }",
"-keep enum org.
* { *; }”,
)
Proguard / proguardOptions ++= Seq("-dontnote", “-dontwarn”, “-ignorewarnings”),
Proguard / proguardVersion := “7.1.0-beta3”,
javaOptions in (Proguard, proguard) := Seq("-Xmx10G"),
Proguard / proguardOptions += ProguardOptions.keepMain(“xxx.Toto”),

The error

[error] Warning: can’t write resource [LICENSE] (Duplicate jar entry [LICENSE])
[error] Warning: can’t write resource [NOTICE] (Duplicate jar entry [NOTICE])
[error] Error: Can’t write [/home/XXX/Project/core/target/scala-2.12/sbt-1.0/proguard/core-0.42.jar] (Can’t read [/home/XXX/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/spire-macros_2.12/0.17.0-M1/spire-macros_2.12-0.17.0-M1.jar(;;;;;;;;!META-INF/MANIFEST.MF)] (Duplicate jar entry [h/e/c.class]))

Hi @KyBe67 ,

This issue is documented in our troubleshooting guide, which you can find here: Overview

It mentions the following;

Your input jars contain multiple resource files with the same name. ProGuard continues copying the resource files as usual, skipping any files with previously used names. Once more, the warning may be an indication of some problem though, so it’s advisable to remove the duplicates. A convenient way to do so is by specifying filters on the input jars. There is no option to switch off these warnings.

You can filter your input using the sbt plugin as described on this page, where an example is given to filter out the files in the META-INF directory; GitHub - sbt/sbt-proguard: Proguard sbt plugin

More on these file filters can be found in the ProGuard manual page linked below under “File Filters”; Usage

Kind regards,

Jonas

Thank you but despite adding the exclusion filter it still does not work.

I had add this in my build.sbt

    			Proguard / proguardInputFilter := { file =>
    			file.name match {
    				case "spire-macros_2.12-0.17.0-M1.jar" => {
    						Some("!META-INF/**")
    				}

And get the following error.

[error] Warning: can’t write resource [META-INF/MANIFEST.MF] (Duplicate jar entry [META-INF/MANIFEST.MF])
[error] Warning: can’t write resource [META-INF/MANIFEST.MF] (Duplicate jar entry [META-INF/MANIFEST.MF])
[error] Error: Can’t write [/home/XXX/lib/tataa/target/scala-2.12/sbt-1.0/proguard/tataa-0.42.jar] (Can’t read [/home/XXX/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/spire-macros_2.12/0.17.0-M1/spire-macros_2.12-0.17.0-M1.jar(;;;;;;;;!META-INF/**)] (Duplicate jar entry [e/e/c.class]))

!META-INF/** means exlcude all files under META-INF folder if i’m writte and that is what is supposed to solve my specific problem.
I don’t get something and i hope you will help me to understand what.
Thank you.

Is it possible there are multiple jars (other than spire-macros_2.12-0.17.0-M1.jar) presented in the input? In that case you want to exclude all the content in their META-INF directories, if not that would cause duplicates in the final jar.

I’m not familiar with the Scala syntax at all but something similar to the snippet below should help you filter the META-INF directories for all jars from the input;

val Pattern = "(.*\.jar)".r
proguardInputFilter in Proguard := { file =>
  file.name match {
    case Pattern => Some("!META-INF/**")
    case _                   => None
  }
}

Alternatively, you can list the different jars individually as you already did for spire-macros_2.12-0.17.0-M1.jar

Kind regards,

Jonas

You’re right there is other jar in the input, i catch them all by applying the given strategy :

                        val allJarsRegex = ".jar".r


			Proguard / proguardInputFilter := { file =>
				file.name match {
					case str if allJarsRegex.findFirstIn(str).isDefined => {
						Some("!META-INF/**")
					}
                                        case _ => None

But still same issue

Error: Can't write [/home/xxx/lib/tataa/target/scala-2.12/sbt-1.0/proguard/tataa-0.42.jar] (Can't read [/home/xxx/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/spire-macros_2.12/0.17.0/spire-macros_2.12-0.17.0-M1.jar(;;;;;;;;!META-INF/**)] (Duplicate jar entry [e/e/c.class]))

Is there another way to not obfuscate dependencies than excluding classes with

"-keep interface org.** { *; }",

Can we exclude the whole jar to be obfuscated ?

Kind regards,
Gaël

The fact that an obfuscated class was found as a duplicate is a bit weird and might be a bug in PG;
Duplicate jar entry [e/e/c.class]

Could you share the full build log and full configuration file, that might give us additional clues. The latter one can be obtained by adding -printconfiguration fullconfig.pro to your ProGuard configuration file. We would need the content of the fullconfig.pro file that will be generated.

If there is a possibility to share a sample or project that reproduces the issue, that would be highly appreciated but I understand this can be difficult to do.

Disabling obfuscation is possible by setting -dontobfuscate in your config file, in that case ProGuard will only shrink and optimize the code.

Kind regards,

Jonas

I fix it by separating primary input jar from other ones.

        val allJarsRegex = ".jar".r

		Proguard / proguardInputFilter := { file =>
			file.name match {
				case "myMainJar.jar" => {
					Some("!META-INF/**")
				}
				case str if allJarsRegex.findFirstIn(str).isDefined => {
					Some("!META-INF/**,org/**")
				}
				case _ => None
			}
		}

Thank you !

2 Likes

Wonderful, thanks for sharing!