Error when import/override/validate proxy with jar resource

[#1]

Background:

I have a requirement to verify a digital signature in apigee taking the request.content and consumer secret and generating a SHA256 (or SHA512 or even SHA3 256bit) hash and verify the client's hash (passed as a header). I tried to do this via javascript and the performance was absolutely horrible. For a 100k payload, the javascript policy was taking upwards of 600ms to complete, causing timeouts, etc. I was using the CryptoJS library. The benchmark for the javascript snippet outside of Apigee over 10,000 requests averages about 10ms, so we're off by just a little bit. With a more realistic payload, I'm seeing times around 6000ms

Java Solution:

I decided to write the policy in java in any hopes of satisfying my requirement. I built the policy using the bouncycastle library (development done offline, not through the UI since all of my code has to live in git). I usually use the apigee maven deploy plugin to zip up and deploy my proxies. When I place my jar inside the apiproxy/resources/java directory, I get a deployment error. I get this same error when attempting to use the management API to deploy the bundle:

{

"code": "messaging.runtime.CouldNotExtractJavaResource",

"message": "Could not extract the java resource myJar.jar",

"contexts": [],

"cause": {

"message": "/tmp/myJar.jar_tempResourceDir7895215265943120255.tmp/myJar.jar (Is a directory)",

"contexts": []

}

}

I am able to remove the jar, import the bundle, upload the jar to the the environment or org, then deploy just fine, but I'd really like to keep the jar bundled with the proxy.

I use an on-prem installation and am on version 4.15.04.00 running java 7.

[#2]

Hi @Jonathan Baney, thanks for the context. AFAIK, Maven plugin is supposed to include all files under resource/java folder. Could you check if the zip file deployed by Maven contains the jar in it? Also, could follow the steps provided my colleague @Mukundha Madhavan in this thread? Also make sure you're leveraging 1.0.1 version of the plugin.

[#3]

The problem seems to be that the jar is included with the bundle. If I deploy the jar to the organization or environment, I don't get this problem. When I bundle the jar in the apiproxy/resources/java directory, that's when I get the error.

I verified that my jar is in the zip file. I also verified that I'm running 1.0.1 of the maven plugin.

[#4]

@Birute Awasthi

[#5]

Can you please attach the bundle or send it to me directly? I can try deploying it in my org dzuluaga@apigee.com.

[birute]

Just FYI that anything attachments should be posted in an answer instead of comment. We currently have a bug that prevents from having attachments in comments.

[#7]

Diego, I'm not sure I can share my code (legal and compliance stuff). I have a workaround for the time being using the management api directly to import and then deploy the proxy. Presumably this is what the maven plugin is doing, but seems to fail at. If I get a chance, I'll debug that issue and try to provide a solution via a pull request.

[#8]

Sure. Actually Maven plugin AS IS doesn't compile and package the jar for you. We can get on a call to troubleshoot the issue on your machine. Please email me to the email above.

[#9]

Since I see more suggestions on tools, let me also recommend leveraging Apigee Grunt Plugin as it will compile, and build the bundle for you. Here are the lines that need to be uncommented to enable Java Callouts. Please make sure the source code is under java/src/** directory. https://github.com/apigeecs/apigee-deploy-grunt-plugin/blob/master/Gruntfile.js#L142-L148

And if you're still struggling with this, let's get on a call and figure it out.

[DChiesa]

Hi @Jonathan Baney

Sorry to hear you're having trouble. are you still experiencing this problem?

A couple things for you.

1st. My experience performing encryption / decryption and signature validation in Javascript vs Java is consistent with yours. The performance with implementations in Java is much much better. The CryptoJS library works but is not speedy.

2nd. The libraries bcpkix-jdk15on-1.50.jar and bcprov-jdk15on-1.50.jar are available in Apigee Edge for you. Considering this, if you are content to use that specific version number of the BC libraries, you can compile against against them, and then not include them in the uploaded bundle. I think in Maven you can mark them compile dependencies. Sorry I'm not a maven expert, but I think that's what it's called.

3rd. I use an alternative tool for uploading api proxies including proxies that use Java callouts into Edge. It is available at https://github.com/carloseberhardt/apiploy . I've never had a problem uploading a proxy with that tool, including proxies that have Java callouts that rely on multiple third party Jars. I've used the Jose-nimbus libraries for JWT, the Guava libraries from Google, JAckson libraries, JSON Schema libraries, and so on. And just this week I uploaded a Java callout that does XSLT to my own OPDK running 4.15.04.00. I'm sure this tool doesn't do all of what the maven tool does. But I've had good results with it.

4th. Finally, if you're just doing HMACs, I have a Java callout that does exactly that without relying on the BC libraries. It works nicely for SHA1 SHA256 and 384 and 512. I also have a demonstration api proxy for this. Both are Attached here. there are Readme files.

5th (bonus!) If by chance you are not just doing HMACs but are doing HttpSignature, I have another Java callout that shows THAT. let me know if you want it.

[#11]

Thanks for the response.

1. As far as CryptoJS, I benchmarked it outside of apigee over 1000 calls and it performs really well, averaging about 10ms. Inside of apigee, the performance is horrible, probably due to rhino. No biggie, since we have java to fall back on.

2. I did get things working, although not with the maven plugin. Basically, I generate the zip bundle and use the management API to import the proxy into the organization. Then I use the management API to deploy the new version and this seems to work.

While this all seems fun and dandy, we have an automated build pipeline based on the apigee maven plugin which seems to still result in the same problem as outlined in my original comment. I know that the plugin is not officially supported, and that's ok. I was more wondering if anyone else had run into the problems outlined. It was such a weird error that I had assumed someone had run into it.

This is our busy season, leading up the holidays. If I can spare a few cycles, I'll be more scientific and provide a very detailed outline of all the paths I attempted, what worked, what failed, etc.

[kurtkanaskie]

OK, cool, I think I found the right location for my issue, but unfortunately not the answer.

I'm seeing the same issue when including a jar file (hmac-edge-callout.jar from iloveapis conf) in a Jenkins build using 1.0.3 of the Apigee plugin.

Funny thing is if I download the revision we created manually in Edge and load that jar file in the Jenkins/Maven built proxy it works.

Then I noticed that the size of the jar file is different, the Jenkin/Maven jar file deployed is 11K while the original is 7K.

Could Maven be whacking the jar file in route?

@Jonathan Baney, @Dino

[#13]

Interesting. I'll have to check this out too. Since my original post, we have upgraded to 4.15.07.00 and have the same issue.

[kurtkanaskie]

Well, I don't think its Apigee Edge, we're running in public cloud. I'm pretty sure its in the Maven Plugin and specifically the copy-resources plugin because the jar file changes before Apigee plugin runs.

[#15]

I think you are on to something. I just ran my own tests and found a similar issue. I use raw maven to generate my jar in a separate build step and the jar size is 11130. After I use the apigee maven plugin to build the zip bundle, if I unzip it and look at the jar, the size is 17856, so something is definitely up.

When I run 'jar tf myJar.jar' on the jar I extracted from the zip bundle, i get:

java.util.zip.ZipException: error in opening zip file

at java.util.zip.ZipFile.open(Native Method)

at java.util.zip.ZipFile.<init>(ZipFile.java:220)

at java.util.zip.ZipFile.<init>(ZipFile.java:150)

at java.util.zip.ZipFile.<init>(ZipFile.java:121)

at sun.tools.jar.Main.list(Main.java:1060)

at sun.tools.jar.Main.run(Main.java:291)

at sun.tools.jar.Main.main(Main.java:1233)

Which is what I'm seeing during deployment.

[kurtkanaskie · Accepted solution]

I think I got it! @Jonathan Baney

I'm also using the maven-resources-plugin as shown and when I change filtering from true to false it copied my jar file without modification.

<filtering>false</filtering>

Now I just have to figure out what side effects that has.

    <plugin>
        <artifactId>maven-resources-plugin</artifactId>
        <version>3.0.1</version>
        <executions>
            <execution>
                <id>copy-resources</id>
                <phase>process-resources</phase>
                <goals>
                    <goal>copy-resources</goal>
                </goals>
                <configuration>
                    <overwrite>true</overwrite>
                    <resources>
                        <resource>
                            <!--source -->
                            <directory>${project.root.dir}</directory>
                            <filtering>false</filtering>
                            <includes>
                                <include>apiproxy/**</include>
                                <include>test/**</include>
                            </includes>
                        </resource>
                    </resources>
                    <!--target -->
                    <outputDirectory>${target.root.dir}</outputDirectory>
                </configuration>
            </execution>
        </executions>
    </plugin>

[#17]

I think if filtering is false, then maven will not attempt to do placeholder substitution in any of your files. Unfortunately, we take advantage of that. With that said, I wonder if t here is a way to use filtering with the 'true' value, but yet exclude jars. I'll have to dig into that.

[#18]

A little more research indeed shows that the filtering flag will corrupt binary files. There is an option that we can give to exclude jar files so they are not corrupted. I haven't tested it yet, but the maven docs say that this is the way to do it:

...

<configuration>

...

<nonFilteredFileExtensions>

<nonFilteredFileExtension>jar</nonFilteredFileExtension>

</nonFilteredFileExtensions>

...

</configuration>

[kurtkanaskie]

Yep, I saw that and just tested. That works!

Just an FYI, the use of filtering in maven-resources-plugin doesn't affect the replacement of values by the apigee-edge-maven-plugin that is done for policies via the top level config.json file.

But this solution is better:

<plugin>
    <artifactId>maven-resources-plugin</artifactId>
    <version>2.6</version>
    <executions>
        <execution>
            <id>copy-resources</id>
            <phase>process-resources</phase>
            <goals>
                <goal>copy-resources</goal>
            </goals>
            <configuration>
                <!-- Be sure not to filter JAR files -->
                <nonFilteredFileExtensions>
                   <nonFilteredFileExtension>jar</nonFilteredFileExtension>
                </nonFilteredFileExtensions>
                <overwrite>true</overwrite>
                <resources>
                    <resource>
                        <!--source -->
                        <directory>${project.root.dir}</directory>
                        <filtering>true</filtering>
                        <includes>
                            <include>apiproxy/**</include>
                            <include>test/**</include>
                        </includes>
                    </resource>
                </resources>
                <!--target -->
                <outputDirectory>${target.root.dir}</outputDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>

Oh an by the way, we are switching our HMAC from CryptoJS to Java.

Thanks!

FYI @Dino, @oseymen@apigee.com