GCS Extension only provides Listfiles, but does not work for Download

Has anyone had success using the download feature of the GCS Extension?

It is not behaving like the example documented on the APIGEE site. https://docs.apigee.com/api-platform/reference/extensions/google-cloud-storage/google-cloud-storage-....

I use a extension callout to get a list of files from Google Storage. That is successful, and I am able to get the list in order to check it to see if the requested file exists (listfiles). Then I use another extension callout to retrieve the file (download). However, using an extension callout policy to download (not list) a file from Google Storage, the Output is not established, it does not get created and therefore errors when I check it for the file. The response body is empty, but the ConnectorCallout.response is loaded with some but not all of the file content.

I had APIGEE support try to help, but they have not yet identified if it is a bug or not, but no matter, it does not work as the documentation describes, and after many hours of trial and erro, we cannot use it. Maybe @Dino-at-Google will know something about this?

0 5 292
5 REPLIES 5

Let me see if I can get someone to help you....

I already had APIGEE tech try to help, over 1.5 hour call. But he was not sure if it was a bug, or sytax issues. I hoped he put in a ticket, but I do not know.

Hi @Dino Gregorich

You need to set parsed=true|false correctly. I have a proxy that does the following:

GET /files 
GCS-Ext <Output parsed="false">gcs_response</Output>
AM-response 
    <Set>
        <Payload contentType="application/json">{gcs_response}</Payload>
    </Set>

GET /urls/_filename_ 
GCS-Ext <Output parsed="false">gcs_response</Output>
AM-response 
    <Set>
        <Payload contentType="application/json">{gcs_response}</Payload>
    </Set>

GET /files/_filename_ 
GCS-Ext <Output parsed="true">gcs_response</Output>
AM-file 
    <Set>
        <!-- Response doesn't have any headers -->
        <Headers>
            <Header name="content-type">{gcs_response.header.content-type}</Header>
            <Header name="X-headers-names">{gcs_response.headers.names}</Header>
        </Headers>
        <!--<Payload contentType="application/xml">{gcs_response.content}</Payload>-->
        <Payload>{gcs_response.content}</Payload>
    </Set>
The issue I noticed, is that there is no header information, so GET /files/file.xml or GET /files/file.json needs extra work to set the correct Content-Type header.
Also note that the response size limit is 2 Meg as per: https://docs.apigee.com/api-platform/reference/limits#extensions

I can try it, but all examples show json as file type, but I am actually working with binary files (application/octet-stream), firmware files requested by headless IOT devices.

Agree, its a bit vague, according to the docs: https://docs.apigee.com/api-platform/reference/policies/extension-callout-policy#output_element the default value of parsed is "true", so given:

<Output>variable_name</Output> 

Edge returns a parsed object. The object JSON Schema is

{
"type":"object",
"properties":{
"content":{
"type":"string"
}
},
"required":["content"]
}

so you can access the content via: {variable_name.content}.

Notice no other metadata, such as content-type.
Using:

<Output parsed="false">variable_name</Output>

the {variable_name} is:

{"content":"<?xml version=\"1.0\" ..."} 

but again, no header info.