Handling Streaming API Proxies in java callout

[razor]

Hi,

I'd like to deploy a sharedflow via flowhook that includes a java callout.

I have a few questions around streaming, these are the things I have already done:

1. Got a working sharedflow that reads content and headers, applied via flowhook on all API Proxies.
2. Googled, searched here in the apigee cloud forum for other questions and answers.
3. Read documentation a few times.

The questions I have:

1. I know we are not supposed to read message content when streaming is enabled, the docs clearly state that. BUT, since I'd basically like to access all transactions, on all API Proxies, I use a flowhook which applies to all API Proxies - Regardless of streaming.
   And so, I do call message.getContent() - What is the outcome of this call on streaming API Proxies? Do I cause any harm? Do I cause any kind of buffering somewhere? Right now I am seeing empty body when streaming is enabled, which I understand is a limitation of Apigee and streaming APIs.
2. I understand that when the server responds with chunked-encoding, we can't know how many chunks are going to arrive until chunk-size 0, but what about just plain HTTP response with Content-Length header for a large file beyond 10MB? In that cause if streaming is enabled, why can't I read the content?
3. Can we still run the Java Policy on streaming APIs if we avoid calling message.getContent()?
   Will that make sure we are affecting behavior for the client and server?
4. How would we go about still using flowhooks, on all proxies, but not call message.getContent() for specific API Proxies that have streaming enabled?
5. Can we potentially read the stream ourselves, run some logic for each chunk, like counting the length, and pass over the chunk to a new stream that will be fed down the chain that we interrupted?
6. Is the behavior, limitations and practices similar to edge/x and hybrid?

One final request, can someone please elaborate a little, for clarity sake, about the implementation around streaming?

Thanks a lot and in advance!
Razor.

[arghyadas]

Great questions. First of all, the implementation is subject to change as the platform gets upgraded to support newer protocols or gets optimized for high performance, low latency and ultra high scale. For the sake of understanding, lets say when streaming is enabled it creates a pipe that allows data returned by your target server to be streamed directly to your caller/client in smaller chunks without any buffering. What this means is that a lot of the flow variables that Apigee would otherwise be able to populate at runtime, won't be populated and hence you would not have access to those. Since the response payload is not buffered, policies that operate on the payload or the response content are unavailable. The same is true for request, if you are doing request streaming. 

I understand that you intend to apply the shared flow as a flow hook in all API proxies. That would be difficult to do here given that neither flow hooks or shared flows have support for conditions whereby you can skip the javacallout or policies that are looking to tap into the request or response payload. I can think of a couple of options:

1) for your streaming api proxies, use a separate environment where this flow hook is not deployed. Instead, use a flow callout policy (supports conditional) to apply a different sharedflow that does similar things but doesn't need the payload

2) this is less than ideal but still an option. In your java callout you have conditions (using flow variables) to exclude certain environments or api proxy names so that the common logic doesn't get applied to the streaming enabled api proxies

I like the separare environment option because it also allows you to shard your api proxies by usage or function. Streaming api proxies have very different scaling and performance characteristics so that kind of logical separation is desirable.

Coming to the question of can you write your own java callout code to process the payload and gather stats or telemetry - I think it would be very expensive and somewhat sub-optimal given that its not required for functional reasons. Any foreign code that tries to do this would be less than optimal when compared to the core runtime platform which decided not to perform these tasks for performance and scalability reasons. I hope this helps. 

[razor]

Hi arghyadas!

Thanks a lot for your prompt response, I do have some follow up questions if that is ok 🙂

I can 100% understand that backend / service implementation is something that is subject to change at any point - especially for SaaS.  At the same time, we have to work with the limitations imposed by the service.  In this case - the limitations are:

• Maximum payload size without Streaming enabled.
• Strong guidance from documentation to not access payload for Streaming API Proxies.

Under these limitations, I am examining your suggestions:

• Separate environment for API Proxies where flowhooks are not applied, or rather probably a different set of flowhooks - This is problematic since the API Proxies are already deployed in production, so requires significant effort and probably down time.
• Skipping certain API Proxies using conditions in a FlowCallout - This has the downside where the list has to be maintained separately.  For example, if an existing API Proxy is marked set to streaming mode, but the list is not updated, there is an issue that may surface only after production impact is detected.
  Or, new API Proxies are created with Streaming enabled, but once again, not added to the list - similar outcome.

I would really appreciate if you could clarify if calling messageContext.getContent() will cause impact such as Buffering or other side-effects on Streaming API proxies even if the flow variable holding the content was not populated due to streaming?  Or is invoking a shred flow enough to cause problems on streaming APIs?

In long-term, it would be very beneficial to allow detecting that Streaming is enabled from a flow-variable that can be read from the Java policy (or even JS).

Thanks a lot!
Razor.