Request/Reponse streaming - Performance implications.

[SurajKamal]

Dear Team,

I am dealing with a scenario where I need to send payloads or files larger than the default 10MB request payload size limit. To work around this limitation, I've implemented streaming, which allows me to successfully upload payloads exceeding 10MB in size.

However, I've noticed that as the payload size increases (for instance, when uploading 1GB files), the response times also significantly increase. This has raised a few concerns about the efficiency and scalability of my approach.

Some of the issues I'm already aware of include:

• Performance Impact: Larger payload sizes result in longer response times, which can affect the overall system performance.
• No Interception Policies: I should not attach policies that intercept the request or response payload.

I have the following questions.

• Are there any additional problems or limitations associated with using the streaming approach for large payloads that I should be aware of?
• Is there a specific payload size limit or constraint to consider when implementing the streaming approach?
• Is there a recommended best practice or alternative approach for handling large payloads effectively?

Our current business process involves users uploading substantial documents to our backend system. As part of our efforts to streamline and regulate this process, we are considering the implementation of an API management solution. It is important for us to ensure that any changes introduced do not lead to disruptions or backward compatibility issues.

I am aware of an alternative approach involving the use of 'signed URLs' for handling large payload transfers. However, this method requires a minimum of two requests, which, does not align with our operational constraints.

I'd greatly appreciate any insights, experiences, or advice from the community regarding this situation. Thank you in advance!

@dchiesa1 @dknezic 

[dchiesa1]

Are there any additional problems or limitations associated with using the streaming approach for large payloads that I should be aware of?

I can't think of any. I would say, though, that streaming 1 GB of data is ALWAYS going to take longer than streaming 10mb of data. The network is a factor with such payload sizes. Think about a network link rated at 10Gbps. That is 10 gigabits per second. Eight bits to the byte, that means 1,250 megabytes per second (MBps). Now if I want to send 10gigabytes (10GB) from one point to the other on such a network, that will require 8 seconds. That is assuming no network overhead and full network capacity. But as we know, HTTP and TCP add overhead. For TCP alone, it can be in the range of 6-8%. (cite). For HTTP, more, but hard to say how much more. So figure ~10 seconds. Now add contention on the line, and the transmission time can easily go to 20 seconds. This is just point to point networking, it has nothing to do with Apigee. With Apigee acting as a proxy, there are TWO such links: one from the client to Apigee, and another from Apigee to the upstream system. If there is network contention on either link, it can degrade the flow and cause more delay. If there is contention for other resources at the endpoints (for example the client CPU is busy and cannot send data as fast as the network would allow), then more delay. As another example, if there is "latency" on the upstream side, a time expended between receiving the payload and doing something with it, then there is more delay in the chain. There is no way to know how much delay or where it comes from, without analyzing the distributed system as it is actually provisioned.

Is there a specific payload size limit or constraint to consider when implementing the streaming approach?

The defined limit for Apigee is 10mb. Beyond which you must resort to streaming. You know about this. For larger payloads, if I were architecing the system, I would not want to flow them through Apigee. I'd want to use Apigee as a control channell and some other link as a data channel - as with signed URLs. Much more efficient, especially with large payloads.

Is there a recommended best practice or alternative approach for handling large payloads effectively?

Streaming. And use signed URLs if possible.

I am aware of an alternative approach involving the use of 'signed URLs' for handling large payload transfers. However, this method requires a minimum of two requests, which, does not align with our operational constraints.

"One request" is required, huh? I cannot imagine that's driven by the business, but maybe there is some other reason that is a requirement. You'll have to balance your desired to have "one request" against the performance implications of doing so. If I were the architect on this project I would want to test both ways to see and measure the tradeoffs. But if you require one request for some reason, then... you will simply need to optimize the actors in the chain. Minimize latency.

Do you have a target latency? Or maybe you just "want it to be fast" and you have no specific target?