Enabling streaming - How it works internally

[dpatty]

Hi ,

I have to enable request and response streaming for one of the API where expected payload size might exceed 10 mb limit . 

It is clear from the documentation how to enable the streaming in proxy and target endpoint but not clear on below points .Could anyone please clarify .

Response Streaming(Backend->apigee->client) : 

                                     - enabling response streaming only in target endpoint will work?

                                     - enabling response streaming only in proxy endpoint will work?

                                     - enabling response streaming required both in proxy and target end point ?

                                     - Does enabling streaming for response required any change in client consumption code? 

Request Streaming(client->apigee->backend) : 

                                     - enabling request streaming only in target endpoint will work?

                                     - enabling request streaming only in proxy endpoint will work?

                                     - enabling request streaming required both in proxy and target end point ?

                                     - Does enabling streaming for request required any change in backend receiving code ? 

Note - Data flow should use same connection stream to transfer data direct between client and backend

Best Regards,

Patty

[nithish_02]

Enabling streaming in Google Cloud Platform (GCP) involves several steps and components that work together to allow real-time streaming of data.

Data collection: Data is collected from various sources such as IoT devices, logs, and sensor data. This data can be sent to GCP using various protocols such as HTTP, MQTT, and gRPC.

Ingestion: The collected data is then ingested into GCP using a service such as Cloud Pub/Sub. This service acts as a messaging system that allows data to be sent and received in a reliable and scalable manner.

Processing: Once the data is in GCP, it can be processed using a service such as Cloud Dataflow or Cloud Dataflow SQL. These services allow you to perform various operations on the data, such as filtering, aggregation, and transformation.

Storage: The processed data can then be stored in a service such as BigQuery or Cloud Storage for long-term retention and analysis.

Streaming: The stored data can then be streamed to other services such as Cloud Data Studio, BigQuery, or Cloud Storage for real-time analysis. This can be done using a service such as Cloud Dataflow or Cloud Dataflow SQL.

[gonzalezruben · Accepted solution]

Hi Patty!

Response Streaming(Backend->apigee->client) : 

• enabling response streaming only in target endpoint will work?

Yes, it will work for the communication between a client and Apigee.

• enabling response streaming only in proxy endpoint will work?

Yes, same as previous answer. The client must be prepared to process chunked responses.

• enabling response streaming required both in proxy and target end point ?

No, but it is recommended for most use cases.

• Does enabling streaming for response required any change in client consumption code? 

Your client must be prepared to process chunked requests or responses.

Request Streaming(client->apigee->backend) : 

• enabling request streaming only in target endpoint will work?

Yes, it will work for the communication between Apigee and your target. You can conditionally control streaming for target endpoints using route rules.

• enabling request streaming only in target endpoint will work?

Yes, it will work for the communication between a client and Apigee. The target should be prepared to process requests or responses in chunks.

• enabling request streaming required both in proxy and target end point ?

No, but it's recommended for most use cases.

• Does enabling streaming for request required any change in backend receiving code ?

Your backend must be prepared to process chunked requests or responses.