Developer Portal giving response " An internal error has occurred,retry again" when fetching 10000 records till 5000 its working

[atulranjansit]

I am trying to fetch 10000 records(limit=10000) from Drupal developer portal but its returning " An internal error has occurred,retry again" the same request working fine till limit 5000 to 6000. please help us to resolve the issue how i can get the response for 10000 records in portal. When trying to fetch the records using curl its working fine and getting 200 response for 10000 records. I also tried streaming in smartdocs proxy(/v1/sendrequest) but not displaying complete response.

[dchiesa1]

probably there are some limits, with the browser XHR with request timeouts.

I don't think there is an easy solution.

But aside from that, maybe there should not be a solution. Maybe, solving this is a bad idea.

Do you really need to display 10000 records in a smartdoc/ portal? It seems you're using it for the wrong thing. The Smartdocs tool is intended to provide interactive documentation, not a working client for all cases of data. Just show a few example records, and done. Developers don't need to see 5000 or more records to understand how an API works. How about setting an upper threshold for the limit= parameter at 10 ?

Or maybe I am misunderstanding something.

[atulranjansit]

Hi Dino,

Thankyou so much for your reply,we have limit to 10000 that's why I was testing with 10000 limit in portal but I receved below error 502 in smartdoc proxy v1/sendrequest

{"fault":"{\"detail\":{\"errorcode\":\"protocol.http.TooBigBody\"},\"faultstring\":\"Body buffer overflow\"}"}

I tried to apply streaming also in smartdoc proxy but its giving message with Http ok 200 response without body

Not displaying complete output.

However I am getting response through curl for limit 10000 without an issue.

Regards

Atul

[dchiesa1]

Yes, ok, I understand. It's not a timeout as I guessed. It's a size overflow.

The problem is with the smartdocs proxy.

This is probably not a worthwhile thing to spend time trying to solve.

Smartdocs are not intended to serve the general case of production APIs with production payloads. Some production APIs will be ok, but in general, not a good idea.

Smartdocs are for interactive documentation.

So maybe check your requirements. Re-read my previous message.

How about setting an upper threshold at limit=10 for the smartdocs case?