I am working on a project where I will likely have a collection containing several million entities. On occasion, I will need to delete several hundred to several thousand entities based on a query. Looking at the documentation, and experimenting with a test collection in the free development environment, it appears I should be able to delete the records in groups and receive a cursor pointing to where the next delete should start. That is, I should be able to repeatedly:
DELETE "[....]/collectionname?ql=where value=deleteme&limit=100&cursor=xxxxxxx"
The limit appears to be needed in case there are a lot of records to delete - if I have too many, the DELETE does not return a body, and I do not know if I deleted everything or not. By limiting the number deleted in a single request, I am sure to get a response with a cursor if there are more records to be deleted. The first delete call does not have a cursor of course, and all subsequent calls use the cursor returned from the previous call. In theory, the cursor should be returned until no more records exist to be deleted.
Great story, however, I am seeing inconsistencies when I tried this out in the free environment. The limit clause worked as I expected, but the cursor was not always returned even if there were records that should be deleted remaining. Is it better to keep deleting until the number of deleted entities returned is zero, or is the missing cursor a bug in the free environment? Or should I not bother with the cursor at all, and just keep deleting until the deleted entities returned is empty?
Bottom line question, how can I be sure I deleted everything I should have? I will be implementing this in a node.js script.