Creating an API layer over files stored in Amazon S3

Not applicable

Here is our use case:

We create a .cvs file of daily transactions information (offline commerce logs). Daily a batch is run and the files are stored in our AWS S3 server.

We now want to build an API layer over top of this, so we can query and download all, latest or by date range.

Any guidance for the best practice to set this up.

0 5 1,518
5 REPLIES 5

Dear @Mike Davie ,

I am glad to hear that you are thing about API Layer to access Amazon Services. You are right on money to manage and consume Amazon S3 services.

  • You can build API Layer using Powerful Enterprise Apigee Edge Cloud API Platform or Open Source API Platform Apigee-127.
  • Amazon S3 Provides Rest APIs. You can consume REST APIs in Apigee Edge or Apigee-127 to build API Layer.

Cheers,

Anil Sagar

We are working the Edge Cloud API Platform. So I take it we will just create the API Proxy to point to the Amazon S3 Rest APIs. Any best practices on setting up the file structures so Consumers can download all, a range or latest?

Dear @Mike Davie , Did you get a chance to look at Web API Design E-Book ?

Thanks, this was a good reason. Have now designed the API. Next step figure out the S3 connection (without knowing Node.js)

Not applicable

Hi @Mike Davie - Volos S3 is an NPM module worth a try. It essentially builds a Node.js API Proxy connected to S3. You can extend it to suit your needs. It will also provide best practices around design patterns.