Analyse cost per dataform schedule

[benjikj-cera]

Hi there,

We are doing a project on cost optimisation and wanting to try and understand how much each schedule run is costing us. This will then allow us to potentially reduce the frequency of the highest costing schedules or direct our efforts in query optimisation to specific pipelines/schedules.

We can get cost per model by looking at which table is modified, but that's too granular at this point, as we are wanting to work on amended schedules on a whole.

Any where we can get this data?

Thanks,

Benji

[MaxImbrox]

Hi @benjikj-cera,

Have you tried to understand costs using Google Pricing Calculator (legacy version and not-lagecy version) or using the official product documentation in the pricing section? 

--> https://cloud.google.com/dataform/pricing

[benjikj-cera]

Do you just mean understanding how costs work in general? Yes, we know that. We are trying to break it down so we can see (for instance) total bytes per schedule. I get it can get complicated with slots and all, but surely we can get an indication for just analysis of comparing schedules?

[davidregalado25]

I am also interested in a feature to calculate the cost per workflow execution.

--
Best regards
David Regalado
Web | Linkedin | Cloudskillsboost