How to get DAG Task last run timestamp (start and end timestamps) dynamically during DAG execution.
Ex: We have a DAG and a TASK within it and I want to extract last run timestamp for Task (not the DAG).
I tried few options. Like below.
import sys, os
import requests
composer-agent="composer-agent"
dag_id = "DAG_ID"
task_id = "task_id"
url=f"https://{composer-agent}/api/v1/dags/{dag_id}/tasks/{task_id}"
response = requests.get(url)
print(response.json())
Though response I am getting response.status_code=200 but when I print the response.json() it error out.
but when I copy the URL manually in browser it works fine.
Error Output:
Traceback (most recent call last):
File "C:\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\requests\models.py", line 971, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\json\__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\requests\models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
LinkedIn
Twitter
