Load collection error

[ValentinaHutter]

When the product is not available a ValueError is thrown.
When the product is available, but the spatial or temporal extent is not available a ValueError is thrown again.
When the specified band name is not available a KeyError is thrown.
Therefore I check for ValueErrors and KeyErrors.

[clausmichele]

Keep in mind that there are predefined error messages in the openEO API, like https://processes.openeo.org/#load_collection
DimensionNotAvailable
Message: A dimension with the specified name does not exist.
for load_collection. You could return the standard message (if available) plus a more detailed one.

[sophieherrmann]

Fixes #50

[sophieherrmann]

@ValentinaHutter please test this change on eodc's dev instance (with two failing jobs - no data because of extent / band does not exits) and if everything works as expected this can be merged.

[clausmichele]

I don't know what's the status on this, but I did a quick test (requesting a band which is not available) and the returned error in the log is not so explicit.
@ValentinaHutter is this still on development?

[2022-03-02 09:44:51,762] {{taskinstance.py:655}} INFO - Dependencies all met for <TaskInstance: jb-2e0b3d60-6d15-4161-add1-23d9aa0bb6b1.odc_job 2022-03-02T09:44:07+00:00 [queued]> [2022-03-02 09:44:51,785] {{taskinstance.py:655}} INFO - Dependencies all met for <TaskInstance: jb-2e0b3d60-6d15-4161-add1-23d9aa0bb6b1.odc_job 2022-03-02T09:44:07+00:00 [queued]> [2022-03-02 09:44:51,785] {{taskinstance.py:866}} INFO - -------------------------------------------------------------------------------- [2022-03-02 09:44:51,785] {{taskinstance.py:867}} INFO - Starting attempt 1 of 1 [2022-03-02 09:44:51,785] {{taskinstance.py:868}} INFO - -------------------------------------------------------------------------------- [2022-03-02 09:44:51,805] {{taskinstance.py:887}} INFO - Executing <Task(PythonOperator): odc_job> on 2022-03-02T09:44:07+00:00 [2022-03-02 09:44:51,810] {{standard_task_runner.py:53}} INFO - Started process 26564 to run task [2022-03-02 09:44:51,893] {{logging_mixin.py:112}} INFO - Running %s on host %s <TaskInstance: jb-2e0b3d60-6d15-4161-add1-23d9aa0bb6b1.odc_job 2022-03-02T09:44:07+00:00 [running]> e750f3211937 [2022-03-02 09:44:57,273] {{taskinstance.py:1128}} ERROR - proj_create: Open of /opt/conda/share/proj failed proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode Traceback (most recent call last): File "/data_out/us-ecf0c7ee-8dc8-4da8-a533-e74113a3ace2/jobs/jb-2e0b3d60-6d15-4161-add1-23d9aa0bb6b1/jr-20220302T093941328506/odc_job/odc_job.py", line 11, in <module> _1_0 = oeop.load_collection(odc_cube=cube, **{'product': 'boa_sentinel_2', 'dask_chunks': {'time': 'auto', 'x': 1000, 'y': 1000}, 'x': (11.092607006368292, 11.241410608793315), 'y': (45.98649100123467, 46.09789188984943), 'time': ['2020-02-01T00:00:00Z', '2020-02-19T00:00:00Z'], 'measurements': ['B0A']}) File "/usr/local/openeo-processes-python/src/openeo_processes/utils.py", line 116, in fun_wrapper return cls_fun(*args, **kwargs) File "/usr/local/openeo-processes-python/src/openeo_processes/cubes.py", line 85, in exec_odc return odc_load_helper(odc_cube, odc_params) File "/usr/local/openeo-processes-python/src/openeo_processes/cubes.py", line 29, in odc_load_helper datacube = odc_cube.load(**params) File "/opt/conda/lib/python3.8/site-packages/datacube/api/core.py", line 416, in load measurement_dicts = datacube_product.lookup_measurements(measurements) File "/opt/conda/lib/python3.8/site-packages/datacube/model/__init__.py", line 605, in lookup_measurements return OrderedDict((m, mm[m]) for m in measurements) File "/opt/conda/lib/python3.8/site-packages/datacube/model/__init__.py", line 605, in <genexpr> return OrderedDict((m, mm[m]) for m in measurements) KeyError: 'B0A' Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 966, in _run_raw_task result = task_copy.execute(context=context) File "/usr/local/lib/python3.7/site-packages/airflow/operators/python_operator.py", line 113, in execute return_value = self.execute_callable() File "/usr/local/lib/python3.7/site-packages/airflow/operators/python_operator.py", line 118, in execute_callable return self.python_callable(*self.op_args, **self.op_kwargs) File "/usr/local/airflow/dags/dag_jb-2e0b3d60-6d15-4161-add1-23d9aa0bb6b1.py", line 44, in exec_odc_job raise RuntimeError(stderr.decode()) RuntimeError: proj_create: Open of /opt/conda/share/proj failed proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode Traceback (most recent call last): File "/data_out/us-ecf0c7ee-8dc8-4da8-a533-e74113a3ace2/jobs/jb-2e0b3d60-6d15-4161-add1-23d9aa0bb6b1/jr-20220302T093941328506/odc_job/odc_job.py", line 11, in <module> _1_0 = oeop.load_collection(odc_cube=cube, **{'product': 'boa_sentinel_2', 'dask_chunks': {'time': 'auto', 'x': 1000, 'y': 1000}, 'x': (11.092607006368292, 11.241410608793315), 'y': (45.98649100123467, 46.09789188984943), 'time': ['2020-02-01T00:00:00Z', '2020-02-19T00:00:00Z'], 'measurements': ['B0A']}) File "/usr/local/openeo-processes-python/src/openeo_processes/utils.py", line 116, in fun_wrapper return cls_fun(*args, **kwargs) File "/usr/local/openeo-processes-python/src/openeo_processes/cubes.py", line 85, in exec_odc return odc_load_helper(odc_cube, odc_params) File "/usr/local/openeo-processes-python/src/openeo_processes/cubes.py", line 29, in odc_load_helper datacube = odc_cube.load(**params) File "/opt/conda/lib/python3.8/site-packages/datacube/api/core.py", line 416, in load measurement_dicts = datacube_product.lookup_measurements(measurements) File "/opt/conda/lib/python3.8/site-packages/datacube/model/__init__.py", line 605, in lookup_measurements return OrderedDict((m, mm[m]) for m in measurements) File "/opt/conda/lib/python3.8/site-packages/datacube/model/__init__.py", line 605, in <genexpr> return OrderedDict((m, mm[m]) for m in measurements) KeyError: 'B0A' [2022-03-02 09:44:57,283] {{taskinstance.py:1185}} INFO - Marking task as FAILED.dag_id=jb-2e0b3d60-6d15-4161-add1-23d9aa0bb6b1, task_id=odc_job, execution_date=20220302T094407, start_date=20220302T094451, end_date=20220302T094457 [2022-03-02 09:45:01,736] {{logging_mixin.py:112}} INFO - [2022-03-02 09:45:01,735] {local_task_job.py:103} INFO - Task exited with return code 1

    error

    ID: 1_odc_job