INFO [872888] ______________________________________________________________________ _____ ____ __ ____ __ _ _____ _ | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | | | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| | | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | | |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_| ______________________________________________________________________ ESMValTool - Earth System Model Evaluation Tool. http://www.esmvaltool.org CORE DEVELOPMENT TEAM AND CONTACTS: Birgit Hassler (Co-PI; DLR, Germany - birgit.hassler@dlr.de) Alistair Sellar (Co-PI; Met Office, UK - alistair.sellar@metoffice.gov.uk) Bouwe Andela (Netherlands eScience Center, The Netherlands - b.andela@esciencecenter.nl) Lee de Mora (PML, UK - ledm@pml.ac.uk) Niels Drost (Netherlands eScience Center, The Netherlands - n.drost@esciencecenter.nl) Veronika Eyring (DLR, Germany - veronika.eyring@dlr.de) Bettina Gier (UBremen, Germany - gier@uni-bremen.de) Remi Kazeroni (DLR, Germany - remi.kazeroni@dlr.de) Nikolay Koldunov (AWI, Germany - nikolay.koldunov@awi.de) Axel Lauer (DLR, Germany - axel.lauer@dlr.de) Saskia Loosveldt-Tomas (BSC, Spain - saskia.loosveldt@bsc.es) Ruth Lorenz (ETH Zurich, Switzerland - ruth.lorenz@env.ethz.ch) Benjamin Mueller (LMU, Germany - b.mueller@iggf.geo.uni-muenchen.de) Valeriu Predoi (URead, UK - valeriu.predoi@ncas.ac.uk) Mattia Righi (DLR, Germany - mattia.righi@dlr.de) Manuel Schlund (DLR, Germany - manuel.schlund@dlr.de) Breixo Solino Fernandez (DLR, Germany - breixo.solinofernandez@dlr.de) Javier Vegas-Regidor (BSC, Spain - javier.vegas@bsc.es) Klaus Zimmermann (SMHI, Sweden - klaus.zimmermann@smhi.se) For further help, please read the documentation at http://docs.esmvaltool.org. Have fun! INFO [872888] Package versions INFO [872888] ---------------- INFO [872888] ESMValCore: 2.8.0 INFO [872888] ESMValTool: 2.8.0 INFO [872888] ---------------- INFO [872888] Using config file /nird/home/oyvinds/.esmvaltool/config-user.yml INFO [872888] Writing program log files to: /projects/NS9560K/www/diagnostics/esmvaltool/oyvinds/tmp/recipe_python_20230829_114202/run/main_log.txt /projects/NS9560K/www/diagnostics/esmvaltool/oyvinds/tmp/recipe_python_20230829_114202/run/main_log_debug.txt INFO [872888] Starting the Earth System Model Evaluation Tool at time: 2023-08-29 11:42:12 UTC INFO [872888] ---------------------------------------------------------------------- INFO [872888] RECIPE = /projects/NS2345K/users/oyvinds/esmval/recipe_python.yml INFO [872888] RUNDIR = /projects/NS9560K/www/diagnostics/esmvaltool/oyvinds/tmp/recipe_python_20230829_114202/run INFO [872888] WORKDIR = /projects/NS9560K/www/diagnostics/esmvaltool/oyvinds/tmp/recipe_python_20230829_114202/work INFO [872888] PREPROCDIR = /projects/NS9560K/www/diagnostics/esmvaltool/oyvinds/tmp/recipe_python_20230829_114202/preproc INFO [872888] PLOTDIR = /projects/NS9560K/www/diagnostics/esmvaltool/oyvinds/tmp/recipe_python_20230829_114202/plots INFO [872888] ---------------------------------------------------------------------- INFO [872888] Running tasks using at most 2 processes INFO [872888] If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory. INFO [872888] If you experience memory problems, try reducing 'max_parallel_tasks' in your user configuration file. WARNING [872888] 'CMIP6' rootpaths '/scratch/oyvinds/ESGF/CMIP6' set in config-user.yml do not exist WARNING [872888] 'CMIP5' rootpaths '/scratch/oyvinds/ESGF/cmip5/output1' set in config-user.yml do not exist INFO [872888] Creating tasks from recipe INFO [872888] Creating tasks for diagnostic map INFO [872888] Creating diagnostic task map/script1 INFO [872888] Creating preprocessor task map/tas INFO [872888] Creating preprocessor 'to_degrees_c' task for variable 'tas' INFO [872888] Found input files for Dataset: tas, Amon, CMIP6, BCC-ESM1, CMIP, historical, r1i1p1f1, gn, v20181214 INFO [872888] Found input files for Dataset: tas, Amon, CMIP5, bcc-csm1-1, historical, r1i1p1, v1 INFO [872888] PreprocessingTask map/tas created. INFO [872888] Creating tasks for diagnostic timeseries INFO [872888] Creating diagnostic task timeseries/script1 INFO [872888] Creating preprocessor task timeseries/tas_amsterdam INFO [872888] Creating preprocessor 'annual_mean_amsterdam' task for variable 'tas_amsterdam' INFO [872888] Found input files for Dataset: tas, Amon, CMIP6, BCC-ESM1, CMIP, historical, r1i1p1f1, gn, v20181214 INFO [872888] Found input files for Dataset: tas, Amon, CMIP5, bcc-csm1-1, historical, r1i1p1, v1 INFO [872888] PreprocessingTask timeseries/tas_amsterdam created. INFO [872888] Creating preprocessor task timeseries/tas_global INFO [872888] Creating preprocessor 'annual_mean_global' task for variable 'tas_global' INFO [872888] Found input files for Dataset: tas, Amon, CMIP6, BCC-ESM1, CMIP, historical, r1i1p1f1, gn, v20181214, supplementaries: areacella, fx, 1pctCO2, v20190613 INFO [872888] Found input files for Dataset: tas, Amon, CMIP5, bcc-csm1-1, historical, r1i1p1, v1, supplementaries: areacella, fx, 1pctCO2, r0i0p0 INFO [872888] PreprocessingTask timeseries/tas_global created. INFO [872888] These tasks will be executed: timeseries/tas_amsterdam, timeseries/tas_global, timeseries/script1, map/tas, map/script1 INFO [872888] Wrote recipe with version numbers and wildcards to: file:///projects/NS9560K/www/diagnostics/esmvaltool/oyvinds/tmp/recipe_python_20230829_114202/run/recipe_python_filled.yml INFO [872888] Running 5 tasks using 2 processes INFO [873070] Starting task timeseries/tas_amsterdam in process [873070] INFO [873069] Starting task map/tas in process [873069] INFO [872888] Progress: 2 tasks running, 3 tasks waiting for ancestors, 0/5 done INFO [873070] Extracting data for Amsterdam, Noord-Holland, Nederland (52.3730796 °N, 4.8924534 °E) INFO [873069] Successfully completed task map/tas (priority 1) in 0:00:06.718453 INFO [872888] Progress: 1 tasks running, 3 tasks waiting for ancestors, 1/5 done INFO [873069] Starting task map/script1 in process [873069] INFO [873069] Running command ['/diagnostics/esmvaltool/2.8.0/bin/python', '/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvaltool/diag_scripts/examples/diagnostic.py', '/projects/NS9560K/www/diagnostics/esmvaltool/oyvinds/tmp/recipe_python_20230829_114202/run/map/script1/settings.yml'] INFO [873069] Writing output to /projects/NS9560K/www/diagnostics/esmvaltool/oyvinds/tmp/recipe_python_20230829_114202/work/map/script1 INFO [873069] Writing plots to /projects/NS9560K/www/diagnostics/esmvaltool/oyvinds/tmp/recipe_python_20230829_114202/plots/map/script1 INFO [873069] Writing log to /projects/NS9560K/www/diagnostics/esmvaltool/oyvinds/tmp/recipe_python_20230829_114202/run/map/script1/log.txt INFO [873069] To re-run this diagnostic script, run: cd /projects/NS9560K/www/diagnostics/esmvaltool/oyvinds/tmp/recipe_python_20230829_114202/run/map/script1; MPLBACKEND="Agg" /diagnostics/esmvaltool/2.8.0/bin/python /diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvaltool/diag_scripts/examples/diagnostic.py /projects/NS9560K/www/diagnostics/esmvaltool/oyvinds/tmp/recipe_python_20230829_114202/run/map/script1/settings.yml INFO [872888] Progress: 2 tasks running, 2 tasks waiting for ancestors, 1/5 done WARNING [873070] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='nominatim.openstreetmap.org', port=443): Read timed out. (read timeout=1)")': /search?q=Amsterdam&format=json&limit=1 WARNING [873070] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='nominatim.openstreetmap.org', port=443): Read timed out. (read timeout=1)")': /search?q=Amsterdam&format=json&limit=1 ERROR [873070] Failed to run preprocessor function 'extract_location' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/BCC/bcc-csm1-1/historical/mon/atmos/Amon/r1i1p1/v1/tas/tas_Amon_bcc-csm1-1_historical_r1i1p1_185001-201212.nc')] with function argument(s) location = 'Amsterdam', scheme = 'linear' INFO [872888] Maximum memory used (estimate): 1.3 GB INFO [872888] Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. ERROR [872888] Program terminated abnormally, see stack trace below for more information: multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 536, in _make_request response = conn.getresponse() File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/urllib3/connection.py", line 454, in getresponse httplib_response = super().getresponse() File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/http/client.py", line 1375, in getresponse response.begin() File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/http/client.py", line 318, in begin version, status, reason = self._read_status() File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/http/client.py", line 279, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/socket.py", line 705, in readinto return self._sock.recv_into(b) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/ssl.py", line 1274, in recv_into return self.read(nbytes, buffer) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/ssl.py", line 1130, in read return self._sslobj.read(len, buffer) TimeoutError: The read operation timed out The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen response = self._make_request( File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 538, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 370, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='nominatim.openstreetmap.org', port=443): Read timed out. (read timeout=1) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/requests/adapters.py", line 486, in send resp = conn.urlopen( File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 874, in urlopen return self.urlopen( File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 874, in urlopen return self.urlopen( File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/urllib3/connectionpool.py", line 844, in urlopen retries = retries.increment( File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/urllib3/util/retry.py", line 515, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='nominatim.openstreetmap.org', port=443): Max retries exceeded with url: /search?q=Amsterdam&format=json&limit=1 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='nominatim.openstreetmap.org', port=443): Read timed out. (read timeout=1)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/geopy/adapters.py", line 457, in _request resp = self.session.get(url, timeout=timeout, headers=headers) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/requests/adapters.py", line 519, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='nominatim.openstreetmap.org', port=443): Max retries exceeded with url: /search?q=Amsterdam&format=json&limit=1 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='nominatim.openstreetmap.org', port=443): Read timed out. (read timeout=1)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_task.py", line 795, in _run_task output_files = task.run() File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_task.py", line 262, in run self.output_files = self._run(input_files) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/preprocessor/__init__.py", line 670, in _run product.apply(step, self.debug) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/preprocessor/__init__.py", line 470, in apply self.cubes = preprocess(self.cubes, step, File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/preprocessor/__init__.py", line 379, in preprocess result.append(_run_preproc_function(function, item, settings, File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/preprocessor/__init__.py", line 328, in _run_preproc_function return function(items, **kwargs) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/preprocessor/_regrid.py", line 364, in extract_location geolocation = geolocator.geocode(location) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/geopy/geocoders/nominatim.py", line 297, in geocode return self._call_geocoder(url, callback, timeout=timeout) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/geopy/geocoders/base.py", line 368, in _call_geocoder result = self.adapter.get_json(url, timeout=timeout, headers=req_headers) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/geopy/adapters.py", line 447, in get_json resp = self._request(url, timeout=timeout, headers=headers) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/geopy/adapters.py", line 469, in _request raise GeocoderUnavailable(message) geopy.exc.GeocoderUnavailable: HTTPSConnectionPool(host='nominatim.openstreetmap.org', port=443): Max retries exceeded with url: /search?q=Amsterdam&format=json&limit=1 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='nominatim.openstreetmap.org', port=443): Read timed out. (read timeout=1)")) """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_main.py", line 521, in run fire.Fire(ESMValTool()) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/fire/core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/fire/core.py", line 475, in _Fire component, remaining_args = _CallAndUpdateTrace( File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_main.py", line 408, in run self._run(recipe, session) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_main.py", line 450, in _run process_recipe(recipe_file=recipe, session=session) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_main.py", line 122, in process_recipe recipe.run() File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_recipe/recipe.py", line 1308, in run self.tasks.run(max_parallel_tasks=self.session['max_parallel_tasks']) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_task.py", line 724, in run self._run_parallel(max_parallel_tasks) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_task.py", line 767, in _run_parallel _copy_results(task, running[task]) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_task.py", line 790, in _copy_results task.output_files, task.products = future.get() File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/multiprocessing/pool.py", line 774, in get raise self._value File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_task.py", line 795, in _run_task output_files = task.run() File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_task.py", line 262, in run self.output_files = self._run(input_files) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/preprocessor/__init__.py", line 670, in _run product.apply(step, self.debug) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/preprocessor/__init__.py", line 470, in apply self.cubes = preprocess(self.cubes, step, File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/preprocessor/__init__.py", line 379, in preprocess result.append(_run_preproc_function(function, item, settings, File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/preprocessor/__init__.py", line 328, in _run_preproc_function return function(items, **kwargs) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/preprocessor/_regrid.py", line 364, in extract_location geolocation = geolocator.geocode(location) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/geopy/geocoders/nominatim.py", line 297, in geocode return self._call_geocoder(url, callback, timeout=timeout) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/geopy/geocoders/base.py", line 368, in _call_geocoder result = self.adapter.get_json(url, timeout=timeout, headers=req_headers) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/geopy/adapters.py", line 447, in get_json resp = self._request(url, timeout=timeout, headers=headers) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/geopy/adapters.py", line 469, in _request raise GeocoderUnavailable(message) geopy.exc.GeocoderUnavailable: HTTPSConnectionPool(host='nominatim.openstreetmap.org', port=443): Max retries exceeded with url: /search?q=Amsterdam&format=json&limit=1 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='nominatim.openstreetmap.org', port=443): Read timed out. (read timeout=1)")) INFO [872888] If you have a question or need help, please start a new discussion on https://github.com/ESMValGroup/ESMValTool/discussions If you suspect this is a bug, please open an issue on https://github.com/ESMValGroup/ESMValTool/issues To make it easier to find out what the problem is, please consider attaching the files run/recipe_*.yml and run/main_log_debug.txt from the output directory.