2023-05-30 11:28:18,952 UTC [324632] INFO esmvalcore._main:495 ______________________________________________________________________ _____ ____ __ ____ __ _ _____ _ | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | | | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| | | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | | |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_| ______________________________________________________________________ ESMValTool - Earth System Model Evaluation Tool. http://www.esmvaltool.org CORE DEVELOPMENT TEAM AND CONTACTS: Birgit Hassler (Co-PI; DLR, Germany - birgit.hassler@dlr.de) Alistair Sellar (Co-PI; Met Office, UK - alistair.sellar@metoffice.gov.uk) Bouwe Andela (Netherlands eScience Center, The Netherlands - b.andela@esciencecenter.nl) Lee de Mora (PML, UK - ledm@pml.ac.uk) Niels Drost (Netherlands eScience Center, The Netherlands - n.drost@esciencecenter.nl) Veronika Eyring (DLR, Germany - veronika.eyring@dlr.de) Bettina Gier (UBremen, Germany - gier@uni-bremen.de) Remi Kazeroni (DLR, Germany - remi.kazeroni@dlr.de) Nikolay Koldunov (AWI, Germany - nikolay.koldunov@awi.de) Axel Lauer (DLR, Germany - axel.lauer@dlr.de) Saskia Loosveldt-Tomas (BSC, Spain - saskia.loosveldt@bsc.es) Ruth Lorenz (ETH Zurich, Switzerland - ruth.lorenz@env.ethz.ch) Benjamin Mueller (LMU, Germany - b.mueller@iggf.geo.uni-muenchen.de) Valeriu Predoi (URead, UK - valeriu.predoi@ncas.ac.uk) Mattia Righi (DLR, Germany - mattia.righi@dlr.de) Manuel Schlund (DLR, Germany - manuel.schlund@dlr.de) Breixo Solino Fernandez (DLR, Germany - breixo.solinofernandez@dlr.de) Javier Vegas-Regidor (BSC, Spain - javier.vegas@bsc.es) Klaus Zimmermann (SMHI, Sweden - klaus.zimmermann@smhi.se) For further help, please read the documentation at http://docs.esmvaltool.org. Have fun! 2023-05-30 11:28:18,952 UTC [324632] INFO esmvalcore._main:496 Package versions 2023-05-30 11:28:18,952 UTC [324632] INFO esmvalcore._main:497 ---------------- 2023-05-30 11:28:18,952 UTC [324632] INFO esmvalcore._main:498 ESMValCore: 2.8.0 2023-05-30 11:28:18,952 UTC [324632] INFO esmvalcore._main:500 ESMValTool: 2.8.0 2023-05-30 11:28:18,952 UTC [324632] INFO esmvalcore._main:501 ---------------- 2023-05-30 11:28:18,952 UTC [324632] INFO esmvalcore._main:502 Using config file /nird/home/oskaral/.esmvaltool/config-user.yml 2023-05-30 11:28:18,953 UTC [324632] INFO esmvalcore._main:503 Writing program log files to: /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/run/main_log.txt /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/run/main_log_debug.txt 2023-05-30 11:28:19,699 UTC [324632] INFO esmvalcore._main:85 Starting the Earth System Model Evaluation Tool at time: 2023-05-30 11:28:19 UTC 2023-05-30 11:28:19,699 UTC [324632] INFO esmvalcore._main:89 ---------------------------------------------------------------------- 2023-05-30 11:28:19,699 UTC [324632] INFO esmvalcore._main:90 RECIPE = /nird/home/oskaral/your-recipes/recipe_quantilebias.yml 2023-05-30 11:28:19,699 UTC [324632] INFO esmvalcore._main:91 RUNDIR = /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/run 2023-05-30 11:28:19,699 UTC [324632] INFO esmvalcore._main:92 WORKDIR = /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/work 2023-05-30 11:28:19,699 UTC [324632] INFO esmvalcore._main:93 PREPROCDIR = /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc 2023-05-30 11:28:19,700 UTC [324632] INFO esmvalcore._main:94 PLOTDIR = /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/plots 2023-05-30 11:28:19,700 UTC [324632] INFO esmvalcore._main:95 ---------------------------------------------------------------------- 2023-05-30 11:28:19,701 UTC [324632] INFO esmvalcore._main:98 Running tasks using at most 2 processes 2023-05-30 11:28:19,701 UTC [324632] INFO esmvalcore._main:100 If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory. 2023-05-30 11:28:19,701 UTC [324632] INFO esmvalcore._main:103 If you experience memory problems, try reducing 'max_parallel_tasks' in your user configuration file. 2023-05-30 11:28:19,702 UTC [324632] DEBUG esmvalcore._recipe.check:52 Checking recipe against schema /diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_recipe/recipe_schema.yml 2023-05-30 11:28:19,719 UTC [324632] DEBUG esmvalcore._recipe.to_datasets:344 Populating list of datasets for variable pr in diagnostic quantilebias 2023-05-30 11:28:19,720 UTC [324632] DEBUG esmvalcore.config._config:48 Loading extra facets from /diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/config/extra_facets/cmip5-fx.yml 2023-05-30 11:28:19,720 UTC [324632] DEBUG esmvalcore.config._config:48 Loading extra facets from /diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/config/extra_facets/cmip5-institutes.yml 2023-05-30 11:28:19,749 UTC [324632] DEBUG esmvalcore.config._config:48 Loading extra facets from /diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/config/extra_facets/cmip5-product.yml 2023-05-30 11:28:19,755 UTC [324632] WARNING esmvalcore.local:426 'CMIP5' rootpaths '/scratch/oskaral/ESGF/cmip5/output1' set in config-user.yml do not exist 2023-05-30 11:28:19,755 UTC [324632] DEBUG esmvalcore.local:458 Looking for files matching: /projects/NS9034K/CMIP5/output1/*/CanESM2/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_CanESM2_*_**.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/*/CanESM2/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_CanESM2_*_**.nc /scratch/oskaral/ESGF/cmip5/output1/*/CanESM2/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_CanESM2_*_**.nc 2023-05-30 11:28:19,758 UTC [324632] DEBUG esmvalcore._recipe.to_datasets:368 Found Dataset: pr, Amon, CMIP5, CanESM2, historical, r1i1p1, supplementaries: sftlf, fx, r0i0p0 2023-05-30 11:28:19,759 UTC [324632] DEBUG esmvalcore.local:458 Looking for files matching: /projects/NS9560K-datalake/ESGF/obs4MIPs/Tier1/GPCP-SG/sftlf_*.nc /scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/sftlf_*.nc 2023-05-30 11:28:19,759 UTC [324632] INFO esmvalcore.dataset:314 For Dataset: pr, Amon, obs4MIPs, GPCP-SG, v2.3, supplementaries: sftlf, *: ignoring supplementary variable 'sftlf', unable to expand wildcards 'version', 'institute'. 2023-05-30 11:28:19,759 UTC [324632] DEBUG esmvalcore._recipe.to_datasets:368 Found Dataset: pr, Amon, obs4MIPs, GPCP-SG, v2.3 2023-05-30 11:28:19,759 UTC [324632] DEBUG esmvalcore._recipe.recipe:1014 Retrieving diagnostics from recipe 2023-05-30 11:28:19,759 UTC [324632] DEBUG esmvalcore._recipe.recipe:1042 Setting script for diagnostic quantilebias 2023-05-30 11:28:19,760 UTC [324632] INFO esmvalcore._recipe.recipe:1232 Creating tasks from recipe 2023-05-30 11:28:19,760 UTC [324632] INFO esmvalcore._recipe.recipe:1241 Creating tasks for diagnostic quantilebias 2023-05-30 11:28:19,761 UTC [324632] INFO esmvalcore._recipe.recipe:1163 Creating diagnostic task quantilebias/main 2023-05-30 11:28:19,761 UTC [324632] INFO esmvalcore._recipe.recipe:1216 Creating preprocessor task quantilebias/pr 2023-05-30 11:28:19,761 UTC [324632] INFO esmvalcore._recipe.recipe:849 Creating preprocessor 'mask_regrid_preproc' task for variable 'pr' 2023-05-30 11:28:19,762 UTC [324632] DEBUG esmvalcore.local:458 Looking for files matching: /projects/NS9034K/CMIP5/output1/CCCMA/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc /projects/NS9034K/CMIP5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCMA/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc /scratch/oskaral/ESGF/cmip5/output1/CCCMA/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc /scratch/oskaral/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc 2023-05-30 11:28:19,765 UTC [324632] DEBUG esmvalcore.local:458 Looking for files matching: /projects/NS9034K/CMIP5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_CanESM2_historical_r0i0p0*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_CanESM2_historical_r0i0p0*.nc /scratch/oskaral/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_CanESM2_historical_r0i0p0*.nc 2023-05-30 11:28:19,767 UTC [324632] DEBUG esmvalcore._recipe.recipe:446 Using input files for variable pr of dataset CMIP5: /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc 2023-05-30 11:28:19,767 UTC [324632] INFO esmvalcore._recipe.recipe:717 Found input files for Dataset: pr, Amon, CMIP5, CanESM2, historical, r1i1p1, v20120718, supplementaries: sftlf, fx, r0i0p0, v20120410 2023-05-30 11:28:19,767 UTC [324632] WARNING esmvalcore._recipe.check:204 Preprocessor function mask_landsea works best when at least one supplementary variable of ['sftlf', 'sftof'] is defined in the recipe for Dataset: {'diagnostic': 'quantilebias', 'variable_group': 'pr', 'dataset': 'GPCP-SG', 'project': 'obs4MIPs', 'mip': 'Amon', 'short_name': 'pr', 'alias': 'obs4MIPs', 'frequency': 'mon', 'level': 'L3', 'long_name': 'Precipitation', 'modeling_realm': [], 'original_short_name': 'pr', 'preprocessor': 'mask_regrid_preproc', 'recipe_dataset_index': 1, 'reference_dataset': 'GPCP-SG', 'standard_name': 'precipitation_flux', 'tier': 1, 'timerange': '1979/2005', 'units': 'kg m-2 s-1', 'version': 'v2.3'} session: 'recipe_quantilebias_20230530_112818'. 2023-05-30 11:28:19,768 UTC [324632] DEBUG esmvalcore.local:458 Looking for files matching: /projects/NS9560K-datalake/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_*.nc /scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_*.nc 2023-05-30 11:28:19,768 UTC [324632] DEBUG esmvalcore._recipe.recipe:446 Using input files for variable pr of dataset obs4MIPs: /scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc 2023-05-30 11:28:19,768 UTC [324632] INFO esmvalcore._recipe.recipe:717 Found input files for Dataset: pr, Amon, obs4MIPs, GPCP-SG, v2.3 2023-05-30 11:28:19,770 UTC [324632] INFO esmvalcore._recipe.recipe:873 PreprocessingTask quantilebias/pr created. 2023-05-30 11:28:19,770 UTC [324632] DEBUG esmvalcore._recipe.recipe:874 PreprocessingTask quantilebias/pr will create the files: /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/obs4MIPs_GPCP-SG_pr_1979-2005.nc 2023-05-30 11:28:19,770 UTC [324632] DEBUG esmvalcore._recipe.recipe:1091 Linking tasks for diagnostic quantilebias script main 2023-05-30 11:28:19,770 UTC [324632] DEBUG esmvalcore._recipe.recipe:1100 Pattern quantilebias/pr matches ['quantilebias/pr'] 2023-05-30 11:28:19,770 UTC [324632] INFO esmvalcore._recipe.recipe:1281 These tasks will be executed: quantilebias/main, quantilebias/pr 2023-05-30 11:28:19,771 UTC [324632] DEBUG esmvalcore._main:120 Recipe summary: DiagnosticTask: quantilebias/main script: quantilebias/quantilebias.R settings: {'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'log_level': 'info', 'output_file_type': 'png', 'perc_lev': 75, 'plot_dir': '/projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/plots/quantilebias/main', 'profile_diagnostic': False, 'recipe': PosixPath('recipe_quantilebias.yml'), 'run_dir': '/projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/run/quantilebias/main', 'script': 'main', 'version': '2.8.0', 'work_dir': '/projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/work/quantilebias/main'} ancestors: PreprocessingTask: quantilebias/pr order: ['load', 'mask_landsea', 'regrid', 'remove_supplementary_variables', 'save'] PreprocessorFile: /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc input files: [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc'), LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] settings: {'load': {'callback': 'default'}, 'mask_landsea': {'mask_out': 'sea'}, 'regrid': {'scheme': 'area_weighted', 'target_grid': '2x2'}, 'remove_supplementary_variables': {}, 'save': {'compress': False, 'filename': PosixPath('/projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc')}} PreprocessorFile: /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/obs4MIPs_GPCP-SG_pr_1979-2005.nc input files: [LocalFile('/scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc')] settings: {'load': {'callback': 'default'}, 'mask_landsea': {'mask_out': 'sea'}, 'regrid': {'scheme': 'area_weighted', 'target_grid': '2x2'}, 'remove_supplementary_variables': {}, 'save': {'compress': False, 'filename': PosixPath('/projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/obs4MIPs_GPCP-SG_pr_1979-2005.nc')}} ancestors: None 2023-05-30 11:28:19,777 UTC [324632] INFO esmvalcore._recipe.recipe:1344 Wrote recipe with version numbers and wildcards to: file:///projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/run/recipe_quantilebias_filled.yml 2023-05-30 11:28:19,777 UTC [324632] INFO esmvalcore._task:746 Running 2 tasks using 2 processes 2023-05-30 11:28:19,818 UTC [324903] INFO esmvalcore._task:259 Starting task quantilebias/pr in process [324903] 2023-05-30 11:28:19,915 UTC [324632] INFO esmvalcore._task:778 Progress: 1 tasks running, 1 tasks waiting for ancestors, 0/2 done 2023-05-30 11:28:19,937 UTC [324903] DEBUG esmvalcore.preprocessor:660 Running block ['mask_landsea', 'regrid'] 2023-05-30 11:28:19,937 UTC [324903] DEBUG esmvalcore.preprocessor:667 Applying single-model steps to PreprocessorFile: /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc 2023-05-30 11:28:19,938 UTC [324903] DEBUG esmvalcore.esgf._download:535 All required data is available locally, not downloading anything. 2023-05-30 11:28:19,938 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step fix_file 2023-05-30 11:28:19,938 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'fix_file' on the data LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc') loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) output_dir = PosixPath('/projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/fixed_files/pr_Amon_CMIP5_CanESM2_historical_r1i1p1_v20120718_'), add_unique_suffix = True, preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-SG', mip = 'Amon', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r1i1p1', short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 0, alias = 'CMIP5', product = ['output1', 'output2'], institute = ['CCCma', 'CCCMA'], original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = ['atmos'], frequency = 'mon', version = 'v20120718' 2023-05-30 11:28:19,940 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] 2023-05-30 11:28:19,940 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step load 2023-05-30 11:28:19,940 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'load' on the data LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc') loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) callback = 'default' 2023-05-30 11:28:19,940 UTC [324903] DEBUG esmvalcore.preprocessor._io:152 Loading: /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc 2023-05-30 11:28:20,088 UTC [324903] DEBUG esmvalcore.preprocessor._io:182 Done with loading /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc 2023-05-30 11:28:20,089 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:20,092 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/00_load.nc 2023-05-30 11:28:20,507 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step fix_metadata 2023-05-30 11:28:20,509 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'fix_metadata' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) check_level = , preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-SG', mip = 'Amon', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r1i1p1', short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 0, alias = 'CMIP5', product = ['output1', 'output2'], institute = ['CCCma', 'CCCMA'], original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = ['atmos'], frequency = 'mon', version = 'v20120718' 2023-05-30 11:28:20,515 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:20,516 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/01_fix_metadata.nc 2023-05-30 11:28:20,710 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step concatenate 2023-05-30 11:28:20,712 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'concatenate' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) 2023-05-30 11:28:20,712 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:20,713 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/02_concatenate.nc 2023-05-30 11:28:20,899 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step cmor_check_metadata 2023-05-30 11:28:20,900 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'cmor_check_metadata' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) check_level = , cmor_table = 'CMIP5', mip = 'Amon', frequency = 'mon', short_name = 'pr' 2023-05-30 11:28:20,905 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:20,906 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/03_cmor_check_metadata.nc 2023-05-30 11:28:21,111 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step clip_timerange 2023-05-30 11:28:21,112 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'clip_timerange' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) timerange = '1979/2005' 2023-05-30 11:28:21,124 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:21,125 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/04_clip_timerange.nc 2023-05-30 11:28:21,262 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step fix_data 2023-05-30 11:28:21,263 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'fix_data' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) check_level = , preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-SG', mip = 'Amon', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r1i1p1', short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 0, alias = 'CMIP5', product = ['output1', 'output2'], institute = ['CCCma', 'CCCMA'], original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = ['atmos'], frequency = 'mon', version = 'v20120718' 2023-05-30 11:28:21,263 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:21,264 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/05_fix_data.nc 2023-05-30 11:28:21,392 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step cmor_check_data 2023-05-30 11:28:21,393 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'cmor_check_data' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) check_level = , cmor_table = 'CMIP5', mip = 'Amon', frequency = 'mon', short_name = 'pr' 2023-05-30 11:28:21,394 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:21,394 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/06_cmor_check_data.nc 2023-05-30 11:28:21,530 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step fix_file 2023-05-30 11:28:21,530 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'fix_file' on the data LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc') loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) output_dir = PosixPath('/projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/fixed_files/sftlf_fx_CMIP5_CanESM2_historical_r0i0p0_v20120410_'), add_unique_suffix = True, reference_dataset = 'GPCP-SG', mip = 'fx', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r0i0p0', short_name = 'sftlf', institute = 'CCCma', product = ['output1', 'output2'], original_short_name = 'sftlf', standard_name = 'land_area_fraction', long_name = 'Land Area Fraction', units = '%', modeling_realm = ['atmos'], frequency = 'fx', version = 'v20120410' 2023-05-30 11:28:21,530 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] 2023-05-30 11:28:21,530 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step load 2023-05-30 11:28:21,531 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'load' on the data LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc') loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) callback = 'default' 2023-05-30 11:28:21,531 UTC [324903] DEBUG esmvalcore.preprocessor._io:152 Loading: /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc 2023-05-30 11:28:21,537 UTC [324903] DEBUG esmvalcore.preprocessor._io:182 Done with loading /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc 2023-05-30 11:28:21,537 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:21,538 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/CMIP5_CanESM2_fx_historical_r0i0p0_sftlf/00_load.nc 2023-05-30 11:28:21,556 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step fix_metadata 2023-05-30 11:28:21,557 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'fix_metadata' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) check_level = , reference_dataset = 'GPCP-SG', mip = 'fx', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r0i0p0', short_name = 'sftlf', institute = 'CCCma', product = ['output1', 'output2'], original_short_name = 'sftlf', standard_name = 'land_area_fraction', long_name = 'Land Area Fraction', units = '%', modeling_realm = ['atmos'], frequency = 'fx', version = 'v20120410' 2023-05-30 11:28:21,557 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:21,558 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/CMIP5_CanESM2_fx_historical_r0i0p0_sftlf/01_fix_metadata.nc 2023-05-30 11:28:21,586 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step concatenate 2023-05-30 11:28:21,587 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'concatenate' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) 2023-05-30 11:28:21,587 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:21,587 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/CMIP5_CanESM2_fx_historical_r0i0p0_sftlf/02_concatenate.nc 2023-05-30 11:28:21,608 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step cmor_check_metadata 2023-05-30 11:28:21,609 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'cmor_check_metadata' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) check_level = , cmor_table = 'CMIP5', mip = 'fx', frequency = 'fx', short_name = 'sftlf' 2023-05-30 11:28:21,609 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:21,610 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/CMIP5_CanESM2_fx_historical_r0i0p0_sftlf/03_cmor_check_metadata.nc 2023-05-30 11:28:21,629 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step fix_data 2023-05-30 11:28:21,630 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'fix_data' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) check_level = , reference_dataset = 'GPCP-SG', mip = 'fx', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r0i0p0', short_name = 'sftlf', institute = 'CCCma', product = ['output1', 'output2'], original_short_name = 'sftlf', standard_name = 'land_area_fraction', long_name = 'Land Area Fraction', units = '%', modeling_realm = ['atmos'], frequency = 'fx', version = 'v20120410' 2023-05-30 11:28:21,630 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:21,631 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/CMIP5_CanESM2_fx_historical_r0i0p0_sftlf/04_fix_data.nc 2023-05-30 11:28:21,649 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step cmor_check_data 2023-05-30 11:28:21,649 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'cmor_check_data' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) check_level = , cmor_table = 'CMIP5', mip = 'fx', frequency = 'fx', short_name = 'sftlf' 2023-05-30 11:28:21,650 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:21,650 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/CMIP5_CanESM2_fx_historical_r0i0p0_sftlf/05_cmor_check_data.nc 2023-05-30 11:28:21,669 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step add_supplementary_variables 2023-05-30 11:28:21,670 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'add_supplementary_variables' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc'), LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) supplementary_cubes = [] 2023-05-30 11:28:21,671 UTC [324903] DEBUG esmvalcore.preprocessor._supplementary_vars:169 Added sftlf as ancillary variable in cube of pr. 2023-05-30 11:28:21,671 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:21,680 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/07_add_supplementary_variables.nc 2023-05-30 11:28:21,846 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step mask_landsea 2023-05-30 11:28:21,857 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'mask_landsea' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc'), LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) mask_out = 'sea' 2023-05-30 11:28:21,926 UTC [324903] DEBUG esmvalcore.preprocessor._mask:129 Applying land-sea mask: sftlf 2023-05-30 11:28:21,926 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:21,938 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/08_mask_landsea.nc 2023-05-30 11:28:22,041 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step regrid 2023-05-30 11:28:22,052 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'regrid' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc'), LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) target_grid = '2x2', scheme = 'area_weighted' 2023-05-30 11:28:24,124 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:24,125 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/09_regrid.nc 2023-05-30 11:28:24,260 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step save 2023-05-30 11:28:24,262 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'save' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc'), LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) compress = False, filename = PosixPath('/projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc') 2023-05-30 11:28:24,262 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc 2023-05-30 11:28:24,448 UTC [324903] DEBUG esmvalcore.preprocessor:667 Applying single-model steps to PreprocessorFile: /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/obs4MIPs_GPCP-SG_pr_1979-2005.nc 2023-05-30 11:28:24,449 UTC [324903] DEBUG esmvalcore.esgf._download:535 All required data is available locally, not downloading anything. 2023-05-30 11:28:24,449 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step fix_file 2023-05-30 11:28:24,449 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'fix_file' on the data LocalFile('/scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc') loaded from original input file(s) [LocalFile('/scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc')] with function argument(s) output_dir = PosixPath('/projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/fixed_files/pr_Amon_obs4MIPs_GPCP-SG_v2.3_'), add_unique_suffix = True, preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-SG', mip = 'Amon', dataset = 'GPCP-SG', project = 'obs4MIPs', level = 'L3', version = 'v2.3', tier = 1, short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 1, alias = 'obs4MIPs', original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = [], frequency = 'mon' 2023-05-30 11:28:24,451 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [LocalFile('/scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc')] 2023-05-30 11:28:24,451 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step load 2023-05-30 11:28:24,451 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'load' on the data LocalFile('/scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc') loaded from original input file(s) [LocalFile('/scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc')] with function argument(s) callback = 'default' 2023-05-30 11:28:24,451 UTC [324903] DEBUG esmvalcore.preprocessor._io:152 Loading: /scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc 2023-05-30 11:28:24,473 UTC [324903] DEBUG esmvalcore.preprocessor._io:182 Done with loading /scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc 2023-05-30 11:28:24,473 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:24,474 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/obs4MIPs_GPCP-SG_pr_1979-2005/00_load.nc 2023-05-30 11:28:24,565 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step fix_metadata 2023-05-30 11:28:24,566 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'fix_metadata' on the data [] loaded from original input file(s) [LocalFile('/scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc')] with function argument(s) check_level = , preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-SG', mip = 'Amon', dataset = 'GPCP-SG', project = 'obs4MIPs', level = 'L3', version = 'v2.3', tier = 1, short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 1, alias = 'obs4MIPs', original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = [], frequency = 'mon' 2023-05-30 11:28:24,569 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:24,569 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/obs4MIPs_GPCP-SG_pr_1979-2005/01_fix_metadata.nc 2023-05-30 11:28:24,654 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step concatenate 2023-05-30 11:28:24,655 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'concatenate' on the data [] loaded from original input file(s) [LocalFile('/scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc')] with function argument(s) 2023-05-30 11:28:24,655 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:24,656 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/obs4MIPs_GPCP-SG_pr_1979-2005/02_concatenate.nc 2023-05-30 11:28:24,748 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step cmor_check_metadata 2023-05-30 11:28:24,749 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'cmor_check_metadata' on the data loaded from original input file(s) [LocalFile('/scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc')] with function argument(s) check_level = , cmor_table = 'obs4MIPs', mip = 'Amon', frequency = 'mon', short_name = 'pr' 2023-05-30 11:28:24,751 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:24,752 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/obs4MIPs_GPCP-SG_pr_1979-2005/03_cmor_check_metadata.nc 2023-05-30 11:28:24,852 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step clip_timerange 2023-05-30 11:28:24,852 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'clip_timerange' on the data loaded from original input file(s) [LocalFile('/scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc')] with function argument(s) timerange = '1979/2005' 2023-05-30 11:28:24,857 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:24,857 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/obs4MIPs_GPCP-SG_pr_1979-2005/04_clip_timerange.nc 2023-05-30 11:28:24,917 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step fix_data 2023-05-30 11:28:24,918 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'fix_data' on the data loaded from original input file(s) [LocalFile('/scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc')] with function argument(s) check_level = , preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-SG', mip = 'Amon', dataset = 'GPCP-SG', project = 'obs4MIPs', level = 'L3', version = 'v2.3', tier = 1, short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 1, alias = 'obs4MIPs', original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = [], frequency = 'mon' 2023-05-30 11:28:24,918 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:24,919 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/obs4MIPs_GPCP-SG_pr_1979-2005/05_fix_data.nc 2023-05-30 11:28:24,971 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step cmor_check_data 2023-05-30 11:28:24,972 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'cmor_check_data' on the data loaded from original input file(s) [LocalFile('/scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc')] with function argument(s) check_level = , cmor_table = 'obs4MIPs', mip = 'Amon', frequency = 'mon', short_name = 'pr' 2023-05-30 11:28:24,973 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:24,973 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/obs4MIPs_GPCP-SG_pr_1979-2005/06_cmor_check_data.nc 2023-05-30 11:28:25,035 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step add_supplementary_variables 2023-05-30 11:28:25,035 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'add_supplementary_variables' on the data loaded from original input file(s) [LocalFile('/scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc')] with function argument(s) supplementary_cubes = [] 2023-05-30 11:28:25,035 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:25,036 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/obs4MIPs_GPCP-SG_pr_1979-2005/07_add_supplementary_variables.nc 2023-05-30 11:28:25,091 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step mask_landsea 2023-05-30 11:28:25,092 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'mask_landsea' on the data loaded from original input file(s) [LocalFile('/scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc')] with function argument(s) mask_out = 'sea' 2023-05-30 11:28:25,092 UTC [324903] DEBUG esmvalcore.preprocessor._mask:121 Ancillary variables land/sea area fraction not found in cube. Check fx_file availability. 2023-05-30 11:28:25,093 UTC [324903] DEBUG fiona._env:315 GDAL_DATA found in environment. 2023-05-30 11:28:25,093 UTC [324903] DEBUG fiona._env:315 PROJ_DATA found in environment. 2023-05-30 11:28:25,121 UTC [324903] DEBUG fiona._env:315 GDAL_DATA found in environment. 2023-05-30 11:28:25,121 UTC [324903] DEBUG fiona._env:315 PROJ_DATA found in environment. 2023-05-30 11:28:25,121 UTC [324903] DEBUG fiona.ogrext:444 OLC_FASTSETNEXTBYINDEX: 1 2023-05-30 11:28:25,121 UTC [324903] DEBUG fiona.ogrext:444 OLC_FASTFEATURECOUNT: 1 2023-05-30 11:28:25,121 UTC [324903] DEBUG fiona.ogrext:444 Next index: 0 2023-05-30 11:28:25,121 UTC [324903] DEBUG fiona.ogrext:203 Next index: 1 2023-05-30 11:28:25,172 UTC [324903] DEBUG fiona.collection:665 Flushed buffer 2023-05-30 11:28:25,172 UTC [324903] DEBUG fiona.collection:667 Stopped session 2023-05-30 11:28:25,242 UTC [324903] DEBUG esmvalcore.preprocessor._mask:135 Applying land-sea mask from Natural Earth shapefile: /diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/preprocessor/ne_masks/ne_50m_ocean.shp 2023-05-30 11:28:25,242 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:25,243 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/obs4MIPs_GPCP-SG_pr_1979-2005/08_mask_landsea.nc 2023-05-30 11:28:25,267 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step regrid 2023-05-30 11:28:25,268 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'regrid' on the data loaded from original input file(s) [LocalFile('/scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc')] with function argument(s) target_grid = '2x2', scheme = 'area_weighted' 2023-05-30 11:28:26,393 UTC [324903] DEBUG esmvalcore.preprocessor:390 Result [] 2023-05-30 11:28:26,394 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/obs4MIPs_GPCP-SG_pr_1979-2005/09_regrid.nc 2023-05-30 11:28:26,509 UTC [324903] DEBUG esmvalcore.preprocessor:369 Running preprocessor step save 2023-05-30 11:28:26,511 UTC [324903] DEBUG esmvalcore.preprocessor:323 Running preprocessor function 'save' on the data [] loaded from original input file(s) [LocalFile('/scratch/oskaral/ESGF/obs4MIPs/Tier1/GPCP-SG/pr_GPCP-SG_L3_v2.2_197901-201312.nc')] with function argument(s) compress = False, filename = PosixPath('/projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/obs4MIPs_GPCP-SG_pr_1979-2005.nc') 2023-05-30 11:28:26,511 UTC [324903] DEBUG esmvalcore.preprocessor._io:307 Saving cubes [] to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/preproc/quantilebias/pr/obs4MIPs_GPCP-SG_pr_1979-2005.nc 2023-05-30 11:28:26,607 UTC [324903] INFO esmvalcore._task:264 Successfully completed task quantilebias/pr (priority 1) in 0:00:06.787974 2023-05-30 11:28:26,624 UTC [324632] INFO esmvalcore._task:778 Progress: 0 tasks running, 1 tasks waiting for ancestors, 1/2 done 2023-05-30 11:28:26,629 UTC [324904] INFO esmvalcore._task:259 Starting task quantilebias/main in process [324904] 2023-05-30 11:28:26,632 UTC [324904] INFO esmvalcore._task:500 Running command ['/diagnostics/esmvaltool/2.8.0/bin/Rscript', '/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvaltool/diag_scripts/quantilebias/quantilebias.R', '/projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/run/quantilebias/main/settings.yml'] 2023-05-30 11:28:26,633 UTC [324904] DEBUG esmvalcore._task:501 in environment {'diag_scripts': '/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvaltool/diag_scripts'} 2023-05-30 11:28:26,633 UTC [324904] DEBUG esmvalcore._task:503 in current working directory: /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/run/quantilebias/main 2023-05-30 11:28:26,633 UTC [324904] INFO esmvalcore._task:504 Writing output to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/work/quantilebias/main 2023-05-30 11:28:26,633 UTC [324904] INFO esmvalcore._task:505 Writing plots to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/plots/quantilebias/main 2023-05-30 11:28:26,633 UTC [324904] INFO esmvalcore._task:506 Writing log to /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/run/quantilebias/main/log.txt 2023-05-30 11:28:26,633 UTC [324904] INFO esmvalcore._task:516 To re-run this diagnostic script, run: cd /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/run/quantilebias/main; diag_scripts="/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvaltool/diag_scripts" /diagnostics/esmvaltool/2.8.0/bin/Rscript /diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvaltool/diag_scripts/quantilebias/quantilebias.R /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/run/quantilebias/main/settings.yml 2023-05-30 11:28:26,761 UTC [324632] INFO esmvalcore._task:778 Progress: 1 tasks running, 0 tasks waiting for ancestors, 1/2 done 2023-05-30 11:28:30,647 UTC [324904] INFO esmvalcore._task:126 Maximum memory used (estimate): 0.3 GB 2023-05-30 11:28:30,648 UTC [324904] INFO esmvalcore._task:128 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2023-05-30 11:28:30,648 UTC [324904] DEBUG esmvalcore._task:589 Script quantilebias/quantilebias.R completed successfully 2023-05-30 11:28:30,649 UTC [324904] DEBUG esmvalcore._task:608 Collecting provenance from /projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/run/quantilebias/main/diagnostic_provenance.yml 2023-05-30 11:28:30,740 UTC [324904] DEBUG PIL.PngImagePlugin:204 STREAM b'IHDR' 16 13 2023-05-30 11:28:30,740 UTC [324904] DEBUG PIL.PngImagePlugin:204 STREAM b'IDAT' 41 8192 2023-05-30 11:28:30,769 UTC [324904] DEBUG esmvalcore._task:683 Collecting provenance of task quantilebias/main took 0.1 seconds 2023-05-30 11:28:30,769 UTC [324904] INFO esmvalcore._task:264 Successfully completed task quantilebias/main (priority 0) in 0:00:04.139652 2023-05-30 11:28:30,867 UTC [324632] INFO esmvalcore._task:778 Progress: 0 tasks running, 0 tasks waiting for ancestors, 2/2 done 2023-05-30 11:28:30,867 UTC [324632] INFO esmvalcore._task:783 Successfully completed all tasks. 2023-05-30 11:28:30,894 UTC [324632] INFO esmvalcore._recipe.recipe:1309 Wrote recipe with version numbers and wildcards to: file:///projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/run/recipe_quantilebias_filled.yml 2023-05-30 11:28:30,956 UTC [324632] INFO esmvalcore.experimental.recipe_output:205 Wrote recipe output to: file:///projects/NS9560K/www/diagnostics/esmvaltool/oskaral/tmp/recipe_quantilebias_20230530_112818/index.html 2023-05-30 11:28:30,956 UTC [324632] INFO esmvalcore._main:125 Ending the Earth System Model Evaluation Tool at time: 2023-05-30 11:28:30 UTC 2023-05-30 11:28:30,956 UTC [324632] INFO esmvalcore._main:128 Time for running the recipe was: 0:00:11.257283 2023-05-30 11:28:30,993 UTC [324632] INFO esmvalcore._task:126 Maximum memory used (estimate): 2.2 GB 2023-05-30 11:28:30,994 UTC [324632] INFO esmvalcore._task:128 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2023-05-30 11:28:30,994 UTC [324632] INFO esmvalcore._main:453 Run was successful