2023-05-31 10:06:00,890 UTC [677370] INFO esmvalcore._main:495 ______________________________________________________________________ _____ ____ __ ____ __ _ _____ _ | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | | | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| | | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | | |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_| ______________________________________________________________________ ESMValTool - Earth System Model Evaluation Tool. http://www.esmvaltool.org CORE DEVELOPMENT TEAM AND CONTACTS: Birgit Hassler (Co-PI; DLR, Germany - birgit.hassler@dlr.de) Alistair Sellar (Co-PI; Met Office, UK - alistair.sellar@metoffice.gov.uk) Bouwe Andela (Netherlands eScience Center, The Netherlands - b.andela@esciencecenter.nl) Lee de Mora (PML, UK - ledm@pml.ac.uk) Niels Drost (Netherlands eScience Center, The Netherlands - n.drost@esciencecenter.nl) Veronika Eyring (DLR, Germany - veronika.eyring@dlr.de) Bettina Gier (UBremen, Germany - gier@uni-bremen.de) Remi Kazeroni (DLR, Germany - remi.kazeroni@dlr.de) Nikolay Koldunov (AWI, Germany - nikolay.koldunov@awi.de) Axel Lauer (DLR, Germany - axel.lauer@dlr.de) Saskia Loosveldt-Tomas (BSC, Spain - saskia.loosveldt@bsc.es) Ruth Lorenz (ETH Zurich, Switzerland - ruth.lorenz@env.ethz.ch) Benjamin Mueller (LMU, Germany - b.mueller@iggf.geo.uni-muenchen.de) Valeriu Predoi (URead, UK - valeriu.predoi@ncas.ac.uk) Mattia Righi (DLR, Germany - mattia.righi@dlr.de) Manuel Schlund (DLR, Germany - manuel.schlund@dlr.de) Breixo Solino Fernandez (DLR, Germany - breixo.solinofernandez@dlr.de) Javier Vegas-Regidor (BSC, Spain - javier.vegas@bsc.es) Klaus Zimmermann (SMHI, Sweden - klaus.zimmermann@smhi.se) For further help, please read the documentation at http://docs.esmvaltool.org. Have fun! 2023-05-31 10:06:00,891 UTC [677370] INFO esmvalcore._main:496 Package versions 2023-05-31 10:06:00,891 UTC [677370] INFO esmvalcore._main:497 ---------------- 2023-05-31 10:06:00,891 UTC [677370] INFO esmvalcore._main:498 ESMValCore: 2.8.0 2023-05-31 10:06:00,891 UTC [677370] INFO esmvalcore._main:500 ESMValTool: 2.8.0 2023-05-31 10:06:00,891 UTC [677370] INFO esmvalcore._main:501 ---------------- 2023-05-31 10:06:00,891 UTC [677370] INFO esmvalcore._main:502 Using config file /nird/home/adagj/.esmvaltool/config-user.yml 2023-05-31 10:06:00,891 UTC [677370] INFO esmvalcore._main:503 Writing program log files to: /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/run/main_log.txt /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/run/main_log_debug.txt 2023-05-31 10:06:01,004 UTC [677370] INFO esmvalcore.config._esgf_pyclient:116 Using default ESGF configuration, configuration file /nird/home/adagj/.esmvaltool/esgf-pyclient.yml not present. 2023-05-31 10:06:01,005 UTC [677370] WARNING esmvalcore.config._esgf_pyclient:181 ESGF credentials missing, only data that is accessible without logging in will be available. See https://esgf.github.io/esgf-user-support/user_guide.html for instructions on how to create an account if you do not have one yet. Next, configure your system so esmvaltool can use your credentials. This can be done using the keyring package, or you can just enter them in /nird/home/adagj/.esmvaltool/esgf-pyclient.yml. keyring ======= First install the keyring package (requires a supported backend, see https://pypi.org/project/keyring/): $ pip install keyring Next, set your username and password by running the commands: $ keyring set ESGF hostname $ keyring set ESGF username $ keyring set ESGF password To check that you entered your credentials correctly, run: $ keyring get ESGF hostname $ keyring get ESGF username $ keyring get ESGF password configuration file ================== You can store the hostname, username, and password or your OpenID account in a plain text in the file /nird/home/adagj/.esmvaltool/esgf-pyclient.yml like this: logon: hostname: "your-hostname" username: "your-username" password: "your-password" or your can configure an interactive log in: logon: interactive: true Note that storing your password in plain text in the configuration file is less secure. On shared systems, make sure the permissions of the file are set so only you can read it, i.e. $ ls -l /nird/home/adagj/.esmvaltool/esgf-pyclient.yml shows permissions -rw-------. 2023-05-31 10:06:01,591 UTC [677370] INFO esmvalcore._main:85 Starting the Earth System Model Evaluation Tool at time: 2023-05-31 10:06:01 UTC 2023-05-31 10:06:01,592 UTC [677370] INFO esmvalcore._main:89 ---------------------------------------------------------------------- 2023-05-31 10:06:01,592 UTC [677370] INFO esmvalcore._main:90 RECIPE = /nird/home/adagj/esmvaltool_workshop/recipe_mydiag.yml 2023-05-31 10:06:01,592 UTC [677370] INFO esmvalcore._main:91 RUNDIR = /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/run 2023-05-31 10:06:01,592 UTC [677370] INFO esmvalcore._main:92 WORKDIR = /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/work 2023-05-31 10:06:01,593 UTC [677370] INFO esmvalcore._main:93 PREPROCDIR = /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/preproc 2023-05-31 10:06:01,593 UTC [677370] INFO esmvalcore._main:94 PLOTDIR = /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/plots 2023-05-31 10:06:01,593 UTC [677370] INFO esmvalcore._main:95 ---------------------------------------------------------------------- 2023-05-31 10:06:01,593 UTC [677370] INFO esmvalcore._main:98 Running tasks using at most 2 processes 2023-05-31 10:06:01,593 UTC [677370] INFO esmvalcore._main:100 If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory. 2023-05-31 10:06:01,593 UTC [677370] INFO esmvalcore._main:103 If you experience memory problems, try reducing 'max_parallel_tasks' in your user configuration file. 2023-05-31 10:06:01,594 UTC [677370] DEBUG esmvalcore._recipe.check:52 Checking recipe against schema /diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_recipe/recipe_schema.yml 2023-05-31 10:06:01,612 UTC [677370] DEBUG esmvalcore._recipe.recipe:1014 Retrieving diagnostics from recipe 2023-05-31 10:06:01,613 UTC [677370] DEBUG esmvalcore._recipe.recipe:1042 Setting script for diagnostic map 2023-05-31 10:06:01,613 UTC [677370] INFO esmvalcore._recipe.recipe:1232 Creating tasks from recipe 2023-05-31 10:06:01,613 UTC [677370] INFO esmvalcore._recipe.recipe:1241 Creating tasks for diagnostic map 2023-05-31 10:06:01,613 UTC [677370] INFO esmvalcore._recipe.recipe:1163 Creating diagnostic task map/myscript 2023-05-31 10:06:01,613 UTC [677370] DEBUG esmvalcore._recipe.recipe:1091 Linking tasks for diagnostic map script myscript 2023-05-31 10:06:01,613 UTC [677370] INFO esmvalcore._recipe.recipe:1281 These tasks will be executed: map/myscript 2023-05-31 10:06:01,614 UTC [677370] DEBUG esmvalcore._main:120 Recipe summary: DiagnosticTask: map/myscript script: ~/esmvaltool_workshop/mydiag.py settings: {'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'log_level': 'info', 'output_file_type': 'png', 'plot_dir': '/projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/plots/map/myscript', 'profile_diagnostic': False, 'recipe': PosixPath('recipe_mydiag.yml'), 'run_dir': '/projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/run/map/myscript', 'script': 'myscript', 'version': '2.8.0', 'work_dir': '/projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/work/map/myscript'} ancestors: None 2023-05-31 10:06:01,617 UTC [677370] INFO esmvalcore._recipe.recipe:1344 Wrote recipe with version numbers and wildcards to: file:///projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/run/recipe_mydiag_filled.yml 2023-05-31 10:06:01,617 UTC [677370] DEBUG esmvalcore.esgf._download:535 All required data is available locally, not downloading anything. 2023-05-31 10:06:01,617 UTC [677370] INFO esmvalcore._task:746 Running 1 tasks using 1 processes 2023-05-31 10:06:01,654 UTC [677512] INFO esmvalcore._task:259 Starting task map/myscript in process [677512] 2023-05-31 10:06:01,657 UTC [677512] INFO esmvalcore._task:500 Running command ['/diagnostics/esmvaltool/2.8.0/bin/python', '/nird/home/adagj/esmvaltool_workshop/mydiag.py', '/projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/run/map/myscript/settings.yml'] 2023-05-31 10:06:01,657 UTC [677512] DEBUG esmvalcore._task:501 in environment {'MPLBACKEND': 'Agg'} 2023-05-31 10:06:01,657 UTC [677512] DEBUG esmvalcore._task:503 in current working directory: /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/run/map/myscript 2023-05-31 10:06:01,657 UTC [677512] INFO esmvalcore._task:504 Writing output to /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/work/map/myscript 2023-05-31 10:06:01,658 UTC [677512] INFO esmvalcore._task:505 Writing plots to /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/plots/map/myscript 2023-05-31 10:06:01,658 UTC [677512] INFO esmvalcore._task:506 Writing log to /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/run/map/myscript/log.txt 2023-05-31 10:06:01,658 UTC [677512] INFO esmvalcore._task:516 To re-run this diagnostic script, run: cd /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/run/map/myscript; MPLBACKEND="Agg" /diagnostics/esmvaltool/2.8.0/bin/python /nird/home/adagj/esmvaltool_workshop/mydiag.py /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/run/map/myscript/settings.yml 2023-05-31 10:06:01,753 UTC [677370] INFO esmvalcore._task:778 Progress: 1 tasks running, 0 tasks waiting for ancestors, 0/1 done 2023-05-31 10:06:05,670 UTC [677512] INFO esmvalcore._task:126 Maximum memory used (estimate): 0.5 GB 2023-05-31 10:06:05,671 UTC [677512] INFO esmvalcore._task:128 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2023-05-31 10:06:05,671 UTC [677512] DEBUG esmvalcore._task:589 Script ~/esmvaltool_workshop/mydiag.py completed successfully 2023-05-31 10:06:05,671 UTC [677512] WARNING esmvalcore._task:602 No provenance information was written to /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/run/map/myscript/diagnostic_provenance.yml. Unable to record provenance for files created by diagnostic script ~/esmvaltool_workshop/mydiag.py in task map/myscript 2023-05-31 10:06:05,672 UTC [677512] INFO esmvalcore._task:264 Successfully completed task map/myscript (priority 0) in 0:00:04.017077 2023-05-31 10:06:05,767 UTC [677370] INFO esmvalcore._task:778 Progress: 0 tasks running, 0 tasks waiting for ancestors, 1/1 done 2023-05-31 10:06:05,767 UTC [677370] INFO esmvalcore._task:783 Successfully completed all tasks. 2023-05-31 10:06:05,781 UTC [677370] INFO esmvalcore._recipe.recipe:1309 Wrote recipe with version numbers and wildcards to: file:///projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/run/recipe_mydiag_filled.yml 2023-05-31 10:06:05,814 UTC [677370] INFO esmvalcore.experimental.recipe_output:205 Wrote recipe output to: file:///projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_100600/index.html 2023-05-31 10:06:05,814 UTC [677370] INFO esmvalcore._main:125 Ending the Earth System Model Evaluation Tool at time: 2023-05-31 10:06:05 UTC 2023-05-31 10:06:05,814 UTC [677370] INFO esmvalcore._main:128 Time for running the recipe was: 0:00:04.222487 2023-05-31 10:06:06,026 UTC [677370] INFO esmvalcore._task:126 Maximum memory used (estimate): 1.3 GB 2023-05-31 10:06:06,027 UTC [677370] INFO esmvalcore._task:128 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2023-05-31 10:06:06,027 UTC [677370] INFO esmvalcore._main:453 Run was successful