2023-05-31 10:16:33,097 UTC [680449] INFO esmvalcore._main:495 ______________________________________________________________________ _____ ____ __ ____ __ _ _____ _ | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | | | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| | | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | | |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_| ______________________________________________________________________ ESMValTool - Earth System Model Evaluation Tool. http://www.esmvaltool.org CORE DEVELOPMENT TEAM AND CONTACTS: Birgit Hassler (Co-PI; DLR, Germany - birgit.hassler@dlr.de) Alistair Sellar (Co-PI; Met Office, UK - alistair.sellar@metoffice.gov.uk) Bouwe Andela (Netherlands eScience Center, The Netherlands - b.andela@esciencecenter.nl) Lee de Mora (PML, UK - ledm@pml.ac.uk) Niels Drost (Netherlands eScience Center, The Netherlands - n.drost@esciencecenter.nl) Veronika Eyring (DLR, Germany - veronika.eyring@dlr.de) Bettina Gier (UBremen, Germany - gier@uni-bremen.de) Remi Kazeroni (DLR, Germany - remi.kazeroni@dlr.de) Nikolay Koldunov (AWI, Germany - nikolay.koldunov@awi.de) Axel Lauer (DLR, Germany - axel.lauer@dlr.de) Saskia Loosveldt-Tomas (BSC, Spain - saskia.loosveldt@bsc.es) Ruth Lorenz (ETH Zurich, Switzerland - ruth.lorenz@env.ethz.ch) Benjamin Mueller (LMU, Germany - b.mueller@iggf.geo.uni-muenchen.de) Valeriu Predoi (URead, UK - valeriu.predoi@ncas.ac.uk) Mattia Righi (DLR, Germany - mattia.righi@dlr.de) Manuel Schlund (DLR, Germany - manuel.schlund@dlr.de) Breixo Solino Fernandez (DLR, Germany - breixo.solinofernandez@dlr.de) Javier Vegas-Regidor (BSC, Spain - javier.vegas@bsc.es) Klaus Zimmermann (SMHI, Sweden - klaus.zimmermann@smhi.se) For further help, please read the documentation at http://docs.esmvaltool.org. Have fun! 2023-05-31 10:16:33,097 UTC [680449] INFO esmvalcore._main:496 Package versions 2023-05-31 10:16:33,097 UTC [680449] INFO esmvalcore._main:497 ---------------- 2023-05-31 10:16:33,097 UTC [680449] INFO esmvalcore._main:498 ESMValCore: 2.8.0 2023-05-31 10:16:33,097 UTC [680449] INFO esmvalcore._main:500 ESMValTool: 2.8.0 2023-05-31 10:16:33,097 UTC [680449] INFO esmvalcore._main:501 ---------------- 2023-05-31 10:16:33,097 UTC [680449] INFO esmvalcore._main:502 Using config file /nird/home/adagj/.esmvaltool/config-user.yml 2023-05-31 10:16:33,097 UTC [680449] INFO esmvalcore._main:503 Writing program log files to: /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/run/main_log.txt /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/run/main_log_debug.txt 2023-05-31 10:16:33,208 UTC [680449] INFO esmvalcore.config._esgf_pyclient:116 Using default ESGF configuration, configuration file /nird/home/adagj/.esmvaltool/esgf-pyclient.yml not present. 2023-05-31 10:16:33,208 UTC [680449] WARNING esmvalcore.config._esgf_pyclient:181 ESGF credentials missing, only data that is accessible without logging in will be available. See https://esgf.github.io/esgf-user-support/user_guide.html for instructions on how to create an account if you do not have one yet. Next, configure your system so esmvaltool can use your credentials. This can be done using the keyring package, or you can just enter them in /nird/home/adagj/.esmvaltool/esgf-pyclient.yml. keyring ======= First install the keyring package (requires a supported backend, see https://pypi.org/project/keyring/): $ pip install keyring Next, set your username and password by running the commands: $ keyring set ESGF hostname $ keyring set ESGF username $ keyring set ESGF password To check that you entered your credentials correctly, run: $ keyring get ESGF hostname $ keyring get ESGF username $ keyring get ESGF password configuration file ================== You can store the hostname, username, and password or your OpenID account in a plain text in the file /nird/home/adagj/.esmvaltool/esgf-pyclient.yml like this: logon: hostname: "your-hostname" username: "your-username" password: "your-password" or your can configure an interactive log in: logon: interactive: true Note that storing your password in plain text in the configuration file is less secure. On shared systems, make sure the permissions of the file are set so only you can read it, i.e. $ ls -l /nird/home/adagj/.esmvaltool/esgf-pyclient.yml shows permissions -rw-------. 2023-05-31 10:16:33,741 UTC [680449] INFO esmvalcore._main:85 Starting the Earth System Model Evaluation Tool at time: 2023-05-31 10:16:33 UTC 2023-05-31 10:16:33,741 UTC [680449] INFO esmvalcore._main:89 ---------------------------------------------------------------------- 2023-05-31 10:16:33,742 UTC [680449] INFO esmvalcore._main:90 RECIPE = /nird/home/adagj/esmvaltool_workshop/recipe_mydiag.yml 2023-05-31 10:16:33,742 UTC [680449] INFO esmvalcore._main:91 RUNDIR = /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/run 2023-05-31 10:16:33,742 UTC [680449] INFO esmvalcore._main:92 WORKDIR = /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/work 2023-05-31 10:16:33,742 UTC [680449] INFO esmvalcore._main:93 PREPROCDIR = /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/preproc 2023-05-31 10:16:33,742 UTC [680449] INFO esmvalcore._main:94 PLOTDIR = /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/plots 2023-05-31 10:16:33,742 UTC [680449] INFO esmvalcore._main:95 ---------------------------------------------------------------------- 2023-05-31 10:16:33,742 UTC [680449] INFO esmvalcore._main:98 Running tasks using at most 2 processes 2023-05-31 10:16:33,742 UTC [680449] INFO esmvalcore._main:100 If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory. 2023-05-31 10:16:33,742 UTC [680449] INFO esmvalcore._main:103 If you experience memory problems, try reducing 'max_parallel_tasks' in your user configuration file. 2023-05-31 10:16:33,743 UTC [680449] DEBUG esmvalcore._recipe.check:52 Checking recipe against schema /diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_recipe/recipe_schema.yml 2023-05-31 10:16:33,762 UTC [680449] DEBUG esmvalcore._recipe.recipe:1014 Retrieving diagnostics from recipe 2023-05-31 10:16:33,762 UTC [680449] DEBUG esmvalcore._recipe.recipe:1042 Setting script for diagnostic map 2023-05-31 10:16:33,762 UTC [680449] INFO esmvalcore._recipe.recipe:1232 Creating tasks from recipe 2023-05-31 10:16:33,762 UTC [680449] INFO esmvalcore._recipe.recipe:1241 Creating tasks for diagnostic map 2023-05-31 10:16:33,762 UTC [680449] INFO esmvalcore._recipe.recipe:1163 Creating diagnostic task map/myscript 2023-05-31 10:16:33,763 UTC [680449] DEBUG esmvalcore._recipe.recipe:1091 Linking tasks for diagnostic map script myscript 2023-05-31 10:16:33,763 UTC [680449] INFO esmvalcore._recipe.recipe:1281 These tasks will be executed: map/myscript 2023-05-31 10:16:33,763 UTC [680449] DEBUG esmvalcore._main:120 Recipe summary: DiagnosticTask: map/myscript script: ~/esmvaltool_workshop/mydiag.py settings: {'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'log_level': 'info', 'output_file_type': 'png', 'plot_dir': '/projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/plots/map/myscript', 'profile_diagnostic': False, 'recipe': PosixPath('recipe_mydiag.yml'), 'run_dir': '/projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/run/map/myscript', 'script': 'myscript', 'version': '2.8.0', 'work_dir': '/projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/work/map/myscript'} ancestors: None 2023-05-31 10:16:33,766 UTC [680449] INFO esmvalcore._recipe.recipe:1344 Wrote recipe with version numbers and wildcards to: file:///projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/run/recipe_mydiag_filled.yml 2023-05-31 10:16:33,766 UTC [680449] DEBUG esmvalcore.esgf._download:535 All required data is available locally, not downloading anything. 2023-05-31 10:16:33,766 UTC [680449] INFO esmvalcore._task:746 Running 1 tasks using 1 processes 2023-05-31 10:16:33,796 UTC [680591] INFO esmvalcore._task:259 Starting task map/myscript in process [680591] 2023-05-31 10:16:33,798 UTC [680591] INFO esmvalcore._task:500 Running command ['/diagnostics/esmvaltool/2.8.0/bin/python', '/nird/home/adagj/esmvaltool_workshop/mydiag.py', '/projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/run/map/myscript/settings.yml'] 2023-05-31 10:16:33,799 UTC [680591] DEBUG esmvalcore._task:501 in environment {'MPLBACKEND': 'Agg'} 2023-05-31 10:16:33,799 UTC [680591] DEBUG esmvalcore._task:503 in current working directory: /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/run/map/myscript 2023-05-31 10:16:33,799 UTC [680591] INFO esmvalcore._task:504 Writing output to /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/work/map/myscript 2023-05-31 10:16:33,799 UTC [680591] INFO esmvalcore._task:505 Writing plots to /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/plots/map/myscript 2023-05-31 10:16:33,799 UTC [680591] INFO esmvalcore._task:506 Writing log to /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/run/map/myscript/log.txt 2023-05-31 10:16:33,799 UTC [680591] INFO esmvalcore._task:516 To re-run this diagnostic script, run: cd /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/run/map/myscript; MPLBACKEND="Agg" /diagnostics/esmvaltool/2.8.0/bin/python /nird/home/adagj/esmvaltool_workshop/mydiag.py /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/run/map/myscript/settings.yml 2023-05-31 10:16:33,894 UTC [680449] INFO esmvalcore._task:778 Progress: 1 tasks running, 0 tasks waiting for ancestors, 0/1 done 2023-05-31 10:16:37,811 UTC [680591] INFO esmvalcore._task:126 Maximum memory used (estimate): 0.5 GB 2023-05-31 10:16:37,812 UTC [680591] INFO esmvalcore._task:128 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2023-05-31 10:16:37,813 UTC [680591] DEBUG esmvalcore._task:589 Script ~/esmvaltool_workshop/mydiag.py completed successfully 2023-05-31 10:16:37,813 UTC [680591] WARNING esmvalcore._task:602 No provenance information was written to /projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/run/map/myscript/diagnostic_provenance.yml. Unable to record provenance for files created by diagnostic script ~/esmvaltool_workshop/mydiag.py in task map/myscript 2023-05-31 10:16:37,813 UTC [680591] INFO esmvalcore._task:264 Successfully completed task map/myscript (priority 0) in 0:00:04.017105 2023-05-31 10:16:37,899 UTC [680449] INFO esmvalcore._task:778 Progress: 0 tasks running, 0 tasks waiting for ancestors, 1/1 done 2023-05-31 10:16:37,899 UTC [680449] INFO esmvalcore._task:783 Successfully completed all tasks. 2023-05-31 10:16:37,913 UTC [680449] INFO esmvalcore._recipe.recipe:1309 Wrote recipe with version numbers and wildcards to: file:///projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/run/recipe_mydiag_filled.yml 2023-05-31 10:16:37,946 UTC [680449] INFO esmvalcore.experimental.recipe_output:205 Wrote recipe output to: file:///projects/NS9560K/www/diagnostics/esmvaltool/adagj/tmp/recipe_mydiag_20230531_101633/index.html 2023-05-31 10:16:37,946 UTC [680449] INFO esmvalcore._main:125 Ending the Earth System Model Evaluation Tool at time: 2023-05-31 10:16:37 UTC 2023-05-31 10:16:37,947 UTC [680449] INFO esmvalcore._main:128 Time for running the recipe was: 0:00:04.205510 2023-05-31 10:16:38,228 UTC [680449] INFO esmvalcore._task:126 Maximum memory used (estimate): 1.5 GB 2023-05-31 10:16:38,229 UTC [680449] INFO esmvalcore._task:128 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2023-05-31 10:16:38,229 UTC [680449] INFO esmvalcore._main:453 Run was successful