CINXE.COM

<!DOCTYPE html> <html class="writer-html5" lang="en"> <head> <meta charset="utf-8" /> <meta property="article:modified_time" content="2024-03-26T20:38:25+01:00" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <title>No.214, December 2023 — Technical Documentation</title> <link rel="stylesheet" type="text/css" href="../_static/pygments.css" /> <link rel="stylesheet" type="text/css" href="../_static/css/theme.css" /> <link rel="stylesheet" type="text/css" href="../_static/copybutton.css" /> <link rel="stylesheet" type="text/css" href="../_static/custom.css" /> <link rel="shortcut icon" href="../_static/favicon.ico"/>  <script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script> <script src="../_static/jquery.js"></script> <script src="../_static/underscore.js"></script> <script src="../_static/_sphinx_javascript_frameworks_compat.js"></script> <script src="../_static/doctools.js"></script> <script src="../_static/sphinx_highlight.js"></script> <script src="../_static/clipboard.min.js"></script> <script src="../_static/copybutton.js"></script> <script src="../_static/js/theme.js"></script> <link rel="index" title="Index" href="../genindex.html" /> <link rel="search" title="Search" href="../search.html" /> <link rel="next" title="No.213, August 2023" href="213.html" /> <link rel="prev" title="No.215, April 2024" href="215.html" /> </head> <body class="wy-body-for-nav"> <div class="wy-grid-for-nav"> <nav data-toggle="wy-nav-shift" class="wy-nav-side"> <div class="wy-side-scroll"> <div class="wy-side-nav-search" style="background: #EEEEEE" > <a href="https://www.mpcdf.mpg.de"> <img src="../_static/mpcdflogo.png" class="logo" alt="Logo"/> </a> <div role="search"> <form id="rtd-search-form" class="wy-form" action="../search.html" method="get"> <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" /> <input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="area" value="default" /> </form> </div> </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu"> <ul> <li class="toctree-l1"><a class="reference internal" href="../faq/index.html">Frequently Asked Questions</a><ul> <li class="toctree-l2"><a class="reference internal" href="../faq/account.html">Account registration</a></li> <li class="toctree-l2"><a class="reference internal" href="../faq/connecting.html">Connecting</a><ul> <li class="toctree-l3"><a class="reference internal" href="../faq/connecting.html#how-can-i-log-into-the-machines-at-the-mpcdf">How can I log into the machines at the MPCDF?</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/connecting.html#what-are-the-gateway-machines-for-ssh-login">What are the gateway machines for SSH login?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/connecting.html#two-factor-authentication-2fa">Two-factor authentication (2FA)</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/connecting.html#are-ssh-keys-supported">Are SSH keys supported?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/connecting.html#can-ssh-scp-sftp-performance-be-improved">Can ssh/scp/sftp performance be improved?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/connecting.html#i-get-a-ssh-security-warning-about-a-host-key-change-when-trying-to-log-in-what-does-this-mean-and-what-should-i-do">I get a SSH security warning about a host key change when trying to log in. What does this mean and what should I do?</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/connecting.html#how-can-i-run-applications-with-graphical-user-interfaces-on-mpcdf-systems">How can I run applications with graphical user interfaces on MPCDF systems?</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/connecting.html#x11-forwarding">X11 forwarding</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/connecting.html#vnc">VNC</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/connecting.html#remote-visualization-service">Remote Visualization Service</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="../faq/2fa.html">Two-factor authentication (2FA)</a><ul> <li class="toctree-l3"><a class="reference internal" href="../faq/2fa.html#general-information-about-2fa">General information about 2FA</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#do-i-need-to-enable-2fa">Do I need to enable 2FA?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#do-i-already-have-2fa-activated">Do I already have 2FA activated?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#why-is-2fa-enforced">Why is 2FA enforced?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#what-is-a-token-otp-and-seed">What is a token, OTP, and seed?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#what-kinds-of-tokens-are-there">What kinds of tokens are there?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#why-am-i-asked-for-an-otp">Why am I asked for an OTP?</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/2fa.html#activation-of-2fa">Activation of 2FA</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#how-do-i-enable-2fa">How do I enable 2FA?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#how-do-i-enroll-and-use-an-app-token">How do I enroll and use an app token?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#what-app-do-i-need-to-install">What app do I need to install?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#how-do-i-register-my-existing-token">How do I register my existing token?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#how-do-i-enroll-and-use-a-secondary-backup-token">How do I enroll and use a secondary/backup token?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#why-can-t-i-have-both-an-app-token-and-a-hardware-token">Why can’t I have both an app token and a hardware token?</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/2fa.html#fa-tips-and-tricks">2FA Tips and Tricks</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#do-i-have-to-type-in-an-otp-every-time-i-access-the-secured-systems">Do I have to type in an OTP every time I access the secured systems?</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/2fa.html#fa-troubleshooting">2FA Troubleshooting</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#i-need-to-factory-reset-my-phone-how-can-i-preserve-my-app-token">I need to factory-reset my phone. How can I preserve my app token?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#how-do-i-transfer-my-token-to-a-new-phone">How do I transfer my token to a new phone?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#i-can-t-validate-activate-the-token-i-just-scanned-wrong-otp">I can’t validate/activate the token I just scanned (“Wrong OTP”)</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#i-can-t-log-in-to-the-selfservice-anymore">I can’t log in to the SelfService anymore</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#i-can-t-log-in-to-a-gate-machine-via-ssh">I can’t log in to a gate machine via SSH</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#i-can-t-access-the-hpc-clusters-through-vnc-anymore">I can’t access the HPC clusters through VNC anymore</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/2fa.html#hardware-and-client-support">Hardware and client support</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#how-can-i-use-my-sshfs-rsync-scp-sftp-gui-with-2fa">How can I use my sshfs/rsync/scp/sftp GUI with 2FA?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#how-can-i-use-filezilla-with-2fa">How can I use FileZilla with 2FA?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#how-to-use-2fa-on-a-phone-with-a-time-shift">How to use 2FA on a phone with a time shift?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#do-you-or-are-you-planning-to-support-fido2-u2f-yubikeys">Do you or are you planning to support FIDO2/U2F/YubiKeys?</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/2fa.html#on-security">On security</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#if-the-seed-is-stored-on-the-server-how-secure-is-it">If the seed is stored on the server, how secure is it?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/2fa.html#where-do-the-hardware-tokens-come-from-does-the-vendor-know-the-seeds">Where do the hardware tokens come from? Does the vendor know the seeds?</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="../faq/hpc_software.html">HPC Software and Applications</a><ul> <li class="toctree-l3"><a class="reference internal" href="../faq/hpc_software.html#general-questions">General questions</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#how-and-where-can-i-install-software-myself">How and where can I install software myself?</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/hpc_software.html#environment-modules">Environment modules</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#how-can-i-use-environment-modules-interactively">How can I use environment modules interactively?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#how-can-i-use-environment-modules-in-scripts">How can I use environment modules in scripts?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#how-can-i-use-environment-modules-to-avoid-absolute-paths-in-my-scripts">How can I use environment modules to avoid absolute paths in my scripts?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#examples">Examples</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#how-do-the-hierarchical-environment-modules-work">How do the hierarchical environment modules work?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#how-do-i-quickly-find-a-module">How do I quickly find a module?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#how-can-i-disable-the-mpcdf-specific-note-for-module-avail">How can I disable the “MPCDF specific note” for <code class="docutils literal notranslate"><span class="pre">module</span> <span class="pre">avail</span></code>?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#why-there-are-no-blas-lapack-modules-available-on-the-clusters">Why there are no BLAS/LAPACK modules available on the clusters?</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/hpc_software.html#compiled-languages">Compiled Languages</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#cmake">CMake</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#c-c-and-fortran">C/C++ and Fortran</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#debugging-of-c-c-and-fortran-codes">Debugging of C/C++ and Fortran codes</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/hpc_software.html#interpreted-languages">Interpreted Languages</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#python">Python</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#r">R</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#julia">Julia</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#jupyter-notebooks">Jupyter Notebooks</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#matlab">MATLAB</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/hpc_software.html#message-passing-interface-mpi-libraries-and-runtimes">Message Passing Interface (MPI) Libraries and Runtimes</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#which-mpi-implementations-are-supported">Which MPI implementations are supported?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#how-can-i-compile-and-link-my-mpi-application">How can I compile and link my MPI application?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#cmake-cannot-find-mpi-what-can-i-do">CMake cannot find MPI, what can I do?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#i-cannot-launch-my-mpi-code-using-mpirun-what-am-i-doing-wrong">I cannot launch my MPI code using <code class="docutils literal notranslate"><span class="pre">mpirun</span></code>, what am I doing wrong?</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/hpc_software.html#visualization">Visualization</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#how-can-i-combine-many-image-files-jpg-png-into-a-movie-mp4">How can I combine many image files (JPG, PNG) into a movie (MP4)?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#how-can-i-install-additional-packages-for-tex-latex-locally">How can I install additional packages for TeX/LaTeX locally?</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/hpc_software.html#gui-applications">GUI applications</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#visual-studio-code-in-combination-with-the-remote-ssh-extension-does-not-work-any-more-on-certain-hpc-clusters">Visual Studio Code (in combination with the Remote-SSH extension) does not work any more on certain HPC clusters</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_software.html#why-do-certain-gui-applications-not-work-any-more-on-the-login-nodes-of-hpc-clusters-what-are-the-alternatives">Why do certain GUI applications not work (any more) on the login nodes of HPC clusters? What are the alternatives?</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="../faq/hpc_systems.html">HPC Systems and Services</a><ul> <li class="toctree-l3"><a class="reference internal" href="../faq/hpc_systems.html#raven">Raven</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#what-are-the-recommended-compiler-optimization-flags-for-raven">What are the recommended compiler optimization flags for Raven?</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/hpc_systems.html#slurm-batch-system">Slurm batch system</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#how-do-i-submit-jobs-to-the-slurm-batch-system">How do I submit jobs to the Slurm batch system?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#can-i-submit-jobs-longer-than-24-hours-why-are-job-run-times-limited-to-24-hours">Can I submit jobs longer than 24 hours? Why are job run times limited to 24 hours?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#how-do-i-launch-an-mpi-code">How do I launch an MPI code?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#what-is-the-correct-order-of-executable-commands-and-sbatch-directives-in-a-slurm-job-script">What is the correct order of executable commands and ‘#SBATCH’ directives in a Slurm job script?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#can-i-work-interactively-for-debugging-and-development-work">Can I work interactively for debugging and development work?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#how-can-i-query-the-estimated-start-time-of-my-job">How can I query the estimated start time of my job?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#how-can-i-get-information-about-my-job-at-runtime">How can I get information about my job at runtime?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#how-can-i-get-information-about-my-job-after-it-has-finished">How can I get information about my job after it has finished?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#what-happens-when-a-running-job-is-hit-by-a-hardware-failure">What happens when a running job is hit by a hardware failure?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#how-do-i-do-pinning-correctly">How do I do pinning correctly?</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/hpc_systems.html#parallel-file-systems-gpfs">Parallel File Systems (GPFS)</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#which-file-systems-are-available-and-how-should-i-use-them">Which file systems are available and how should I use them?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#i-do-not-get-the-io-performance-i-would-expect-am-i-doing-something-wrong">I do not get the IO performance I would expect. Am I doing something wrong?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#how-can-i-grant-other-users-access-to-my-files-how-do-i-use-acls">How can I grant other users access to my files? How do I use ACLs?</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/hpc_systems.html#how-can-i-transfer-files-to-and-from-the-hpc-systems">How can I transfer files to and from the HPC systems?</a></li> <li class="toctree-l3"><a class="reference internal" href="../faq/hpc_systems.html#performance-monitoring">Performance Monitoring</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#how-can-i-find-out-about-the-performance-of-my-runs">How can I find out about the performance of my runs?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#how-can-i-stop-the-background-performance-monitoring">How can I stop the background performance monitoring?</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/hpc_systems.html#gpu-computing">GPU Computing</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#how-can-i-launch-the-nvidia-multi-process-service-mps">How can I launch the NVIDIA Multi-Process Service (MPS)?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#how-can-i-profile-my-gpu-code">How can I profile my GPU code?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#are-there-dedicated-gpu-resources-available-for-interactive-gpu-development">Are there dedicated GPU resources available for interactive GPU development?</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/hpc_systems.html#containers">Containers</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#which-container-solutions-are-supported-can-i-run-docker-containers">Which container solutions are supported? Can I run Docker containers?</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../faq/hpc_systems.html#remote-visualization">Remote Visualization</a><ul> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#how-can-i-run-visualization-tools-or-any-software-that-uses-opengl-on-mpcdf-systems">How can I run visualization tools or any software that uses OpenGL on MPCDF systems?</a></li> <li class="toctree-l4"><a class="reference internal" href="../faq/hpc_systems.html#how-can-i-get-access-to-mpcdf-remote-visualization-services">How can I get access to MPCDF remote visualization services?</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="../faq/tricks.html">Hints, Tips, and Tricks</a><ul> <li class="toctree-l3"><a class="reference internal" href="../faq/tricks.html#how-can-i-change-my-default-shell">How can I change my default shell?</a></li> <li class="toctree-l3"><a class="reference internal" href="../faq/tricks.html#how-can-i-launch-ssh-agent-in-a-terminal-session">How can I launch <code class="docutils literal notranslate"><span class="pre">ssh-agent</span></code> in a terminal session?</a></li> <li class="toctree-l3"><a class="reference internal" href="../faq/tricks.html#how-can-i-avoid-having-to-type-my-password-repeatedly-how-can-i-tunnel-through-the-gateway-machines">How can I avoid having to type my password repeatedly? How can I tunnel through the gateway machines?</a></li> <li class="toctree-l3"><a class="reference internal" href="../faq/tricks.html#how-can-i-access-the-clusters-using-a-windows-machine">How can I access the clusters using a Windows machine?</a></li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="../faq/help.html">Help, Support, and Training</a><ul> <li class="toctree-l3"><a class="reference internal" href="../faq/help.html#how-can-i-get-help-and-support">How can I get Help and Support?</a></li> <li class="toctree-l3"><a class="reference internal" href="../faq/help.html#are-you-offering-trainings-on-how-to-use-mpcdf-resources">Are you offering trainings on how to use MPCDF resources?</a></li> <li class="toctree-l3"><a class="reference internal" href="../faq/help.html#how-to-acknowledge-mpcdf">How to acknowledge MPCDF?</a></li> </ul> </li> </ul> </li> </ul> <ul> <li class="toctree-l1"><a class="reference internal" href="../doc/index.html">Documentation</a><ul> <li class="toctree-l2"><a class="reference internal" href="../doc/computing/index.html">Computing</a><ul> <li class="toctree-l3"><a class="reference internal" href="../doc/computing/overview.html">Introduction</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/overview.html#overview">Overview</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/overview.html#compute-facilities">Compute Facilities</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/computing/application-support.html">Application Support for HPC, AI and HPDA</a></li> <li class="toctree-l3"><a class="reference internal" href="../doc/computing/gateways.html">Gateway machines</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/gateways.html#login">Login</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/gateways.html#gssapi-based-logins-to-mpcdf-hosts">GSSAPI-based logins to MPCDF hosts</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/gateways.html#tunneled-access-to-mpcdf-services">Tunneled access to MPCDF services</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/computing/viper-user-guide.html">Viper User Guide</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/viper-user-guide.html#system-overview">System overview</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/viper-user-guide.html#access">Access</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/viper-user-guide.html#hardware-configuration">Hardware configuration</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/viper-user-guide.html#file-systems">File systems</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/viper-user-guide.html#software">Software</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/viper-user-guide.html#slurm-batch-system">Slurm batch system</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/viper-user-guide.html#slurm-example-batch-scripts">Slurm example batch scripts</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/viper-user-guide.html#migration-guide-for-users-coming-from-intel-based-hpc-systems">Migration guide for users coming from Intel-based HPC systems</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/computing/raven-user-guide.html">Raven User Guide</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/raven-user-guide.html#system-overview">System Overview</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/raven-user-guide.html#access">Access</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/raven-user-guide.html#hardware-configuration">Hardware configuration</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/raven-user-guide.html#file-systems">File systems</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/raven-user-guide.html#software">Software</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/raven-user-guide.html#slurm-batch-system">Slurm batch system</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/raven-user-guide.html#slurm-example-batch-scripts">Slurm example batch scripts</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/computing/clusters/index.html">Dedicated clusters</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Astronomy.html">Astronomy</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Astrophysics.html">Astrophysics</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Biochemistry.html">Biochemistry</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Biological_Cybernetics.html">Biological Cybernetics</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Biological_Intelligence.html">Biological Intelligence</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Biophysics.html">Biophysics</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Brain_Research.html">Brain Research</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Chemical_Physics_Solids.html">Chemical Physics of Solids</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Extraterrestrial_Physics.html">Extraterrestrial Physics</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/FHI_MPIE_MPIP_TALOS.html">FHI / MPIE / MPIP</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Gravitational_Physics.html">Gravitational Physics</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Gravitational_Physics_ACR.html">Gravitational Physics - ACR</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Gravitational_Physics_CRA.html">Gravitational Physics - CRA</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Iron_Research.html">Iron Research</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/MPSD_PKS_ADA.html">MPSD / PKS</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Neurobiology_of_Behavior.html">Neurobiology of Behavior – caesar</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Physics.html">Physics</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Plasma_Physics.html">Plasma Physics</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Psychiatry.html">Psychiatry</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Quantum_Optics.html">Quantum Optics</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Radioastronomy.html">Radioastronomy</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/clusters/systems/Science_of_Light.html">Science of Light</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/computing/software/index.html">Software</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/software/environment-modules.html">Environment Modules</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/software/hpc-application-packages.html">HPC Application Packages</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/software/compilers_languages.html">Compilers and languages</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/software/libraries.html">Libraries</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/software/debugging-tools.html">Debugging tools</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/software/performance-tools.html">Performance tools</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/software/mathematical-tools.html">Mathematical tools</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/software/bioinformatics.html">Bioinformatics</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/software/containers.html">Containers</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/software/data_analytics-machine_learning.html">Data Analytics / Machine Learning</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/software/vnc.html">VNC</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/computing/hpc-get-started.html">Quickstart guide to HPC</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/hpc-get-started.html#software-environment-modules">Software environment (modules)</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/hpc-get-started.html#mpi-parallel-hpc-applications">MPI parallel HPC applications</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/hpc-get-started.html#multithreaded-openmp-or-hybrid-mpi-openmp-hpc-applications">Multithreaded (OpenMP) or hybrid (MPI/OpenMP) HPC applications</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/hpc-get-started.html#execution-of-parallel-programs-via-slurm">Execution of (parallel) programs via Slurm</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/computing/performance-monitoring.html">Performance Monitoring</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/performance-monitoring.html#introduction">Introduction</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/performance-monitoring.html#pdf-performance-reports-for-users">PDF Performance Reports for Users</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/performance-monitoring.html#suspending-the-performance-monitoring-system-for-specific-jobs">Suspending the Performance Monitoring System for Specific Jobs</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/performance-monitoring.html#technical-background">Technical Background</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/performance-monitoring.html#overhead">Overhead</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/performance-monitoring.html#further-information">Further information</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/computing/training.html">Training</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/training.html#courses-and-workshops-arranged-by-or-in-collaboration-with-the-mpcdf">Courses and workshops arranged by or in collaboration with the MPCDF</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/computing/training.html#training-programmes-of-other-institutions">Training programmes of other institutions</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="../doc/data/index.html">Data</a><ul> <li class="toctree-l3"><a class="reference internal" href="../doc/data/share/index.html">DataShare: Sync and Share Service</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/data/share/datashare.html">DataShare: An Introduction</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/share/domainchange.html">Update the domain of the client</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/share/faq.html">FAQ</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/share/sync-share-clients.html">Desktop and Mobile clients</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/data/globusonline/index.html">Globus Online: Large-Scale data Transfer and Sharing</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/data/globusonline/mpcdf-datahub-and-globus-online.html">MPCDF DataHub and Globus Online</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/globusonline/datahub-file-staging.html">Staging Files to HPC systems via Globus Online</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/globusonline/datahub-staging-flow.html">Staging data via Flows</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/globusonline/go-nexus.html">GO-Nexus</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/globusonline/globus-demo-videos.html">Globus Demo Videos - Demonstrating Globus Functionality for end users</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/data/object-storage/index.html">Nexus-S3: Object Storage for data Transfer and Sharing</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/data/object-storage/nexus-s3.html">Nexus-S3</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/object-storage/publishing-data-via-s3.html">Publishing Data for public access via S3</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/data/data-transfer/index.html">Small to Medium Scale Data Transfers</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/data/data-transfer/data-transfer.html">Data Transfer: Tools & Tips</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/data-transfer/mpcdf-datahub-and-globus-online.html">MPCDF DataHub and Globus Online</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/data-transfer/sharing-large-files-with-datashare.html">Sharing Large Files with DataShare</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/data/gitlab/index.html">GitLab: Software Development</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/data/gitlab/gitlab.html">The MPCDF GitLab Instance: an introduction</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/gitlab/devop-tutorial.html">Poetry and GitLab: Devops for Python developers</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/gitlab/gitlabrunners.html">GitLab Runners for CI/CD</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/data/publication/index.html">Data Publication and Metadata Management</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/data/publication/datapublishing.html">Service: Data Repositories</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/publication/mmd.html">The MPCDF Metadata Tools: User Documentation</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/publication/mmd-dev.html">The MPCDF Metadata Tools: Developer Documentation</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/publication/metastore-documentation.html">MetaStore User Documentation</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/data/backup-archive/index.html">Backup and Archive</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/data/backup-archive/backup-archive-system.html">Backup & Archive Systems</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/backup-archive/archives.html">How to archive data</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/backup-archive/BA_HPC/index.html">Backup HPC</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/backup-archive/BA_Linux_clusters/index.html">Backup Linux Clusters</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/backup-archive/BA_AFS/index.html">Backup AFS</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/backup-archive/BA_desktops/index.html">Backup Desktops</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/data/store/index.html">Deprecated: The Andrew File System (AFS)</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/data/store/introduction-to-afs.html">Store (AFS)</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/store/introduction/index.html">Introduction</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/store/specific-technical/index.html">Specific-Technical</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/store/glossary/index.html">Glossary</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/data/store/troubleshooting/index.html">Troubleshooting</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="../doc/cloud/index.html">HPC-Cloud</a><ul> <li class="toctree-l3"><a class="reference internal" href="../doc/cloud/technical/index.html">Technical and User Documentation</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/technical/quickstart.html">Quick Start</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/technical/compute.html">Compute</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/technical/storage.html">Storage</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/technical/network.html">Network</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/technical/clients.html">Command Line Interface and Scripting</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/technical/recipes.html">Recipes</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/cloud/renting/index.html">Rental Model</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/renting/index.html#introduction">Introduction</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/renting/index.html#cost-components">Cost components</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/renting/index.html#setup-procedure-and-billing-procedure">Setup procedure and billing procedure</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/renting/index.html#available-resources">Available Resources</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/renting/index.html#compute-resources">Compute resources</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/renting/index.html#storage-resources">Storage resources</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/cloud/terms_of_use/index.html">Terms of Use</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/terms_of_use/index.html#general">General</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/terms_of_use/index.html#backup-and-recovery">Backup and Recovery</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/terms_of_use/index.html#data-privacy-and-sensitive-data">Data Privacy and Sensitive Data</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/terms_of_use/index.html#proprietary-software">Proprietary Software</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/terms_of_use/index.html#service-interventions-and-scheduled-down-times">Service Interventions and Scheduled Down times</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/terms_of_use/index.html#performance">Performance</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/terms_of_use/index.html#support">Support</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/cloud/terms_of_use/index.html#potential-sanctions">Potential Sanctions</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="../doc/visualization/index.html">Visualization</a><ul> <li class="toctree-l3"><a class="reference internal" href="../doc/visualization/index.html#support-for-the-visualization-of-scientific-data">Support for the Visualization of Scientific Data</a></li> <li class="toctree-l3"><a class="reference internal" href="../doc/visualization/index.html#remote-visualization-and-jupyter-notebook-services">Remote Visualization and Jupyter Notebook Services</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/visualization/index.html#web-interface">Web Interface</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/visualization/index.html#command-line-interface">Command line interface</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/visualization/index.html#technical-details">Technical details</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/visualization/index.html#troubleshooting">Troubleshooting</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/visualization/index.html#robin">Robin</a></li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="../doc/campus/index.html">Campus</a><ul> <li class="toctree-l3"><a class="reference internal" href="../doc/campus/software/index.html">Software</a></li> <li class="toctree-l3"><a class="reference internal" href="../doc/campus/vpn/index.html">VPN</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/campus/vpn/index.html#vpn-server-gateway">VPN Server (Gateway)</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/campus/vpn/index.html#connection-profile-group">Connection Profile (Group)</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/campus/vpn/index.html#desktop-and-notebook-windows-mac-os-x-linux">Desktop and Notebook (Windows, Mac OS X, Linux)</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/campus/vpn/index.html#linux-clients">Linux Clients</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/campus/vpn/index.html#mobile-devices-android-and-ios">Mobile Devices (Android and iOS)</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/campus/wifi/index.html">Wi-Fi</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/campus/wifi/index.html#guest-networks">Guest networks</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/campus/wifi/index.html#eduroam">Eduroam</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/campus/wifi/index.html#installation-mpcdf-mpq-staff-only">Installation (<em>MPCDF/MPQ staff only</em>)</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="../doc/campus/email/index.html">Email Configuration</a><ul> <li class="toctree-l4"><a class="reference internal" href="../doc/campus/email/electronic-mail-at-mpcdf.html">Email Service</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/campus/email/email_client_config/config.html">Email Client configuration</a></li> <li class="toctree-l4"><a class="reference internal" href="../doc/campus/email/mailing-lists.html">Mailing Lists</a></li> </ul> </li> </ul> </li> </ul> </li> </ul> <ul class="current"> <li class="toctree-l1 current"><a class="reference internal" href="index.html">Bits and Bytes</a><ul class="current"> <li class="toctree-l2"><a class="reference internal" href="216.html">No.216, August 2024</a><ul> <li class="toctree-l3"><a class="reference internal" href="216.html#high-performance-computing">High-performance Computing</a><ul> <li class="toctree-l4"><a class="reference internal" href="216.html#new-hpc-system-viper-phase-1-cpu">New HPC system <em>Viper</em> (Phase-1: CPU)</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="216.html#hpc-software-news">HPC Software News</a><ul> <li class="toctree-l4"><a class="reference internal" href="216.html#mpcdf-gitlab-module-image-to-be-discontinued-on-october-31">MPCDF GitLab <code class="docutils literal notranslate"><span class="pre">module-image</span></code> to be discontinued on October 31</a></li> <li class="toctree-l4"><a class="reference internal" href="216.html#new-module-images-available-in-mpcdf-gitlab">New module images available in MPCDF GitLab</a></li> <li class="toctree-l4"><a class="reference internal" href="216.html#nvidia-hpc-sdk-version-24-3-available-on-raven">Nvidia HPC SDK version 24.3 available on <em>Raven</em></a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="216.html#major-change-in-the-python-infrastructure-on-the-hpc-clusters">Major Change in the Python Infrastructure on the HPC Clusters</a></li> <li class="toctree-l3"><a class="reference internal" href="216.html#introducing-containerized-applications-as-part-of-the-mpcdf-software-stack">Introducing containerized Applications as Part of the MPCDF Software Stack</a></li> <li class="toctree-l3"><a class="reference internal" href="216.html#nexus-s3-object-storage-in-the-hpc-cloud-and-beyond">Nexus-S3: Object Storage in the HPC-Cloud and beyond</a><ul> <li class="toctree-l4"><a class="reference internal" href="216.html#example-use-case">Example use case</a></li> <li class="toctree-l4"><a class="reference internal" href="216.html#opt-in-via-selfservice">Opt-in via SelfService</a></li> <li class="toctree-l4"><a class="reference internal" href="216.html#object-storage-for-larger-projects">Object storage for larger projects</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="216.html#the-metastore-research-data-publication-platform">The MetaStore Research Data Publication Platform</a><ul> <li class="toctree-l4"><a class="reference internal" href="216.html#managing-data-sets-and-resources-in-metastore">Managing data sets and resources in MetaStore</a></li> <li class="toctree-l4"><a class="reference internal" href="216.html#who-can-use-it">Who can use it</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="216.html#news">News</a><ul> <li class="toctree-l4"><a class="reference internal" href="216.html#password-policy">Password policy</a></li> <li class="toctree-l4"><a class="reference internal" href="216.html#fa-for-datashare-and-gitlab">2FA for DataShare and GitLab</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="216.html#events">Events</a><ul> <li class="toctree-l4"><a class="reference internal" href="216.html#amd-gpu-workshop-hackathon-november-5-7">AMD GPU workshop & hackathon (November 5-7)</a></li> <li class="toctree-l4"><a class="reference internal" href="216.html#introduction-to-mpcdf-services-october-24">Introduction to MPCDF services (October 24)</a></li> <li class="toctree-l4"><a class="reference internal" href="216.html#meet-mpcdf">Meet MPCDF</a></li> <li class="toctree-l4"><a class="reference internal" href="216.html#mpcdf-at-garching-campus-open-doors-october-3">MPCDF at Garching Campus Open Doors (October 3)</a></li> <li class="toctree-l4"><a class="reference internal" href="216.html#hpc-cloud-workshop-september-10-12">HPC-Cloud workshop (September 10-12)</a></li> <li class="toctree-l4"><a class="reference internal" href="216.html#course-on-python-for-hpc-november-26-28">Course on “Python for HPC” (November 26-28)</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="215.html">No.215, April 2024</a><ul> <li class="toctree-l3"><a class="reference internal" href="215.html#high-performance-computing">High-performance computing</a><ul> <li class="toctree-l4"><a class="reference internal" href="215.html#licensed-software-in-slurm-comsol">Licensed software in Slurm (Comsol)</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="215.html#hpc-software-news">HPC Software News</a><ul> <li class="toctree-l4"><a class="reference internal" href="215.html#improved-workflow-for-multimer-predictions-with-alphafold2-on-raven">Improved workflow for multimer predictions with AlphaFold2 on <em>Raven</em></a></li> <li class="toctree-l4"><a class="reference internal" href="215.html#new-version-of-intel-oneapi-with-deprecation-of-ifort-compiler">New version of Intel oneAPI with deprecation of ifort compiler</a></li> <li class="toctree-l4"><a class="reference internal" href="215.html#cuda-modules-on-raven">CUDA modules on <em>Raven</em></a></li> <li class="toctree-l4"><a class="reference internal" href="215.html#new-amd-gpu-elpa-release">New AMD-GPU ELPA release</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="215.html#kubernetes-in-the-hpc-cloud">Kubernetes in the HPC-Cloud</a><ul> <li class="toctree-l4"><a class="reference internal" href="215.html#usage">Usage</a></li> <li class="toctree-l4"><a class="reference internal" href="215.html#function">Function</a></li> <li class="toctree-l4"><a class="reference internal" href="215.html#existing-applications">Existing applications</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="215.html#gitlab-graphs-diagrams">GitLab: Graphs & Diagrams</a><ul> <li class="toctree-l4"><a class="reference internal" href="215.html#alternatives">Alternatives</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="215.html#llms-meet-mpcdf">LLMs meet MPCDF</a></li> <li class="toctree-l3"><a class="reference internal" href="215.html#news-events">News & Events</a><ul> <li class="toctree-l4"><a class="reference internal" href="215.html#introduction-to-mpcdf-services">Introduction to MPCDF services</a></li> <li class="toctree-l4"><a class="reference internal" href="215.html#meet-mpcdf">Meet MPCDF</a></li> </ul> </li> </ul> </li> <li class="toctree-l2 current"><a class="current reference internal" href="#">No.214, December 2023</a><ul> <li class="toctree-l3"><a class="reference internal" href="#hpc-software-news">HPC Software News</a><ul> <li class="toctree-l4"><a class="reference internal" href="#cuda-aware-openmpi-on-raven">CUDA-aware OpenMPI on <em>Raven</em></a></li> <li class="toctree-l4"><a class="reference internal" href="#gpu-accelerated-vasp">GPU-accelerated VASP</a></li> <li class="toctree-l4"><a class="reference internal" href="#intel-oneapi-transition-from-ifort-to-ifx">Intel oneAPI: transition from ifort to ifx</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="#module-software-stacks-for-continuous-integration-pipelines-on-mpcdf-gitlab-shared-cloud-runners">Module Software Stacks for Continuous Integration Pipelines on MPCDF GitLab Shared Cloud Runners</a><ul> <li class="toctree-l4"><a class="reference internal" href="#introducing-a-novel-module-enabled-docker-image-infrastructure">Introducing a novel module-enabled Docker image infrastructure</a></li> <li class="toctree-l4"><a class="reference internal" href="#announcing-legacy-status-and-later-discontinuation-of-the-module-image">Announcing legacy status and later discontinuation of the <code class="docutils literal notranslate"><span class="pre">module-image</span></code></a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="#compressed-portable-conda-environments-for-hpc-systems">Compressed Portable Conda Environments for HPC Systems</a><ul> <li class="toctree-l4"><a class="reference internal" href="#introduction-and-motivation">Introduction and Motivation</a></li> <li class="toctree-l4"><a class="reference internal" href="#move-conda-environments-into-compressed-image-files">Move Conda environments into compressed image files</a></li> <li class="toctree-l4"><a class="reference internal" href="#basic-usage-examples">Basic usage examples</a></li> <li class="toctree-l4"><a class="reference internal" href="#limitations">Limitations</a></li> <li class="toctree-l4"><a class="reference internal" href="#availability">Availability</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="#new-features-in-the-hpc-cloud">New Features in the HPC-Cloud</a><ul> <li class="toctree-l4"><a class="reference internal" href="#expanded-menu-of-flavors-and-images">Expanded menu of flavors and images</a></li> <li class="toctree-l4"><a class="reference internal" href="#ssd-based-block-volumes">SSD-based block volumes</a></li> <li class="toctree-l4"><a class="reference internal" href="#automated-domain-name-service">Automated domain name service</a></li> <li class="toctree-l4"><a class="reference internal" href="#shared-filesystem-service">Shared filesystem service</a></li> <li class="toctree-l4"><a class="reference internal" href="#the-robin-cluster">The Robin cluster</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="#news-events">News & Events</a><ul> <li class="toctree-l4"><a class="reference internal" href="#amd-gpu-development-workshop">AMD-GPU development workshop</a></li> <li class="toctree-l4"><a class="reference internal" href="#meet-mpcdf">Meet MPCDF</a></li> <li class="toctree-l4"><a class="reference internal" href="#rda-deutschland-tagung-2024">RDA Deutschland Tagung 2024</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="213.html">No.213, August 2023</a><ul> <li class="toctree-l3"><a class="reference internal" href="213.html#high-performance-computing">High-performance Computing</a><ul> <li class="toctree-l4"><a class="reference internal" href="213.html#new-gpu-development-partition-on-raven">New GPU development partition on <em>Raven</em></a></li> <li class="toctree-l4"><a class="reference internal" href="213.html#memory-profiling-with-heaptrack">Memory profiling with heaptrack</a></li> <li class="toctree-l4"><a class="reference internal" href="213.html#new-compilers-and-libraries-intel-oneapi-2023-1">New compilers and libraries: Intel oneAPI 2023.1</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="213.html#using-linters-to-improve-and-maintain-code-quality">Using linters to improve and maintain code quality</a></li> <li class="toctree-l3"><a class="reference internal" href="213.html#hpc-cloud-object-storage">HPC-Cloud Object Storage</a></li> <li class="toctree-l3"><a class="reference internal" href="213.html#jade-automated-slurm-deployments-in-the-hpc-cloud">JADE - Automated Slurm deployments in the HPC-Cloud</a></li> <li class="toctree-l3"><a class="reference internal" href="213.html#gitlab-tips-tricks">GitLab: Tips & Tricks</a><ul> <li class="toctree-l4"><a class="reference internal" href="213.html#online-editing-of-source-code-revisited">Online editing of source code revisited</a></li> <li class="toctree-l4"><a class="reference internal" href="213.html#custom-badges">Custom badges</a></li> <li class="toctree-l4"><a class="reference internal" href="213.html#security-warning">Security warning</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="213.html#new-ibm-tape-library-and-tape-drives-installed-at-mpcdf">New IBM tape library and tape drives installed at MPCDF</a></li> <li class="toctree-l3"><a class="reference internal" href="213.html#news-events">News & Events</a><ul> <li class="toctree-l4"><a class="reference internal" href="213.html#open-positions-at-mpcdf">Open positions at MPCDF</a></li> <li class="toctree-l4"><a class="reference internal" href="213.html#meet-mpcdf">Meet MPCDF</a></li> <li class="toctree-l4"><a class="reference internal" href="213.html#amd-gpu-development-workshop">AMD-GPU development workshop</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="212.html">No.212, April 2023</a><ul> <li class="toctree-l3"><a class="reference internal" href="212.html#high-performance-computing">High-performance Computing</a><ul> <li class="toctree-l4"><a class="reference internal" href="212.html#new-supercomputer-of-the-mpg-cobra-successor">New Supercomputer of the MPG - <em>Cobra</em> successor</a></li> <li class="toctree-l4"><a class="reference internal" href="212.html#documentation-of-hpc-hardware-characteristics">Documentation of HPC hardware characteristics</a></li> <li class="toctree-l4"><a class="reference internal" href="212.html#cmake-recipes-repository">CMake Recipes Repository</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="212.html#mpcdf-hpc-cloud">MPCDF HPC Cloud</a><ul> <li class="toctree-l4"><a class="reference internal" href="212.html#introduction">Introduction</a></li> <li class="toctree-l4"><a class="reference internal" href="212.html#hardware-resources">Hardware Resources</a></li> <li class="toctree-l4"><a class="reference internal" href="212.html#project-support">Project Support</a></li> <li class="toctree-l4"><a class="reference internal" href="212.html#summary">Summary</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="212.html#news">News</a><ul> <li class="toctree-l4"><a class="reference internal" href="212.html#mpcdf-selfservice">MPCDF SelfService</a></li> <li class="toctree-l4"><a class="reference internal" href="212.html#pushing-fusion-plasma-simulations-towards-exascale">Pushing Fusion-Plasma Simulations Towards Exascale</a></li> <li class="toctree-l4"><a class="reference internal" href="212.html#base4nfdi-creating-nfdi-wide-basic-services-in-a-world-of-specific-domains">Base4NFDI: Creating NFDI-wide basic services in a world of specific domains</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="212.html#events">Events</a><ul> <li class="toctree-l4"><a class="reference internal" href="212.html#meet-mpcdf">Meet MPCDF</a></li> <li class="toctree-l4"><a class="reference internal" href="212.html#introduction-to-mpcdf-services">Introduction to MPCDF services</a></li> <li class="toctree-l4"><a class="reference internal" href="212.html#ai-training-course">AI Training Course</a></li> <li class="toctree-l4"><a class="reference internal" href="212.html#course-on-python-for-hpc">Course on “Python for HPC”</a></li> <li class="toctree-l4"><a class="reference internal" href="212.html#rda-deutschland-conference">RDA-Deutschland Conference</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="211.html">No.211, December 2022</a><ul> <li class="toctree-l3"><a class="reference internal" href="211.html#high-performance-computing">High-performance Computing</a><ul> <li class="toctree-l4"><a class="reference internal" href="211.html#anaconda-python-modules">Anaconda Python modules</a></li> <li class="toctree-l4"><a class="reference internal" href="211.html#hints-for-architecture-specific-and-optimized-cuda-compilation">Hints for architecture-specific and optimized CUDA compilation</a></li> <li class="toctree-l4"><a class="reference internal" href="211.html#new-intel-c-c-compilers-and-associated-mpcdf-software-stack">New Intel C/C++ compilers and associated MPCDF software stack</a></li> <li class="toctree-l4"><a class="reference internal" href="211.html#turbomole-license-for-mpg-renewed">Turbomole license for MPG renewed</a></li> <li class="toctree-l4"><a class="reference internal" href="211.html#apptainer-on-hpc-clusters-the-linux-foundation-successor-to-singularity">Apptainer on HPC clusters, the Linux Foundation successor to Singularity</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="211.html#gitlab-tips-tricks-use-of-docker-images-in-gitlab-ci">GitLab Tips & Tricks: Use of Docker Images in GitLab CI</a></li> <li class="toctree-l3"><a class="reference internal" href="211.html#go-nexus">GO-Nexus</a></li> <li class="toctree-l3"><a class="reference internal" href="211.html#news-events">News & Events</a><ul> <li class="toctree-l4"><a class="reference internal" href="211.html#discontinuation-of-general-vpn">Discontinuation of General VPN</a></li> <li class="toctree-l4"><a class="reference internal" href="211.html#meet-mpcdf">Meet MPCDF</a></li> <li class="toctree-l4"><a class="reference internal" href="211.html#python-for-hpc">Python for HPC</a></li> <li class="toctree-l4"><a class="reference internal" href="211.html#advanced-hpc-workshop">Advanced HPC Workshop</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="210.html">No.210, August 2022</a><ul> <li class="toctree-l3"><a class="reference internal" href="210.html#high-performance-computing">High-performance Computing</a><ul> <li class="toctree-l4"><a class="reference internal" href="210.html#cobra-successor-procurement">Cobra successor procurement</a></li> <li class="toctree-l4"><a class="reference internal" href="210.html#co-sub-2-sub-footprint-of-mpcdf">CO<sub>2</sub> footprint of MPCDF</a></li> <li class="toctree-l4"><a class="reference internal" href="210.html#software-news">Software news</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="210.html#gitlab-ci-distributed-cache">GitLab CI Distributed Cache</a><ul> <li class="toctree-l4"><a class="reference internal" href="210.html#introduction">Introduction</a></li> <li class="toctree-l4"><a class="reference internal" href="210.html#adding-a-cache-to-your-ci-configuration">Adding a cache to your CI configuration</a></li> <li class="toctree-l4"><a class="reference internal" href="210.html#ci-of-software-that-depends-on-third-party-packages">CI of software that depends on third-party packages</a></li> <li class="toctree-l4"><a class="reference internal" href="210.html#ci-of-a-complex-hpc-code-that-requires-cpu-and-partly-gpu-resources">CI of a complex HPC code that requires CPU and partly GPU resources</a></li> <li class="toctree-l4"><a class="reference internal" href="210.html#concluding-remarks">Concluding remarks</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="210.html#globus-flows">Globus Flows</a><ul> <li class="toctree-l4"><a class="reference internal" href="210.html#id1">Introduction</a></li> <li class="toctree-l4"><a class="reference internal" href="210.html#flows-in-detail">Flows in detail</a></li> <li class="toctree-l4"><a class="reference internal" href="210.html#example">Example</a></li> <li class="toctree-l4"><a class="reference internal" href="210.html#summary">Summary</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="210.html#news-events">News & Events</a><ul> <li class="toctree-l4"><a class="reference internal" href="210.html#introduction-to-mpcdf-services">Introduction to MPCDF services</a></li> <li class="toctree-l4"><a class="reference internal" href="210.html#meet-mpcdf">Meet MPCDF</a></li> <li class="toctree-l4"><a class="reference internal" href="210.html#advanced-hpc-workshop">Advanced HPC workshop</a></li> <li class="toctree-l4"><a class="reference internal" href="210.html#python-for-hpc">Python for HPC</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="209.html">No.209, April 2022</a><ul> <li class="toctree-l3"><a class="reference internal" href="209.html#high-performance-computing">High-performance Computing</a><ul> <li class="toctree-l4"><a class="reference internal" href="209.html#alphafold2-on-the-hpc-system-raven">AlphaFold2 on the HPC system <em>Raven</em></a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="209.html#gitlab-ci">GitLab CI</a><ul> <li class="toctree-l4"><a class="reference internal" href="209.html#gitlab-shared-runners-on-gpus">GitLab shared runners on GPUs</a></li> <li class="toctree-l4"><a class="reference internal" href="209.html#continuous-integration-testing-for-hpc-codes-on-mpcdf-gitlab">Continuous integration testing for HPC codes on MPCDF GitLab</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="209.html#globus-online">Globus Online</a><ul> <li class="toctree-l4"><a class="reference internal" href="209.html#datahub-access-via-the-globus-online-portal">DataHub access via the Globus Online Portal</a></li> <li class="toctree-l4"><a class="reference internal" href="209.html#enhanced-functionality">Enhanced functionality</a></li> <li class="toctree-l4"><a class="reference internal" href="209.html#more-information">More information</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="209.html#new-selfservice-features-and-improvements">New SelfService Features and Improvements</a><ul> <li class="toctree-l4"><a class="reference internal" href="209.html#redesign-of-the-login-process">Redesign of the login process</a></li> <li class="toctree-l4"><a class="reference internal" href="209.html#viewing-accounting-data">Viewing accounting data</a></li> <li class="toctree-l4"><a class="reference internal" href="209.html#additional-improvements">Additional improvements</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="209.html#access-to-afs-restricted-for-local-access-only">Access to AFS restricted for local Access only</a></li> <li class="toctree-l3"><a class="reference internal" href="209.html#news-events">News & Events</a><ul> <li class="toctree-l4"><a class="reference internal" href="209.html#ai-bootcamp">AI bootcamp</a></li> <li class="toctree-l4"><a class="reference internal" href="209.html#international-hpc-summer-school-2022">International HPC Summer School 2022</a></li> <li class="toctree-l4"><a class="reference internal" href="209.html#workshop-introduction-to-mpcdf-services-online">Workshop “Introduction to MPCDF services (online)”</a></li> <li class="toctree-l4"><a class="reference internal" href="209.html#meet-mpcdf-new-online-forum-and-lectures-for-mpcdf-users">“Meet MPCDF”: New online forum and lectures for MPCDF users</a></li> <li class="toctree-l4"><a class="reference internal" href="209.html#rda-deutschland-tagung-2022">RDA-Deutschland-Tagung 2022</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="208.html">No.208, December 2021</a><ul> <li class="toctree-l3"><a class="reference internal" href="208.html#high-performance-computing">High-performance Computing</a><ul> <li class="toctree-l4"><a class="reference internal" href="208.html#termination-of-general-user-operation-for-draco-login-nodes">Termination of general user operation for <em>Draco</em> login nodes</a></li> <li class="toctree-l4"><a class="reference internal" href="208.html#announcement-of-cuda-no-defaults-on-cobra-and-raven">Announcement of CUDA no-defaults on <em>Cobra</em> and <em>Raven</em></a></li> <li class="toctree-l4"><a class="reference internal" href="208.html#usage-of-tmp-and-dev-shm-on-cobra-and-raven">Usage of /tmp and /dev/shm on <em>Cobra</em> and <em>Raven</em></a></li> <li class="toctree-l4"><a class="reference internal" href="208.html#eigensolver-library-elpa">Eigensolver library ELPA</a></li> <li class="toctree-l4"><a class="reference internal" href="208.html#using-python-based-hybrid-parallel-codes-on-hpc-systems">Using Python-based hybrid-parallel codes on HPC systems</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="208.html#python-bindings-for-c-using-pybind11-and-scikit-build">Python bindings for C++ using pybind11 and scikit-build</a><ul> <li class="toctree-l4"><a class="reference internal" href="208.html#interfacing-python-numpy-with-c-using-pybind11">Interfacing Python/NumPy with C++ using pybind11</a></li> <li class="toctree-l4"><a class="reference internal" href="208.html#build-with-scikit-build">Build with scikit-build</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="208.html#the-gitlab-package-registry">The Gitlab Package Registry</a><ul> <li class="toctree-l4"><a class="reference internal" href="208.html#example-publishing-python-packages">Example: Publishing Python packages</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="208.html#using-application-tokens-instead-of-passwords">Using Application Tokens instead of Passwords</a><ul> <li class="toctree-l4"><a class="reference internal" href="208.html#datashare">DataShare</a></li> <li class="toctree-l4"><a class="reference internal" href="208.html#gitlab">GitLab</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="208.html#software-publishing">Software Publishing</a><ul> <li class="toctree-l4"><a class="reference internal" href="208.html#do-it-yourself">Do it yourself</a></li> <li class="toctree-l4"><a class="reference internal" href="208.html#publish-via-a-data-archiving-site">Publish via a data archiving site</a></li> <li class="toctree-l4"><a class="reference internal" href="208.html#software-heritage">Software Heritage</a></li> <li class="toctree-l4"><a class="reference internal" href="208.html#final-remarks">Final remarks</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="208.html#news-events">News & Events</a><ul> <li class="toctree-l4"><a class="reference internal" href="208.html#international-hpc-summer-school-2022">International HPC Summer School 2022</a></li> <li class="toctree-l4"><a class="reference internal" href="208.html#advanced-hpc-workshop-2021">Advanced HPC workshop 2021</a></li> <li class="toctree-l4"><a class="reference internal" href="208.html#years-max-planck-computing-centre-in-garching">60 Years Max Planck Computing Centre in Garching</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="207.html">No.207, August 2021</a><ul> <li class="toctree-l3"><a class="reference internal" href="207.html#high-performance-computing">High-performance Computing</a><ul> <li class="toctree-l4"><a class="reference internal" href="207.html#hpc-system-raven-fully-operational">HPC system <em>Raven</em> fully operational</a></li> <li class="toctree-l4"><a class="reference internal" href="207.html#gpu-computing-on-raven">GPU Computing on <em>Raven</em></a></li> <li class="toctree-l4"><a class="reference internal" href="207.html#hpc-system-cobra-module-system-to-be-aligned-with-raven">HPC system <em>Cobra</em> - Module system to be aligned with <em>Raven</em></a></li> <li class="toctree-l4"><a class="reference internal" href="207.html#decommissioning-of-draco">Decommissioning of <em>Draco</em></a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="207.html#hpc-cloud">HPC Cloud</a></li> <li class="toctree-l3"><a class="reference internal" href="207.html#poetry-packaging-and-dependency-management-for-python">Poetry: Packaging and Dependency Management for Python</a><ul> <li class="toctree-l4"><a class="reference internal" href="207.html#introduction">Introduction</a></li> <li class="toctree-l4"><a class="reference internal" href="207.html#a-poetry-project-and-initial-configuration">A Poetry project and initial configuration</a></li> <li class="toctree-l4"><a class="reference internal" href="207.html#dependency-management">Dependency Management</a></li> <li class="toctree-l4"><a class="reference internal" href="207.html#running-your-code">Running your code</a></li> <li class="toctree-l4"><a class="reference internal" href="207.html#package-creation-and-publishing">Package creation and publishing</a></li> <li class="toctree-l4"><a class="reference internal" href="207.html#conclusion">Conclusion</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="207.html#more-functionality-for-the-selfservice-call-to-action-for-2fa-users">More functionality for the SelfService - Call to action for 2FA users</a><ul> <li class="toctree-l4"><a class="reference internal" href="207.html#migration-of-mympcdf-functionality">Migration of MyMPCDF functionality</a></li> <li class="toctree-l4"><a class="reference internal" href="207.html#updated-password-policy">Updated password policy</a></li> <li class="toctree-l4"><a class="reference internal" href="207.html#two-factor-authentication-2fa-call-to-action">Two-Factor Authentication (2FA) - Call to action</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="207.html#news-events">News & Events</a><ul> <li class="toctree-l4"><a class="reference internal" href="207.html#brochure-high-performance-computing-and-data-science-in-the-mpg">Brochure “High-Performance Computing and Data Science in the MPG”</a></li> <li class="toctree-l4"><a class="reference internal" href="207.html#gpu-bootcamp">GPU bootcamp</a></li> <li class="toctree-l4"><a class="reference internal" href="207.html#introductory-course-for-new-users-of-mpcdf">Introductory course for new users of MPCDF</a></li> <li class="toctree-l4"><a class="reference internal" href="207.html#advanced-hpc-workshop">Advanced HPC workshop</a></li> <li class="toctree-l4"><a class="reference internal" href="207.html#python-for-hpc">Python for HPC</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="206.html">No.206, April 2021</a><ul> <li class="toctree-l3"><a class="reference internal" href="206.html#high-performance-computing">High-performance Computing</a><ul> <li class="toctree-l4"><a class="reference internal" href="206.html#hpc-system-raven-deployment-of-the-final-system">HPC System <em>Raven</em>: deployment of the final system</a></li> <li class="toctree-l4"><a class="reference internal" href="206.html#charliecloud-and-singularity-containers-supported-on-cobra-and-raven">Charliecloud and Singularity containers supported on <em>Cobra</em> and <em>Raven</em></a></li> <li class="toctree-l4"><a class="reference internal" href="206.html#control-and-verification-of-the-cpu-affinity-of-processes-and-threads">Control and verification of the CPU affinity of processes and threads</a></li> </ul> </li> <li class="toctree-l3"><a class="reference internal" href="206.html#high-performance-data-analytics-and-ai-software-stack-at-mpcdf">High-performance data analytics and AI software stack at MPCDF</a></li> <li class="toctree-l3"><a class="reference internal" href="206.html#decommissioning-of-afs">Decommissioning of AFS</a></li> <li class="toctree-l3"><a class="reference internal" href="206.html#relaunch-of-mpcdf-website-and-new-technical-documentation-platform">Relaunch of MPCDF website and new technical documentation platform</a></li> <li class="toctree-l3"><a class="reference internal" href="206.html#events">Events</a><ul> <li class="toctree-l4"><a class="reference internal" href="206.html#new-online-introductory-course-for-new-users-of-mpcdf">New online introductory course for new users of MPCDF</a></li> <li class="toctree-l4"><a class="reference internal" href="206.html#advanced-hpc-workshop-save-the-date">Advanced HPC workshop: save the date</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="previous.html">Previous Editions</a></li> </ul> </li> </ul> </div> </div> </nav> <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" style="background: #EEEEEE" > <i data-toggle="wy-nav-top" class="fa fa-bars"></i> <a href="../index.html">Technical Documentation</a> </nav> <div class="wy-nav-content"> <div class="rst-content style-external-links"> <div role="navigation" aria-label="Page navigation"> <ul class="wy-breadcrumbs"> <li><a href="../index.html" class="icon icon-home" aria-label="Home"></a></li> <li class="breadcrumb-item"><a href="index.html">Bits and Bytes</a></li> <li class="breadcrumb-item active">No.214, December 2023</li> <li class="wy-breadcrumbs-aside"> </li> </ul> <hr/> </div> <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article"> <div itemprop="articleBody"> <img src="_static/bblogo.jpg" alt="Bits and Bytes Logo" width="25%" height="25%" style="float:right"><div class="section" id="no-214-december-2023"> <h1>No.214, December 2023<a class="headerlink" href="#no-214-december-2023" title="Permalink to this heading"></a></h1> <div class="contents local topic" id="contents"> <p class="topic-title">Contents</p> <ul class="simple"> <li><p><a class="reference internal" href="#hpc-software-news" id="id1">HPC Software News</a></p> <ul> <li><p><a class="reference internal" href="#cuda-aware-openmpi-on-raven" id="id2">CUDA-aware OpenMPI on <em>Raven</em></a></p></li> <li><p><a class="reference internal" href="#gpu-accelerated-vasp" id="id3">GPU-accelerated VASP</a></p></li> <li><p><a class="reference internal" href="#intel-oneapi-transition-from-ifort-to-ifx" id="id4">Intel oneAPI: transition from ifort to ifx</a></p></li> </ul> </li> <li><p><a class="reference internal" href="#module-software-stacks-for-continuous-integration-pipelines-on-mpcdf-gitlab-shared-cloud-runners" id="id5">Module Software Stacks for Continuous Integration Pipelines on MPCDF GitLab Shared Cloud Runners</a></p> <ul> <li><p><a class="reference internal" href="#introducing-a-novel-module-enabled-docker-image-infrastructure" id="id6">Introducing a novel module-enabled Docker image infrastructure</a></p></li> <li><p><a class="reference internal" href="#announcing-legacy-status-and-later-discontinuation-of-the-module-image" id="id7">Announcing legacy status and later discontinuation of the <code class="docutils literal notranslate"><span class="pre">module-image</span></code></a></p></li> </ul> </li> <li><p><a class="reference internal" href="#compressed-portable-conda-environments-for-hpc-systems" id="id8">Compressed Portable Conda Environments for HPC Systems</a></p> <ul> <li><p><a class="reference internal" href="#introduction-and-motivation" id="id9">Introduction and Motivation</a></p></li> <li><p><a class="reference internal" href="#move-conda-environments-into-compressed-image-files" id="id10">Move Conda environments into compressed image files</a></p></li> <li><p><a class="reference internal" href="#basic-usage-examples" id="id11">Basic usage examples</a></p></li> <li><p><a class="reference internal" href="#limitations" id="id12">Limitations</a></p></li> <li><p><a class="reference internal" href="#availability" id="id13">Availability</a></p></li> </ul> </li> <li><p><a class="reference internal" href="#new-features-in-the-hpc-cloud" id="id14">New Features in the HPC-Cloud</a></p> <ul> <li><p><a class="reference internal" href="#expanded-menu-of-flavors-and-images" id="id15">Expanded menu of flavors and images</a></p></li> <li><p><a class="reference internal" href="#ssd-based-block-volumes" id="id16">SSD-based block volumes</a></p></li> <li><p><a class="reference internal" href="#automated-domain-name-service" id="id17">Automated domain name service</a></p></li> <li><p><a class="reference internal" href="#shared-filesystem-service" id="id18">Shared filesystem service</a></p></li> <li><p><a class="reference internal" href="#the-robin-cluster" id="id19">The Robin cluster</a></p></li> </ul> </li> <li><p><a class="reference internal" href="#news-events" id="id20">News & Events</a></p> <ul> <li><p><a class="reference internal" href="#amd-gpu-development-workshop" id="id21">AMD-GPU development workshop</a></p></li> <li><p><a class="reference internal" href="#meet-mpcdf" id="id22">Meet MPCDF</a></p></li> <li><p><a class="reference internal" href="#rda-deutschland-tagung-2024" id="id23">RDA Deutschland Tagung 2024</a></p></li> </ul> </li> </ul> </div> <p><a class="reference external" href="./pdf/bits_and_bytes_issue_214.pdf">PDF version</a></p> <div class="section" id="hpc-software-news"> <h2><a class="toc-backref" href="#id1">HPC Software News</a><a class="headerlink" href="#hpc-software-news" title="Permalink to this heading"></a></h2> <div class="section" id="cuda-aware-openmpi-on-raven"> <h3><a class="toc-backref" href="#id2">CUDA-aware OpenMPI on <em>Raven</em></a><a class="headerlink" href="#cuda-aware-openmpi-on-raven" title="Permalink to this heading"></a></h3> <p>MPCDF provides CUDA-aware OpenMPI on <em>Raven</em> based on different compilers and CUDA versions. The complete list can be inspected by running <code class="docutils literal notranslate"><span class="pre">find-module</span> <span class="pre">openmpi_gpu</span></code>. Below, we highlight some relevant combinations of compiler and CUDA modules that can be used with the <code class="docutils literal notranslate"><span class="pre">openmpi_gpu/4.1</span></code> module.</p> <p>GCC-based CUDA-aware OpenMPI builds are available after loading <code class="docutils literal notranslate"><span class="pre">gcc/11</span> <span class="pre">cuda/11.6</span></code> or <code class="docutils literal notranslate"><span class="pre">gcc/12</span> <span class="pre">cuda/12.1</span></code>. Recently, a CUDA-aware OpenMPI module has been added which works with the CUDA version and the compilers provided by the Nvidia SDK. To access it, the modules <code class="docutils literal notranslate"><span class="pre">nvhpcsdk/23</span> <span class="pre">cuda/11.8-nvhpcsdk</span></code> must be loaded.</p> <p><em>Tobias Melson, Tilman Dannert</em></p> </div> <div class="section" id="gpu-accelerated-vasp"> <h3><a class="toc-backref" href="#id3">GPU-accelerated VASP</a><a class="headerlink" href="#gpu-accelerated-vasp" title="Permalink to this heading"></a></h3> <p>With the deployment of CUDA-aware OpenMPI for Nvidia compilers (nvhpcsdk, see above) MPCDF provides GPU-accelerated builds of the <a class="reference external" href="https://www.vasp.at/">VASP</a> software package for atomic-scale materials modelling from first principles. Currently, a <code class="docutils literal notranslate"><span class="pre">vasp-gpu/6.4.2</span></code> module is available on <em>Raven</em> and selected institute clusters. Note that MPCDF does not hold a license for VASP. Individual users have to bring in their own license (via MPCDF helpdesk) in order to be enabled for using VASP at MPCDF.</p> <p><em>Markus Rampp</em></p> </div> <div class="section" id="intel-oneapi-transition-from-ifort-to-ifx"> <h3><a class="toc-backref" href="#id4">Intel oneAPI: transition from ifort to ifx</a><a class="headerlink" href="#intel-oneapi-transition-from-ifort-to-ifx" title="Permalink to this heading"></a></h3> <p>The transition to the new LLVM-based compilers in the Intel oneAPI package is progressing. Already in the currently installed module <code class="docutils literal notranslate"><span class="pre">intel/2023.1.0.x</span></code>, <code class="docutils literal notranslate"><span class="pre">icx</span></code> and <code class="docutils literal notranslate"><span class="pre">icpx</span></code> are the default compilers for C and C++, respectively, replacing the “classic” compilers <code class="docutils literal notranslate"><span class="pre">icc</span></code> and <code class="docutils literal notranslate"><span class="pre">icpc</span></code>. As the next step, MPCDF will follow Intel’s recommendation to set <code class="docutils literal notranslate"><span class="pre">ifx</span></code> together with its MPI wrapper <code class="docutils literal notranslate"><span class="pre">mpiifx</span></code> as the default Fortran compiler in the upcoming intel module corresponding to the oneAPI release 2024.0. The “classic” Fortran compiler <code class="docutils literal notranslate"><span class="pre">ifort</span></code> will still be present for some time, but should be considered deprecated, because its development had effectively been frozen some time ago.</p> <p>Users are advised to adjust all Fortran builds to use the new <code class="docutils literal notranslate"><span class="pre">ifx</span></code> compiler. A <a class="reference external" href="https://www.intel.com/content/www/us/en/developer/articles/guide/porting-guide-for-ifort-to-ifx.html">porting guide</a> exists with detailed information on this transition. Further support is provided at the <a class="reference external" href="https://helpdesk.mpcdf.mpg.de/">MPCDF Helpdesk</a>.</p> <p><em>Tobias Melson, Markus Rampp</em></p> </div> </div> <div class="section" id="module-software-stacks-for-continuous-integration-pipelines-on-mpcdf-gitlab-shared-cloud-runners"> <h2><a class="toc-backref" href="#id5">Module Software Stacks for Continuous Integration Pipelines on MPCDF GitLab Shared Cloud Runners</a><a class="headerlink" href="#module-software-stacks-for-continuous-integration-pipelines-on-mpcdf-gitlab-shared-cloud-runners" title="Permalink to this heading"></a></h2> <p>In order to provide the developers of HPC applications with a familiar and comprehensive software environment also within GitLab-based continuous integration (CI) pipelines, the MPCDF is maintaining special Docker images. These images use environment modules to make software accessible, very similar to how <a class="reference external" href="https://docs.mpcdf.mpg.de/doc/computing/software/environment-modules.html">software is handled on the HPC systems</a>, and they can be used on the <a class="reference external" href="https://docs.mpcdf.mpg.de/doc/data/gitlab/gitlabrunners.html">Shared Cloud Runners</a> offered by the MPCDF. Hence, e.g. build scripts would work on both the HPC systems and the CI cloud environment in a consistent way.</p> <p>This article introduces a redesign of the module-enabled Docker image infrastructure which is eventually going to replace the currently used <code class="docutils literal notranslate"><span class="pre">module-image</span></code>. <a class="reference external" href="https://docs.mpcdf.mpg.de/doc/data/gitlab/gitlabrunners.html">Documentation</a> and a <a class="reference external" href="https://mpcdf.pages.mpcdf.de/module-image/modules.list">software list</a> for the current <code class="docutils literal notranslate"><span class="pre">module-image</span></code> are available online.</p> <div class="section" id="introducing-a-novel-module-enabled-docker-image-infrastructure"> <h3><a class="toc-backref" href="#id6">Introducing a novel module-enabled Docker image infrastructure</a><a class="headerlink" href="#introducing-a-novel-module-enabled-docker-image-infrastructure" title="Permalink to this heading"></a></h3> <p>To address some limitations of the <code class="docutils literal notranslate"><span class="pre">module-image</span></code> we have developed a more flexible infrastructure composed of various Docker images, each of which provides a toolchain based on a <em>single</em> combination of compiler and MPI variant. The access to the software is implemented via environment modules. Up-to-date lists of the images together with lists of the software contained are <a class="reference external" href="https://mpcdf.pages.mpcdf.de/ci-module-image/">documented in GitLab</a>. Currently, the images are based on <em>openSUSE Leap 15.5</em> which is largely compatible with the SLES 15 operating system used on many HPC clusters at MPCDF.</p> <p>As indicated by its tag, each image only contains a single toolchain, namely one compiler with optionally one MPI library plus a selection of widely used additional libraries. The initial list of software can be extended upon request. Arbitrary further software from the official OpenSUSE repos may be installed by the users individually, if necessary.</p> <p>Users are encouraged to migrate to the new CI images soon, report potential issues and request additional software modules via the helpdesk, if necessary. Essentially, the <code class="docutils literal notranslate"><span class="pre">module-image</span></code> can simply be replaced in the user’s <code class="docutils literal notranslate"><span class="pre">gitlab-ci.yml</span></code> file with one of the new images that provides the desired software stack for the respective CI job. Please note that the modules inside the images will be updated and extended regularly, similarly to the software modules updated on the HPC systems.</p> <p>To limit the individual growth of these Docker images over time, we will put the following tagging-and-purging strategy in place: Essentially, all images are tagged using ‘latest’ and/or the calendar year. In the course of a year, say 2023, the images tagged with ‘latest’ and the year (‘2023’) are identical and receive regular updates and additions of software. With the beginning of the new year, all images tagged with the previous year stay unchanged (frozen). The newly created images for 2024, say, will start out in early January again in a slim state and will be tagged ‘latest’. Users can then choose to migrate to the more recent images (tagged ‘2024’ and ‘latest’ in our example) or stick with the older (but static!) images (tagged ‘2023’) for a while. In case a user opts for using the tag ‘latest’, please be warned that the software environment will change at the beginning of each year.</p> </div> <div class="section" id="announcing-legacy-status-and-later-discontinuation-of-the-module-image"> <h3><a class="toc-backref" href="#id7">Announcing legacy status and later discontinuation of the <code class="docutils literal notranslate"><span class="pre">module-image</span></code></a><a class="headerlink" href="#announcing-legacy-status-and-later-discontinuation-of-the-module-image" title="Permalink to this heading"></a></h3> <p>The current <code class="docutils literal notranslate"><span class="pre">module-image</span></code> has been provided for several years now to provide easy access to the familiar modules environment also from within GitLab CI jobs. As the image and infrastructure are based on the outdated CentOS 7, the <code class="docutils literal notranslate"><span class="pre">module-image</span></code> will be considered legacy after the test phase of the new Docker image infrastructure and will then not be updated any more. Ultimately, the image will have to be removed (to be announced in due time).</p> <p><em>Tobias Melson, Klaus Reuter</em></p> </div> </div> <div class="section" id="compressed-portable-conda-environments-for-hpc-systems"> <h2><a class="toc-backref" href="#id8">Compressed Portable Conda Environments for HPC Systems</a><a class="headerlink" href="#compressed-portable-conda-environments-for-hpc-systems" title="Permalink to this heading"></a></h2> <div class="section" id="introduction-and-motivation"> <h3><a class="toc-backref" href="#id9">Introduction and Motivation</a><a class="headerlink" href="#introduction-and-motivation" title="Permalink to this heading"></a></h3> <p>The Conda package manager and the related workflows have become an accepted standard when it comes to distributing scientific software for easy installation by end users. Using <code class="docutils literal notranslate"><span class="pre">conda</span></code>, complex software environments can be defined by means of simple descriptive <code class="docutils literal notranslate"><span class="pre">environment.yml</span></code> files. On MPCDF systems, users may use Conda environments, but without support from MPCDF for the software therein.</p> <p>Once installed, large Conda environments can easily amount to several 100k individual (small) files. On the local file systems of a laptop or PC this is typically not an issue. However, in particular on the large shared parallel file systems of HPC systems the vast amount of small files may cause issues, as these file systems are optimized for other scenarios. Inode exhaustion and heavy load due to (millions of) file opens, short reads, and closes happening during the startup phase of Python jobs from the different users on the system are only two examples.</p> </div> <div class="section" id="move-conda-environments-into-compressed-image-files"> <h3><a class="toc-backref" href="#id10">Move Conda environments into compressed image files</a><a class="headerlink" href="#move-conda-environments-into-compressed-image-files" title="Permalink to this heading"></a></h3> <p>MPCDF developed the new open-source tool <em>Condainer</em>, which adresses these issues by moving Conda environments into compressed squashfs images, reducing the number of files stored directly on the host file system by orders of magnitude. Condainer images are standalone and portable: They can be copied between different systems, improving reproducibility and reusability of proven-to-work software environments. In particular, they sidestep the integration of a specific <code class="docutils literal notranslate"><span class="pre">conda</span></code> executable into the user’s <code class="docutils literal notranslate"><span class="pre">.bashrc</span></code> file, which often causes issues and is orthogonal to the module-based software environments provided on HPC systems.</p> <p>Technically, Condainer uses a Python basis from Miniforge (which is a free alternative to Miniconda) and then installs the user-defined software stack from the usual <code class="docutils literal notranslate"><span class="pre">environment.yml</span></code> file. Package dependency resolution and installation are extremely fast thanks to the <code class="docutils literal notranslate"><span class="pre">mamba</span></code> package manager (an optimized replacement for <code class="docutils literal notranslate"><span class="pre">conda</span></code>). As a second step, Condainer creates a compressed squashfs image file from the staging installation, before it deletes the latter to save disk space. Subsequently, the compressed image is mounted (using <code class="docutils literal notranslate"><span class="pre">squashfuse</span></code>) at the very same directory, providing the full Conda environment to the user who can <code class="docutils literal notranslate"><span class="pre">activate</span></code> or <code class="docutils literal notranslate"><span class="pre">deactivate</span></code> it, just as usual. Moreover, Condainer provides functionality to run executables from the Conda environment directly and transparently, without the need to explicitly mount and unmount the image.</p> <p>Please note that the squashfs images used by Condainer are not “containers” in the strict terminology of Docker, Apptainer, or alike. With Condainer, there is no process isolation or similar, rather Condainer is an easy-to-use and highly efficient wrapper around the building, compressing, mounting, and unmounting of Conda environments on top of compressed image files.</p> </div> <div class="section" id="basic-usage-examples"> <h3><a class="toc-backref" href="#id11">Basic usage examples</a><a class="headerlink" href="#basic-usage-examples" title="Permalink to this heading"></a></h3> <div class="section" id="build-a-compressed-environment"> <h4>Build a compressed environment<a class="headerlink" href="#build-a-compressed-environment" title="Permalink to this heading"></a></h4> <p>Follow along the following commands once in order to build a compressed image of a Conda environment that is defined in ‘environment.yml’:</p> <div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="c1"># on MPCDF systems, e.g. Raven:</span> module<span class="w"> </span>load<span class="w"> </span>condainer<span class="w"> </span> <span class="c1"># create specific project directory:</span> mkdir<span class="w"> </span>my_cnd_env<span class="w"> </span><span class="o">&&</span><span class="w"> </span><span class="nb">cd</span><span class="w"> </span>my_cnd_env<span class="w"> </span> <span class="c1"># initialize project directory with a skeleton:</span> cnd<span class="w"> </span>init<span class="w"> </span> ls <span class="c1"># edit the 'environment.yml' example file, </span> <span class="c1"># or copy your own file here</span> <span class="c1"># build the environment and compressed image:</span> cnd<span class="w"> </span>build<span class="w"> </span> ls </pre></div> </div> </div> <div class="section" id="activate-a-compressed-environment"> <h4>Activate a compressed environment<a class="headerlink" href="#activate-a-compressed-environment" title="Permalink to this heading"></a></h4> <p>After building, you can activate the environment for your current shell session, similar to plain Conda or a Python virtual environment:</p> <div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nb">source</span><span class="w"> </span>activate </pre></div> </div> <p>Please note that <code class="docutils literal notranslate"><span class="pre">source</span> <span class="pre">activate</span></code> will only work with bourne shells (e.g. <code class="docutils literal notranslate"><span class="pre">bash</span></code> or <code class="docutils literal notranslate"><span class="pre">zsh</span></code>), not with the older C shells and korn shells.</p> </div> <div class="section" id="alternatively-run-an-executable-from-a-compressed-environment-directly"> <h4>Alternatively, run an executable from a compressed environment directly<a class="headerlink" href="#alternatively-run-an-executable-from-a-compressed-environment-directly" title="Permalink to this heading"></a></h4> <p>In case you do not want to activate the environment, you can run individual executables from the environment directly, e.g.</p> <div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>cnd<span class="w"> </span><span class="nb">exec</span><span class="w"> </span>--<span class="w"> </span>python3 </pre></div> </div> <p>The <code class="docutils literal notranslate"><span class="pre">cnd</span></code> command supports the flag <code class="docutils literal notranslate"><span class="pre">--directory</span></code> to specify a certain Condainer project directory, allowing for arbitrary current working directories.</p> </div> </div> <div class="section" id="limitations"> <h3><a class="toc-backref" href="#id12">Limitations</a><a class="headerlink" href="#limitations" title="Permalink to this heading"></a></h3> <p>As the squashfs fuse mounts are specific to an individual compute node, Condainer currently (v0.1.8) does not support multi-node batch jobs.</p> </div> <div class="section" id="availability"> <h3><a class="toc-backref" href="#id13">Availability</a><a class="headerlink" href="#availability" title="Permalink to this heading"></a></h3> <p>The software including its documentation is freely available via the <a class="reference external" href="https://gitlab.mpcdf.mpg.de/mpcdf/condainer">MPCDF gitlab</a>. Moreover, it is provided via the environment module <code class="docutils literal notranslate"><span class="pre">condainer</span></code> on the <em>Raven</em> HPC system, and will be offered on more systems in the near future.</p> <p><em>Klaus Reuter</em></p> </div> </div> <div class="section" id="new-features-in-the-hpc-cloud"> <h2><a class="toc-backref" href="#id14">New Features in the HPC-Cloud</a><a class="headerlink" href="#new-features-in-the-hpc-cloud" title="Permalink to this heading"></a></h2> <p>After commissioning the initial set of compute and storage resources in <a class="reference external" href="https://docs.mpcdf.mpg.de/bnb/207.html#hpc-cloud">2021</a>, deploying the GPU- and NVMe-focused <a class="reference external" href="https://docs.mpcdf.mpg.de/bnb/212.html#mpcdf-hpc-cloud">extension</a> earlier this year, and rolling-out integrated <a class="reference external" href="https://docs.mpcdf.mpg.de/bnb/213.html#hpc-cloud-object-storage">object storage</a>, MPCDF has recently deployed several new features to better support the diverse technical requirements of current and future projects:</p> <div class="section" id="expanded-menu-of-flavors-and-images"> <h3><a class="toc-backref" href="#id15">Expanded menu of flavors and images</a><a class="headerlink" href="#expanded-menu-of-flavors-and-images" title="Permalink to this heading"></a></h3> <p>Projects now have access by default to more combinations of vCPUs and memory, known as <em>flavors</em> within the cloud environment, so that virtual machines can be sized to match the application. In practice, MPCDF now offers flavors up to 24 vCPUs and 64 GB of memory, subject to certain boundary conditions which ensure they can be efficiently mapped to a physical compute node. As before, even larger flavors as well as local SSD-, GPU-, and NVMe-enabled versions of the default flavors can be created on request.</p> <p>Several new virtual machine templates, known as <em>images</em> within the cloud environment, have been prepared, including AlmaLinux as well as newer releases of CentOS Stream, Debian, and openSUSE Leap. Commercial operating systems such as Red Hat Enterprise Linux and SUSE Linux Enterprise Server are also available on a BYOL (bring your own license) basis.</p> </div> <div class="section" id="ssd-based-block-volumes"> <h3><a class="toc-backref" href="#id16">SSD-based block volumes</a><a class="headerlink" href="#ssd-based-block-volumes" title="Permalink to this heading"></a></h3> <p>There is now an SSD-based block volume type <em>CephSSD</em> representing a middle-ground option between highly-performant local SSDs and the highly-flexible and scalable network-based HDD storage associated with the default volume type. While I/O performance cannot be guaranteed in a shared-resource environment, one can expect a roughly 2X speedup in terms of small I/O operations per second and large transfer bandwidth, as well as a significant reduction in latency.</p> <p>To evaluate whether the new volume type is a good option for your project, please contact the cloud enabling team via the <a class="reference external" href="https://helpdesk.mpcdf.mpg.de">helpdesk</a>. As a tip, existing block volumes can be migrated between types <em>online</em>, making it relatively simple to test an already-deployed application.</p> <p>In addition to the new volume type, all Ceph-based system disks, i.e. the OS root of the VMs not hosted on local SSDs, have been transparently migrated to an SSD pool “for free”, so that routine tasks such as software installation and updates complete more quickly.</p> </div> <div class="section" id="automated-domain-name-service"> <h3><a class="toc-backref" href="#id17">Automated domain name service</a><a class="headerlink" href="#automated-domain-name-service" title="Permalink to this heading"></a></h3> <p>Hostnames are now automatically generated for most devices attached to the public or local cloud networks, including virtual machines and floating IP addresses. The system works like this:</p> <ol> <li><p>Each virtual machine is assigned a hostname of the following form:</p> <p><code class="docutils literal notranslate"><span class="pre">VM_NAME.PROJECT_NAME.hpccloud.mpg.de</span></code></p> <p>If the name of the virtual machine is invalid according to the requirements of DNS, then a unique hostname based on the fixed IP address will be substituted automatically.</p> </li> <li><p>Each floating IP is assigned a hostname of the following form:</p> <p><code class="docutils literal notranslate"><span class="pre">FIP_DESCRIPTION.PROJECT_NAME.hpccloud.mpg.de</span></code></p> <p>If the description field is empty or invalid, then a unique hostname based on the floating IP address will be substituted automatically.</p> </li> <li><p>Hostnames are synchronized with the MPCDF DNS servers every five minutes. For devices on the public cloud network, both forward and reverse entries are propagated to the global DNS, whereas on local networks only the forward (i.e. hostname->IP address) entries are published.</p></li> </ol> <p>Thus, within the framework described above it is possible to deploy and configure many applications on the HPC-Cloud without tracking individual IP addresses.</p> </div> <div class="section" id="shared-filesystem-service"> <h3><a class="toc-backref" href="#id18">Shared filesystem service</a><a class="headerlink" href="#shared-filesystem-service" title="Permalink to this heading"></a></h3> <p>The HPC-Cloud now implements shared filesystem-as-a-service (FSaaS), also known as OpenStack Manila or simply <em>Share</em> within the cloud dashboard, as an alternative to Nexus-Posix project directories. The two technologies are complementary, with differing strengths and weaknesses:</p> <table border="1" class="docutils"> <thead> <tr> <th>Service</th> <th>Technology</th> <th>Protocol</th> <th>External access</th> <th>Typical scale</th> <th>Lifetime</th> <th>Provisioning</th> </tr> </thead> <tbody> <tr> <td>FSaaS</td> <td>CephFS</td> <td>NFS, or native CephFS</td> <td><em>none</em></td> <td>100 GB up to 50 TB</td> <td><em>arbitrary</em></td> <td>self-provisioned via dashboard/API</td> </tr> <tr> <td>Nexus-Posix</td> <td>IBM Storage Scale (GPFS)</td> <td>NFS</td> <td><em>Raven</em>, <em>Robin</em>, GO-Nexus</td> <td>10 TB up to 100 TB, or larger</td> <td>months to years</td> <td>on request via helpdesk ticket</td> </tr> </tbody> </table><p>The key advantages of Manila-based shares are that they can be provisioned quickly by project admins and, being logically isolated from other systems, can easily handle arbitrary UIDs and GIDs including root and/or service users. On the other hand, Nexus-Posix as an MPCDF-administered filesystem supports secure interoperation with <em>Raven</em> and other systems as well as built-in backups. To evaluate filesystem-as-a-service for your project please get in touch with the cloud enabling team.</p> <p>More details about these new features can be found in the <a class="reference external" href="https://docs.mpcdf.mpg.de/doc/cloud/technical/">technical documentation</a>.</p> <p><em>John Alan Kennedy, Brian Standley, Maximiliano Geier</em></p> </div> <div class="section" id="the-robin-cluster"> <h3><a class="toc-backref" href="#id19">The Robin cluster</a><a class="headerlink" href="#the-robin-cluster" title="Permalink to this heading"></a></h3> <p>The Remote Visualization Service at MPCDF has recently been expanded with a new cluster called <em>Robin</em>. <em>Robin</em> is the first compute cluster of MPCDF in the HPC-Cloud and its resources are available to all users with an access to the HPC systems (i.e. <em>Cobra</em> and <em>Raven</em>).</p> <p>One of the main advantages of the <em>Robin</em> cluster is its flexibility: new nodes can be easily deployed in the HPC-Cloud, automatically configured and added to the Slurm cluster, allowing for a convenient scaling of the compute resources on <em>Robin</em> depending on the current demand.</p> <p><em>Robin</em> uses Slurm as a job scheduler and it can currently host up to 20 CPU sessions and 12 GPU sessions, concurrently. Access to the cluster is restricted via our Remote Visualization Service web interface, so that users are not allowed to connect directly via ssh to the login or compute nodes of <em>Robin</em>.</p> <p>Each session on <em>Robin</em> provides 12 virtual CPUs and 64 GB of RAM, with GPU sessions having access to a shared Nvidia A30 GPU (up to 4 sessions can share a single GPU). <em>Robin</em> mounts the <em>Raven</em> file systems, providing access to all the software and data available on the <em>Raven</em> cluster, including the user’s home directory and ptmp folder. A runtime of up to 7 days is currently allowed, with a plan to increase to up to 28 days of maximum runtime in the future, but users are encouraged to stop their sessions once their calculations are completed and should be aware that long-running jobs can be killed in case of maintenance of the cluster.</p> <p>Users requesting GPU sessions are encouraged to limit the memory used by their code to roughly 1/4 of the available GPU memory (~6 GB out of the 24 GB available), in order to avoid disrupting the calculations of other users sharing the same GPU. This is particularly important for Machine Learning software (e.g. Tensorflow, Pytorch) that can allocate the entire available GPU memory for a single process.</p> <p><em>Robin</em> is designed to provide a single solution for the remote visualization needs of future HPC clusters at MPCDF: the filesystem of new clusters (like the upcoming <em>Viper</em>) can be made available on <em>Robin</em>, providing easy access to software and data without the need of a dedicated installation of the Remote Visualization Service on each cluster.</p> <p>Users interested in using the Remote Visualization Service on <em>Robin</em> are reminded to initialize their sessions on the cluster once (before submitting their first session), as described in our <a class="reference external" href="https://docs.mpcdf.mpg.de/doc/visualization/index.html">documentation</a>.</p> <p><em>Michele Compostella</em></p> </div> </div> <div class="section" id="news-events"> <h2><a class="toc-backref" href="#id20">News & Events</a><a class="headerlink" href="#news-events" title="Permalink to this heading"></a></h2> <div class="section" id="amd-gpu-development-workshop"> <h3><a class="toc-backref" href="#id21">AMD-GPU development workshop</a><a class="headerlink" href="#amd-gpu-development-workshop" title="Permalink to this heading"></a></h3> <p>In preparation for the <a class="reference external" href="https://docs.mpcdf.mpg.de/bnb/212.html#new-supercomputer-of-the-mpg-cobra-successor">new supercomputer <em>Viper</em></a> of the MPG with AMD MI300A GPUs to be installed in 2024, the MPCDF in collaboration with AMD offered an online course on AMD Instinct GPU architecture and the corresponding ROCm software ecosystem, including the tools to develop or port HPC or AI applications to AMD GPUs. The workshop was held as an online event, spanning three afternoons on November 28-30, 2023. The workshop material can be found on the <a class="reference external" href="https://www.mpcdf.mpg.de/services/training">MPCDF training website</a>.</p> <p><em>Tilman Dannert</em></p> </div> <div class="section" id="meet-mpcdf"> <h3><a class="toc-backref" href="#id22">Meet MPCDF</a><a class="headerlink" href="#meet-mpcdf" title="Permalink to this heading"></a></h3> <p>The monthly online-seminar series “Meet MPCDF” skips the talk in January 2024. The next edition will take place on February 1st, 2024 with a talk on “ScaLAPACK and ELPA: how to diagonalize really large dense matrices” given by Petr Karpov from the MPCDF. Subsequent dates will be March 7th and April 4th (topics to be announced). All announcements and material can be found on our <a class="reference external" href="https://www.mpcdf.mpg.de/services/training">training webpage</a>.</p> <p>We encourage our users to propose further topics of their interest, e.g. in the fields of high-performance computing, data management, artificial intelligence or high-performance data analytics. Please send an E-mail to <a class="reference external" href="mailto:training%40mpcdf.mpg.de">training<span>@</span>mpcdf<span>.</span>mpg<span>.</span>de</a>.</p> <p><em>Tilman Dannert</em></p> </div> <div class="section" id="rda-deutschland-tagung-2024"> <h3><a class="toc-backref" href="#id23">RDA Deutschland Tagung 2024</a><a class="headerlink" href="#rda-deutschland-tagung-2024" title="Permalink to this heading"></a></h3> <p>The <a class="reference external" href="https://www.rda-deutschland.de/">German chapter of the Research Data Alliance</a> will have its next conference in Potsdam, February 20-21, 2024. This year’s focus is on legal, administrative and organizational topics concerning research data management in Germany and Europe. The early registration deadline is January 12, 2024. Further details including the program are available from <a class="reference external" href="https://indico.desy.de/event/42727/">https://indico.desy.de/event/42727/</a>. As in previous years, MPCDF is contributing to the organization of the event.</p> <p><em>Raphael Ritz</em></p> </div> </div> </div> </div> </div> <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer"> <a href="215.html" class="btn btn-neutral float-left" title="No.215, April 2024" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a> <a href="213.html" class="btn btn-neutral float-right" title="No.213, August 2023" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a> </div> <hr/> <div role="contentinfo"> <p> <span class="lastupdated">Last updated on Mar 26, 2024. </span></p> </div> <p> © Copyright <a href="https://www.mpcdf.mpg.de/">Max Planck Computing and Data Facility</a> | <a href="https://www.mpcdf.mpg.de/imprint">Imprint</a> | <a href="https://www.mpcdf.mpg.de/privacy-policy">Privacy Policy</a> </p> <p> Built with <a href="https://www.sphinx-doc.org">Sphinx</a> using a <a href="https://github.com/rtfd/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>. </p> </footer> </div> </div> </section> </div> <script> jQuery(function () { SphinxRtdTheme.Navigation.enable(true); }); </script> </body> </html>