CINXE.COM
<!doctype html> <html lang="en" dir="ltr" class="plugin-pages plugin-id-default"> <head> <meta charset="UTF-8"> <meta name="generator" content="Docusaurus v2.3.1"> <title data-rh="true">Colossal-AI</title><meta data-rh="true" property="og:title" content="Colossal-AI"><meta data-rh="true" name="viewport" content="width=device-width,initial-scale=1"><meta data-rh="true" name="twitter:card" content="summary_large_image"><meta data-rh="true" property="og:image" content="https://colossalai.org/img/social-card.png"><meta data-rh="true" name="twitter:image" content="https://colossalai.org/img/social-card.png"><meta data-rh="true" property="og:url" content="https://colossalai.org/"><meta data-rh="true" name="docusaurus_locale" content="en"><meta data-rh="true" name="docusaurus_tag" content="default"><meta data-rh="true" name="docsearch:language" content="en"><meta data-rh="true" name="docsearch:docusaurus_tag" content="default"><meta data-rh="true" name="description" content="Making big AI models cheaper, easier, and scalable"><meta data-rh="true" property="og:description" content="Making big AI models cheaper, easier, and scalable"><link data-rh="true" rel="icon" href="/img/favicon.ico"><link data-rh="true" rel="canonical" href="https://colossalai.org/"><link data-rh="true" rel="alternate" href="https://colossalai.org/" hreflang="en"><link data-rh="true" rel="alternate" href="https://colossalai.org/zh-Hans/" hreflang="zh-Hans"><link data-rh="true" rel="alternate" href="https://colossalai.org/" hreflang="x-default"><link data-rh="true" rel="preconnect" href="https://XP2V2KAOVI-dsn.algolia.net" crossorigin="anonymous"><script data-rh="true">function maybeInsertBanner(){window.__DOCUSAURUS_INSERT_BASEURL_BANNER&&insertBanner()}function insertBanner(){var n=document.getElementById("docusaurus-base-url-issue-banner-container");if(n){n.innerHTML='\n<div id="docusaurus-base-url-issue-banner" style="border: thick solid red; background-color: rgb(255, 230, 179); margin: 20px; padding: 20px; font-size: 20px;">\n <p style="font-weight: bold; font-size: 30px;">Your Docusaurus site did not load properly.</p>\n <p>A very common reason is a wrong site <a href="https://docusaurus.io/docs/docusaurus.config.js/#baseurl" style="font-weight: bold;">baseUrl configuration</a>.</p>\n <p>Current configured baseUrl = <span style="font-weight: bold; color: red;">/</span> (default value)</p>\n <p>We suggest trying baseUrl = <span id="docusaurus-base-url-issue-banner-suggestion-container" style="font-weight: bold; color: green;"></span></p>\n</div>\n';var e=document.getElementById("docusaurus-base-url-issue-banner-suggestion-container"),s=window.location.pathname,r="/"===s.substr(-1)?s:s+"/";e.innerHTML=r}}window.__DOCUSAURUS_INSERT_BASEURL_BANNER=!0,document.addEventListener("DOMContentLoaded",maybeInsertBanner)</script><link rel="alternate" type="application/rss+xml" href="/blog/rss.xml" title="Colossal-AI RSS Feed"> <link rel="alternate" type="application/atom+xml" href="/blog/atom.xml" title="Colossal-AI Atom Feed"> <link rel="preconnect" href="https://www.google-analytics.com"> <link rel="preconnect" href="https://www.googletagmanager.com"> <script async src="https://www.googletagmanager.com/gtag/js?id=G-1XKZVCCKRZ"></script> <script>function gtag(){dataLayer.push(arguments)}window.dataLayer=window.dataLayer||[],gtag("js",new Date),gtag("config","G-1XKZVCCKRZ",{anonymize_ip:!0})</script> <link rel="search" type="application/opensearchdescription+xml" title="Colossal-AI" href="/opensearch.xml"> <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/katex@0.13.24/dist/katex.min.css" integrity="sha384-odtC+0UGzzFL/6PNoE8rX/SPcQDXBJ+uRepguP4QkPCm2LBxH3FA3y+fKSiJ+AmM" crossorigin="anonymous"> <script src="https://js-eu1.hs-scripts.com/26563514.js" id="hs-script-loader" async defer="defer"></script><link rel="stylesheet" href="/assets/css/styles.d9a9b3d5.css"> <link rel="preload" href="/assets/js/runtime~main.6b0f350a.js" as="script"> <link rel="preload" href="/assets/js/main.c6c31bc7.js" as="script"> </head> <body class="navigation-with-keyboard"> <script>!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}()</script><div id="__docusaurus"> <div id="docusaurus-base-url-issue-banner-container"></div><div role="region" aria-label="Skip to main content"><a class="skipToContent_fXgn" href="#docusaurus_skipToContent_fallback">Skip to main content</a></div><nav aria-label="Main" class="navbar navbar--fixed-top"><div class="navbar__inner"><div class="navbar__items"><button aria-label="Toggle navigation bar" aria-expanded="false" class="navbar__toggle clean-btn" type="button"><svg width="30" height="30" viewBox="0 0 30 30" aria-hidden="true"><path stroke="currentColor" stroke-linecap="round" stroke-miterlimit="10" stroke-width="2" d="M4 7h22M4 15h22M4 23h22"></path></svg></button><a class="navbar__brand" href="/"><div class="navbar__logo"><img src="/img/logo.svg" alt="project-logo" class="themedImage_ToTc themedImage--light_HNdA"><img src="/img/logo.svg" alt="project-logo" class="themedImage_ToTc themedImage--dark_i4oU"></div><b class="navbar__title text--truncate">Colossal-AI</b></a><a class="navbar__item navbar__link" href="/docs/get_started/installation">Tutorials</a><a href="https://github.com/hpcaitech/ColossalAI/tree/main/examples" target="_blank" rel="noopener noreferrer" class="navbar__item navbar__link">Examples</a><a href="https://www.hpc-ai.tech/blog" target="_blank" rel="noopener noreferrer" class="navbar__item navbar__link">Blogs</a></div><div class="navbar__items navbar__items--right"><a class="navbar__item navbar__link" href="/docs/get_started/installation">Next</a><div class="navbar__item dropdown dropdown--hoverable dropdown--right"><a href="#" aria-haspopup="true" aria-expanded="false" role="button" class="navbar__link"><svg viewBox="0 0 24 24" width="20" height="20" aria-hidden="true" class="iconLanguage_nlXk"><path fill="currentColor" d="M12.87 15.07l-2.54-2.51.03-.03c1.74-1.94 2.98-4.17 3.71-6.53H17V4h-7V2H8v2H1v1.99h11.17C11.5 7.92 10.44 9.75 9 11.35 8.07 10.32 7.3 9.19 6.69 8h-2c.73 1.63 1.73 3.17 2.98 4.56l-5.09 5.02L4 19l5-5 3.11 3.11.76-2.04zM18.5 10h-2L12 22h2l1.12-3h4.75L21 22h2l-4.5-12zm-2.62 7l1.62-4.33L19.12 17h-3.24z"></path></svg>English</a><ul class="dropdown__menu"><li><a href="/" target="_self" rel="noopener noreferrer" class="dropdown__link dropdown__link--active" lang="en">English</a></li><li><a href="/zh-Hans/" target="_self" rel="noopener noreferrer" class="dropdown__link" lang="zh-Hans">简体中文</a></li></ul></div><a href="https://github.com/hpcaitech/ColossalAI" target="_blank" rel="noopener noreferrer" class="navbar__item navbar__link header-github-link"></a><div class="searchBox_ZlJk"><button type="button" class="DocSearch DocSearch-Button" aria-label="Search"><span class="DocSearch-Button-Container"><svg width="20" height="20" class="DocSearch-Search-Icon" viewBox="0 0 20 20"><path d="M14.386 14.386l4.0877 4.0877-4.0877-4.0877c-2.9418 2.9419-7.7115 2.9419-10.6533 0-2.9419-2.9418-2.9419-7.7115 0-10.6533 2.9418-2.9419 7.7115-2.9419 10.6533 0 2.9419 2.9418 2.9419 7.7115 0 10.6533z" stroke="currentColor" fill="none" fill-rule="evenodd" stroke-linecap="round" stroke-linejoin="round"></path></svg><span class="DocSearch-Button-Placeholder">Search</span></span><span class="DocSearch-Button-Keys"></span></button></div></div></div><div role="presentation" class="navbar-sidebar__backdrop"></div></nav><div id="docusaurus_skipToContent_fallback" class="main-wrapper mainWrapper_z2l0"><script id="hs-script-loader" async="" defer="" src="//js-eu1.hs-scripts.com/26563514.js"></script><div class="tw-relative tw-isolate tw-bg-white"><div class="tw-absolute tw-inset-x-0 tw-top-[-10rem] tw--z-10 tw-transform-gputw-overflow-hidden tw-blur-3xl sm:tw-top-[-20rem]"><svg class="tw-relative tw-left-[calc(50%-11rem)] tw--z-10 tw-h-[21.1875rem] tw-max-w-none tw--translate-x-1/2 tw-rotate-[30deg] sm:tw-left-[calc(50%-30rem)] sm:tw-h-[42.375rem]" viewBox="0 0 1155 678"><path fill="url(#9b2541ea-d39d-499b-bd42-aeea3e93f5ff)" fill-opacity=".3" d="M317.219 518.975L203.852 678 0 438.341l317.219 80.634 204.172-286.402c1.307 132.337 45.083 346.658 209.733 145.248C936.936 126.058 882.053-94.234 1031.02 41.331c119.18 108.451 130.68 295.337 121.53 375.223L855 299l21.173 362.054-558.954-142.079z"></path><defs><linearGradient id="9b2541ea-d39d-499b-bd42-aeea3e93f5ff" x1="1155.49" x2="-78.208" y1=".177" y2="474.645" gradientUnits="userSpaceOnUse"><stop stop-color="#9089FC"></stop><stop offset="1" stop-color="#FF80B5"></stop></linearGradient></defs></svg></div><div class="tw-relative tw-px-6 lg:tw-px-8"><div class="tw-mx-auto tw-py-16 md:tw-py-24 lg:tw-py-36"><div class="tw-text-center"><h1 class="gradient-color tw-text-5xl tw-font-bold tw-tracking-tight tw-text-gray-900 sm:tw-text-4xl md:tw-text-5xl lg:tw-text-7xl">Unmatched Speed and Scale</h1><p class="tw-mt-6 tw-text-lg tw-leading-8 tw-text-gray-600">Learn about the distributed techniques of Colossal-AI to maximize the runtime performance of your large neural networks.</p><div class="tw-mt-10 tw-flex tw-flex-col md:tw-flex-row tw-items-center tw-justify-center tw-gap-x-6"><a href="docs/get_started/installation" class="tw-rounded-md tw-bg-indigo-600 tw-px-3.5 tw-py-1.5 tw-my-2 tw-text-base tw-font-semibold tw-leading-7 tw-text-white tw-shadow-sm hover:tw-bg-indigo-500 focus-visible:tw-outline focus-visible:tw-outline-2 focus-visible:tw-outline-offset-2 focus-visible:tw-outline-indigo-600 hover:tw-no-underline hover:tw-text-white"><div class="tw-flex tw-items-center"><svg stroke="currentColor" fill="currentColor" stroke-width="0" viewBox="0 0 512 512" height="1em" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M256 8C119 8 8 119 8 256s111 248 248 248 248-111 248-248S393 8 256 8zm115.7 272l-176 101c-15.8 8.8-35.7-2.5-35.7-21V152c0-18.4 19.8-29.8 35.7-21l176 107c16.4 9.2 16.4 32.9 0 42z"></path></svg> <!-- -->Get started</div></a><a href="https://github.com/hpcaitech/ColossalAI" class="tw-rounded-md tw-bg-indigo-600 tw-px-3.5 tw-py-1.5 tw-my-2 tw-text-base tw-font-semibold tw-leading-7 tw-text-white tw-shadow-sm hover:tw-bg-indigo-500 focus-visible:tw-outline focus-visible:tw-outline-2 focus-visible:tw-outline-offset-2 focus-visible:tw-outline-indigo-600 hover:tw-no-underline hover:tw-text-white"><div class="tw-flex tw-items-center"><svg stroke="currentColor" fill="currentColor" stroke-width="0" viewBox="0 0 496 512" height="1em" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg> <!-- -->GitHub</div></a><a href="https://github.com/hpcaitech/public_assets/tree/main/colossalai/contact/slack" class="tw-rounded-md tw-bg-indigo-600 tw-px-3.5 tw-py-1.5 tw-my-2 tw-text-base tw-font-semibold tw-leading-7 tw-text-white tw-shadow-sm hover:tw-bg-indigo-500 focus-visible:tw-outline focus-visible:tw-outline-2 focus-visible:tw-outline-offset-2 focus-visible:tw-outline-indigo-600 hover:tw-no-underline hover:tw-text-white"><div class="tw-flex tw-items-center"><svg stroke="currentColor" fill="currentColor" stroke-width="0" viewBox="0 0 448 512" height="1em" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M94.12 315.1c0 25.9-21.16 47.06-47.06 47.06S0 341 0 315.1c0-25.9 21.16-47.06 47.06-47.06h47.06v47.06zm23.72 0c0-25.9 21.16-47.06 47.06-47.06s47.06 21.16 47.06 47.06v117.84c0 25.9-21.16 47.06-47.06 47.06s-47.06-21.16-47.06-47.06V315.1zm47.06-188.98c-25.9 0-47.06-21.16-47.06-47.06S139 32 164.9 32s47.06 21.16 47.06 47.06v47.06H164.9zm0 23.72c25.9 0 47.06 21.16 47.06 47.06s-21.16 47.06-47.06 47.06H47.06C21.16 243.96 0 222.8 0 196.9s21.16-47.06 47.06-47.06H164.9zm188.98 47.06c0-25.9 21.16-47.06 47.06-47.06 25.9 0 47.06 21.16 47.06 47.06s-21.16 47.06-47.06 47.06h-47.06V196.9zm-23.72 0c0 25.9-21.16 47.06-47.06 47.06-25.9 0-47.06-21.16-47.06-47.06V79.06c0-25.9 21.16-47.06 47.06-47.06 25.9 0 47.06 21.16 47.06 47.06V196.9zM283.1 385.88c25.9 0 47.06 21.16 47.06 47.06 0 25.9-21.16 47.06-47.06 47.06-25.9 0-47.06-21.16-47.06-47.06v-47.06h47.06zm0-23.72c-25.9 0-47.06-21.16-47.06-47.06 0-25.9 21.16-47.06 47.06-47.06h117.84c25.9 0 47.06 21.16 47.06 47.06 0 25.9-21.16 47.06-47.06 47.06H283.1z"></path></svg> <!-- -->Community</div></a></div></div><div class="tw-hidden tw-mt-8 sm:tw-mb-8 sm:tw-flex sm:tw-justify-center"><div class="tw-relative tw-rounded-full tw-py-1 tw-px-3 tw-text-sm tw-leading-6 tw-text-gray-600 tw-ring-1 tw-ring-gray-900/10 hover:tw-ring-gray-900/20">Need professional help?<!-- --> <a href="https://www.hpc-ai.tech/contact" class="tw-font-semibold tw-text-indigo-600"><span class="tw-absolute tw-inset-0" aria-hidden="true"></span>Talk to our experts<!-- --> <span aria-hidden="true">→</span></a></div></div></div></div></div><main><div class="tw-relative tw-bg-white tw-py-8 sm:tw-py-12 lg:tw-py-16"><div class="tw-mx-auto tw-max-w-md tw-px-6 tw-text-center sm:tw-max-w-3xl lg:tw-max-w-7xl lg:tw-px-8"><p class="tw-mt-2 tw-text-3xl tw-font-bold tw-tracking-tight tw-text-gray-900 sm:tw-ext-4xl">All about Colossal-AI</p><div class="tw-mt-12"><div class="tw-grid tw-grid-cols-1 tw-gap-16 sm:tw-grid-cols-2 lg:tw-grid-cols-3"><div class="pt-6"><div class="tw-flow-root tw-rounded-lg tw-px-6 tw-pb-8 tw-bg-gray-50 tw-drop-shadow-md"><div class="tw--mt-6"><div><span class="tw-inline-flex tw-items-center tw-justify-center tw-rounded-md tw-bg-gradient-to-r tw-from-purple-500 tw-to-pink-500 tw-p-3 tw-shadow-lg"><svg stroke="currentColor" fill="currentColor" stroke-width="0" viewBox="0 0 496 512" class="tw-h-6 tw-w-6 tw-text-white" aria-hidden="true" height="1em" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg></span></div><h3 class="tw-mt-8 tw-text-lg tw-font-medium tw-tracking-tight tw-text-gray-900">Get started</h3><p class="tw-mt-5 tw-text-base tw-text-gray-500 tw-text-left">Start your first Colossal-AI project.</p><ul class="tw-list-disc tw-text-left"><li class="tw-text-left"><a href="docs/get_started/installation" class="tw-text-gray-500 hover:tw-no-underline">Download and installation</a></li><li class="tw-text-left"><a href="docs/get_started/run_demo" class="tw-text-gray-500 hover:tw-no-underline">Quick demo</a></li><li class="tw-text-left"><a href="https://github.com/hpcaitech/ColossalAI/tree/main/examples" class="tw-text-gray-500 hover:tw-no-underline">Usage examples</a></li></ul></div></div></div><div class="pt-6"><div class="tw-flow-root tw-rounded-lg tw-px-6 tw-pb-8 tw-bg-gray-50 tw-drop-shadow-md"><div class="tw--mt-6"><div><span class="tw-inline-flex tw-items-center tw-justify-center tw-rounded-md tw-bg-gradient-to-r tw-from-purple-500 tw-to-pink-500 tw-p-3 tw-shadow-lg"><svg stroke="currentColor" fill="currentColor" stroke-width="0" viewBox="0 0 448 512" class="tw-h-6 tw-w-6 tw-text-white" aria-hidden="true" height="1em" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M448 360V24c0-13.3-10.7-24-24-24H96C43 0 0 43 0 96v320c0 53 43 96 96 96h328c13.3 0 24-10.7 24-24v-16c0-7.5-3.5-14.3-8.9-18.7-4.2-15.4-4.2-59.3 0-74.7 5.4-4.3 8.9-11.1 8.9-18.6zM128 134c0-3.3 2.7-6 6-6h212c3.3 0 6 2.7 6 6v20c0 3.3-2.7 6-6 6H134c-3.3 0-6-2.7-6-6v-20zm0 64c0-3.3 2.7-6 6-6h212c3.3 0 6 2.7 6 6v20c0 3.3-2.7 6-6 6H134c-3.3 0-6-2.7-6-6v-20zm253.4 250H96c-17.7 0-32-14.3-32-32 0-17.6 14.4-32 32-32h285.4c-1.9 17.1-1.9 46.9 0 64z"></path></svg></span></div><h3 class="tw-mt-8 tw-text-lg tw-font-medium tw-tracking-tight tw-text-gray-900">Concepts</h3><p class="tw-mt-5 tw-text-base tw-text-gray-500 tw-text-left">Understand how Colossal-AI works.</p><ul class="tw-list-disc tw-text-left"><li class="tw-text-left"><a href="docs/concepts/colossalai_overview" class="tw-text-gray-500 hover:tw-no-underline">Overview</a></li><li class="tw-text-left"><a href="docs/concepts/distributed_training" class="tw-text-gray-500 hover:tw-no-underline">Distributed Training</a></li><li class="tw-text-left"><a href="docs/concepts/paradigms_of_parallelism/" class="tw-text-gray-500 hover:tw-no-underline">Paradigms of Parallelism</a></li></ul></div></div></div><div class="pt-6"><div class="tw-flow-root tw-rounded-lg tw-px-6 tw-pb-8 tw-bg-gray-50 tw-drop-shadow-md"><div class="tw--mt-6"><div><span class="tw-inline-flex tw-items-center tw-justify-center tw-rounded-md tw-bg-gradient-to-r tw-from-purple-500 tw-to-pink-500 tw-p-3 tw-shadow-lg"><svg stroke="currentColor" fill="currentColor" stroke-width="0" viewBox="0 0 16 16" class="tw-h-6 tw-w-6 tw-text-white" aria-hidden="true" height="1em" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M11.251.068a.5.5 0 0 1 .227.58L9.677 6.5H13a.5.5 0 0 1 .364.843l-8 8.5a.5.5 0 0 1-.842-.49L6.323 9.5H3a.5.5 0 0 1-.364-.843l8-8.5a.5.5 0 0 1 .615-.09z"></path></svg></span></div><h3 class="tw-mt-8 tw-text-lg tw-font-medium tw-tracking-tight tw-text-gray-900">Sample use cases</h3><p class="tw-mt-5 tw-text-base tw-text-gray-500 tw-text-left">Achieve the following with Colossal-AI:</p><ul class="tw-list-disc tw-text-left"><li class="tw-text-left"><a href="docs/advanced_tutorials/train_gpt_using_hybrid_parallelism" class="tw-text-gray-500 hover:tw-no-underline">Train GPT Using Hybrid Parallelism</a></li><li class="tw-text-left"><a href="docs/advanced_tutorials/meet_gemini" class="tw-text-gray-500 hover:tw-no-underline">Meet Gemini:The Heterogeneous Memory Manager of Colossal-AI</a></li></ul></div></div></div><div class="pt-6"><div class="tw-flow-root tw-rounded-lg tw-px-6 tw-pb-8 tw-bg-gray-50 tw-drop-shadow-md"><div class="tw--mt-6"><div><span class="tw-inline-flex tw-items-center tw-justify-center tw-rounded-md tw-bg-gradient-to-r tw-from-purple-500 tw-to-pink-500 tw-p-3 tw-shadow-lg"><svg stroke="currentColor" fill="currentColor" stroke-width="0" viewBox="0 0 24 24" aria-hidden="true" class="tw-h-6 tw-w-6 tw-text-white" height="1em" width="1em" xmlns="http://www.w3.org/2000/svg"><path fill-rule="evenodd" d="M2.25 6a3 3 0 013-3h13.5a3 3 0 013 3v12a3 3 0 01-3 3H5.25a3 3 0 01-3-3V6zm3.97.97a.75.75 0 011.06 0l2.25 2.25a.75.75 0 010 1.06l-2.25 2.25a.75.75 0 01-1.06-1.06l1.72-1.72-1.72-1.72a.75.75 0 010-1.06zm4.28 4.28a.75.75 0 000 1.5h3a.75.75 0 000-1.5h-3z" clip-rule="evenodd"></path></svg></span></div><h3 class="tw-mt-8 tw-text-lg tw-font-medium tw-tracking-tight tw-text-gray-900">Command Line Interface (CLI)</h3><p class="tw-mt-5 tw-text-base tw-text-gray-500 tw-text-left">The Colossal-AI Command Line Interface is a unified tool to manage your Colossal-AI projects.</p><ul class="tw-list-disc tw-text-left"><li class="tw-text-left"><a href="docs/basics/command_line_tool#introduction" class="tw-text-gray-500 hover:tw-no-underline">Introduction</a></li><li class="tw-text-left"><a href="docs/basics/command_line_tool#launcher" class="tw-text-gray-500 hover:tw-no-underline">Launch distributed jobs</a></li><li class="tw-text-left"><a href="docs/basics/command_line_tool#tensor-parallel-micro-benchmarking" class="tw-text-gray-500 hover:tw-no-underline">Tensor Parallel Micro-Benchmarking</a></li></ul></div></div></div><div class="pt-6"><div class="tw-flow-root tw-rounded-lg tw-px-6 tw-pb-8 tw-bg-gray-50 tw-drop-shadow-md"><div class="tw--mt-6"><div><span class="tw-inline-flex tw-items-center tw-justify-center tw-rounded-md tw-bg-gradient-to-r tw-from-purple-500 tw-to-pink-500 tw-p-3 tw-shadow-lg"><svg stroke="currentColor" fill="currentColor" stroke-width="0" viewBox="0 0 16 16" class="tw-h-6 tw-w-6 tw-text-white" aria-hidden="true" height="1em" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M9.405 1.05c-.413-1.4-2.397-1.4-2.81 0l-.1.34a1.464 1.464 0 0 1-2.105.872l-.31-.17c-1.283-.698-2.686.705-1.987 1.987l.169.311c.446.82.023 1.841-.872 2.105l-.34.1c-1.4.413-1.4 2.397 0 2.81l.34.1a1.464 1.464 0 0 1 .872 2.105l-.17.31c-.698 1.283.705 2.686 1.987 1.987l.311-.169a1.464 1.464 0 0 1 2.105.872l.1.34c.413 1.4 2.397 1.4 2.81 0l.1-.34a1.464 1.464 0 0 1 2.105-.872l.31.17c1.283.698 2.686-.705 1.987-1.987l-.169-.311a1.464 1.464 0 0 1 .872-2.105l.34-.1c1.4-.413 1.4-2.397 0-2.81l-.34-.1a1.464 1.464 0 0 1-.872-2.105l.17-.31c.698-1.283-.705-2.686-1.987-1.987l-.311.169a1.464 1.464 0 0 1-2.105-.872l-.1-.34zM8 10.93a2.929 2.929 0 1 1 0-5.86 2.929 2.929 0 0 1 0 5.858z"></path></svg></span></div><h3 class="tw-mt-8 tw-text-lg tw-font-medium tw-tracking-tight tw-text-gray-900">Configuration</h3><p class="tw-mt-5 tw-text-base tw-text-gray-500 tw-text-left">Define your Colossal-AI project configuration as per your needs.</p><ul class="tw-list-disc tw-text-left"><li class="tw-text-left"><a href="docs/basics/define_your_config#configuration-definition" class="tw-text-gray-500 hover:tw-no-underline">Introduction</a></li><li class="tw-text-left"><a href="docs/basics/define_your_config#feature-specification" class="tw-text-gray-500 hover:tw-no-underline">Feature specification</a></li><li class="tw-text-left"><a href="docs/basics/define_your_config#global-hyper-parameters" class="tw-text-gray-500 hover:tw-no-underline">Global hyper-parameter</a></li></ul></div></div></div><div class="pt-6"><div class="tw-flow-root tw-rounded-lg tw-px-6 tw-pb-8 tw-bg-gray-50 tw-drop-shadow-md"><div class="tw--mt-6"><div><span class="tw-inline-flex tw-items-center tw-justify-center tw-rounded-md tw-bg-gradient-to-r tw-from-purple-500 tw-to-pink-500 tw-p-3 tw-shadow-lg"><svg stroke="currentColor" fill="currentColor" stroke-width="0" viewBox="0 0 16 16" class="tw-h-6 tw-w-6 tw-text-white" aria-hidden="true" height="1em" width="1em" xmlns="http://www.w3.org/2000/svg"><path fill-rule="evenodd" d="M8 0a5.53 5.53 0 0 0-3.594 1.342c-.766.66-1.321 1.52-1.464 2.383C1.266 4.095 0 5.555 0 7.318 0 9.366 1.708 11 3.781 11H7.5V5.707L5.354 7.854a.5.5 0 1 1-.708-.708l3-3a.5.5 0 0 1 .708 0l3 3a.5.5 0 0 1-.708.708L8.5 5.707V11h4.188C14.502 11 16 9.57 16 7.773c0-1.636-1.242-2.969-2.834-3.194C12.923 1.999 10.69 0 8 0zm-.5 14.5V11h1v3.5a.5.5 0 0 1-1 0z"></path></svg></span></div><h3 class="tw-mt-8 tw-text-lg tw-font-medium tw-tracking-tight tw-text-gray-900">Do you use Colossal-AI?</h3><p class="tw-mt-5 tw-text-base tw-text-gray-500 tw-text-left">If you are a happy user of our open source Colossal-AI software and implemented a deep learning project with it, please let us know.</p><ul class="tw-list-disc tw-text-left"><li class="tw-text-left"><a href="https://www.hpc-ai.tech/customers/submit?__hstc=252372566.013713e29a075542913de94d73b6e914.1674904620480.1675777274014.1676100247206.3&__hssc=252372566.4.1676100247206&__hsfp=1942671557" class="tw-text-gray-500 hover:tw-no-underline">Submit your Colossal-AI project</a></li></ul></div></div></div></div></div></div></div></main></div><footer class="footer"><div class="container container-fluid"><div class="row footer__links"><div class="col footer__col"><div class="footer__title">Resources</div><ul class="footer__items clean-list"><li class="footer__item"><a class="footer__link-item" href="/docs/get_started/installation">Tutorials</a></li><li class="footer__item"><a href="https://github.com/hpcaitech/ColossalAI/tree/main/examples" target="_blank" rel="noopener noreferrer" class="footer__link-item">Examples</a></li><li class="footer__item"><a href="https://github.com/hpcaitech/ColossalAI/discussions" target="_blank" rel="noopener noreferrer" class="footer__link-item">Forum</a></li></ul></div><div class="col footer__col"><div class="footer__title">Community</div><ul class="footer__items clean-list"><li class="footer__item"><a href="https://github.com/hpcaitech/ColossalAI" target="_blank" rel="noopener noreferrer" class="footer__link-item">GitHub</a></li><li class="footer__item"><a href="https://www.hpc-ai.tech/blog" target="_blank" rel="noopener noreferrer" class="footer__link-item">Blog</a></li><li class="footer__item"><a href="https://twitter.com/HPCAITech" target="_blank" rel="noopener noreferrer" class="footer__link-item">Twitter</a></li></ul></div><div class="col footer__col"><div class="footer__title">About</div><ul class="footer__items clean-list"><li class="footer__item"><a href="https://www.hpc-ai.tech/" target="_blank" rel="noopener noreferrer" class="footer__link-item">Company</a></li><li class="footer__item"><a href="https://www.hpc-ai.tech/services" target="_blank" rel="noopener noreferrer" class="footer__link-item">Services</a></li><li class="footer__item"><a href="https://www.hpc-ai.tech/customers" target="_blank" rel="noopener noreferrer" class="footer__link-item">Customers</a></li></ul></div></div><div class="footer__bottom text--center"><div class="footer__copyright">Copyright © 2024 All Rights Reserved by HPC-AI Technology Inc.</div></div></div></footer></div> <script src="/assets/js/runtime~main.6b0f350a.js"></script> <script src="/assets/js/main.c6c31bc7.js"></script> </body> </html>