CINXE.COM

<!DOCTYPE html><html data-wf-domain="home.convai.com" data-wf-page="659c2b6c4883f18a0487ce17" data-wf-site="63ea75dcbd94093302f1d430" lang="en" data-wf-collection="659c2b6c4883f18a0487cdd7" data-wf-item-slug="realistic-face-animations-ai-characters-convai"><head><meta charset="utf-8"/><title>Realistic Face Animations for AI Characters Using Convai</title><meta content="Create lifelike AI characters with Convai's lip sync and facial animation components. Customize animations, map visemes, integrate with Unreal Engine | Convai" name="description"/><meta content="Realistic Face Animations for AI Characters Using Convai" property="og:title"/><meta content="Create lifelike AI characters with Convai's lip sync and facial animation components. Customize animations, map visemes, integrate with Unreal Engine | Convai" property="og:description"/><meta content="Realistic Face Animations for AI Characters Using Convai" property="twitter:title"/><meta content="Create lifelike AI characters with Convai's lip sync and facial animation components. Customize animations, map visemes, integrate with Unreal Engine | Convai" property="twitter:description"/><meta property="og:type" content="website"/><meta content="summary_large_image" name="twitter:card"/><meta content="width=device-width, initial-scale=1" name="viewport"/><link href="https://cdn.prod.website-files.com/63ea75dcbd94093302f1d430/css/convai.webflow.e4b33a079.css" rel="stylesheet" type="text/css"/><script type="text/javascript">!function(o,c){var n=c.documentElement,t=" w-mod-";n.className+=t+"js",("ontouchstart"in o||o.DocumentTouch&&c instanceof DocumentTouch)&&(n.className+=t+"touch")}(window,document);</script><link href="https://cdn.prod.website-files.com/63ea75dcbd94093302f1d430/63ee2a7074c2ed7891cf985d_favicon.png" rel="shortcut icon" type="image/x-icon"/><link href="https://cdn.prod.website-files.com/63ea75dcbd94093302f1d430/63ee2a726f64513adb30ebe2_webclip.png" rel="apple-touch-icon"/><link href="https://convai.com/blog/realistic-face-animations-ai-characters-convai" rel="canonical"/><script async="" src="https://www.googletagmanager.com/gtag/js?id=G-S3F0DFJ6RR"></script><script type="text/javascript">window.dataLayer = window.dataLayer || [];function gtag(){dataLayer.push(arguments);}gtag('js', new Date());gtag('set', 'developer_id.dZGVlNj', true);gtag('config', 'G-S3F0DFJ6RR');</script> <script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start': new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0], j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src= 'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f); })(window,document,'script','dataLayer','GTM-KF8NNT39');</script>   <script async src="https://www.googletagmanager.com/gtag/js?id=G-S3F0DFJ6RR"></script> <script> window.dataLayer = window.dataLayer || []; function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); gtag('config', 'G-S3F0DFJ6RR'); </script></head><body class="body"><div class="c-nav-wrapper"><div data-animation="default" data-collapse="medium" data-duration="400" data-easing="ease-in-out-cubic" data-easing2="ease-in-out-cubic" role="banner" class="c-nav-2 w-nav"><div class="c-nav-container w-container"><a href="/" class="c-nav_brand w-nav-brand"><img src="https://cdn.prod.website-files.com/63ea75dcbd94093302f1d430/63ea7cbe26f9881f3c6960be_convai-logo.svg" loading="lazy" alt="Convai logo - AI NPCs"/></a><nav role="navigation" class="c-nav_menu-links-2 w-nav-menu"><a href="#" id="Feature" data-tracker="Navbar-Features|Landing" class="c-nav_link w-nav-link">Features</a><a href="#" data-tracker="Navbar-UseCases|Landing" class="c-nav_link w-nav-link">Use Cases</a><a href="/gallery" data-tracker="Navbar-Integrations|Landing" class="c-nav_link webflow-interpage w-nav-link">Gallery</a><a href="https://docs.convai.com/" data-tracker="Navbar-Docs|Landing" target="_blank" class="c-nav_link w-nav-link">Docs</a><a href="https://convai.com/blog" data-tracker="Navbar-Blog|Landing" target="_blank" class="c-nav_link w-nav-link">Blog</a><a href="https://convai.com/pricing" data-tracker="Navbar-Pricing|Landing" target="_blank" class="c-nav_link w-nav-link">Pricing</a><a href="https://docs.convai.com/api-docs-restructure/plugins-and-integrations/introduction" data-tracker="Navbar-Plugins|Landing" target="_blank" class="c-nav_link w-nav-link">Plugins</a><a href="https://www.youtube.com/@convai/videos" data-tracker="Navbar-Videos|Landing" target="_blank" class="c-nav_link cc-last w-nav-link">Videos</a></nav><a id="navBar-sign-in-button" data-tracker="Navbar-SignIn|Landing" href="https://convai.com/auth/login" class="c-button-3 cc-nav w-inline-block"><div>Sign In</div></a><div class="c-nav_button-2 w-nav-button"><div class="c-nav_open-icon w-embed"><svg width="25" height="19" viewBox="0 0 25 19" fill="none" xmlns="http://www.w3.org/2000/svg"> <line x1="1.68188" y1="1.19922" x2="24" y2="1.19922" stroke="white" stroke-width="2" stroke-linecap="round"/> <line x1="10.6357" y1="9.19922" x2="24.0001" y2="9.19922" stroke="white" stroke-width="2" stroke-linecap="round"/> <line x1="1.68188" y1="17.1992" x2="24" y2="17.1992" stroke="white" stroke-width="2" stroke-linecap="round"/> </svg></div><div class="c-nav_close-icon w-embed"><svg width="19" height="19" viewBox="0 0 19 19" fill="none" xmlns="http://www.w3.org/2000/svg"> <line x1="1.80444" y1="17.1836" x2="17.5857" y2="1.40227" stroke="white" stroke-width="2" stroke-linecap="round"/> <line x1="1.84683" y1="1.40222" x2="17.6281" y2="17.1835" stroke="white" stroke-width="2" stroke-linecap="round"/> </svg></div></div></div></div></div><div class="c-custom-code w-embed"><style> /* ----- Settings ----- */ body { -webkit-font-smoothing: antialiased; /*font-size: 1.1111111111111112vw;*/ } a { color: inherit } .c-nav_link:before, .c-footer_link:before { content: url(https://cdn.prod.website-files.com/63ea75dcbd94093302f1d430/63ef7342ee092d1b9c953eeb_arrow.svg); display: block; width: 13px; height: 13px; position: absolute; right: -2px; top: 13px; opacity: 0; transform: translate(-5px, 5px); -webkit-transition: .2s cubic-bezier(.645, .045, .355, 1); transition: .2s cubic-bezier(.645, .045, .355, 1); } .c-footer_link:before { opacity: 0; right: -30px; top: 0px; } .c-nav_link:hover:before, .c-footer_link:hover:before { opacity: 1; transform: translate(0px, 0px); } /* Max Font Size */ @media screen and (min-width:1440px) { body {font-size: 1rem;} } /* ----- Utility Attributes ----- */ [no-pointer="1"] { pointer-events: none } /* ----- Navbar ----- */ .c-nav:has(.c-nav_button.w--open) { border-bottom-left-radius: 0; border-bottom-right-radius: 0; border-bottom: 0; background-color: rgba(3, 16, 9, 0.85); backdrop-filter: blur(0px); } .c-nav_button.w--open .c-nav_open-icon { opacity: 0; } .c-nav_button.w--open .c-nav_close-icon { opacity: 1; } </style></div><div class="post-header"><div class="container w-container"><div class="featured-post-details post-page"><h1 class="heading">Realistic Face Animations for AI Characters</h1><div class="hr-line"></div><div class="w-layout-hflex flex-block w-clearfix"><div class="author-div post-page"><a href="#" class="author w-inline-block"><img alt="" src="https://cdn.prod.website-files.com/63ea75dcbd94095356f1d43b/6503808c9f830c92c94cd831_convai_blob.jpeg" class="image-17"/></a><div class="div-block-17"><div class="text-block-4">By </div><div>Convai Team</div></div></div><div class="space-between"></div><div class="publish-date"><div class="text-block-6">May 30, 2024</div></div></div></div></div></div><div style="background-image:url("https://cdn.prod.website-files.com/63ea75dcbd94095356f1d43b/66587ba94b340c00df188486_banner_hero.png")" class="featued-image-image"></div><div class="post-body"><div class="post-container w-container"><div class="post-container1"><div class="text-block1 w-richtext"><p>Creating realistic and expressive facial animations for AI characters is a complex challenge in developing virtual worlds and interactive experiences. Developers and animators often struggle to achieve natural and believable facial movements that accurately synchronize with speech and emotions. This can lead to a disconnect between the character's appearance and behavior, diminishing the overall immersion and engagement of users.</p><p>Fortunately, recent advancements in AI (LLMs, diffusion models, agents) have given rise to new solutions that promise to revolutionize lip sync and facial animation. Convai’s plugin for platforms such as <a href="https://docs.convai.com/api-docs/plugins-and-integrations/unreal-engine">Unreal Engine</a>, <a href="https://docs.convai.com/api-docs/plugins-and-integrations/unity-plugin">Unity</a>, <a href="https://docs.convai.com/api-docs/plugins-and-integrations/roblox">Roblox</a>, etc., includes components for applying context-aware face animations and lip-sync to talking characters.</p><p>In this article, we will:</p><ul role="list"><li>Explore the challenges of traditional lip sync and facial animation methods.</li><li>Introduce Convai's AI-powered lip sync and facial animation components.</li><li>Discuss the benefits of using AI for lip sync and facial animation, including increased efficiency, reduced costs, and improved realism.</li><li>Provide an overview of Convai's workflow and integration with popular game engines like Unreal Engine.</li></ul><p>By the end of this article, you will know how to create highly realistic characters with precise lip sync and expressive facial animations.</p><p><strong>See Also: </strong><a href="https://convai.com/blog/lip-syncing-virtual-ai-characters-techniques-integration-and-future-trends"><strong>Lip-Syncing Virtual AI Characters with Convai in Unreal Engine</strong></a><strong>.</strong></p><p>‍</p><h2>Importance of Facial Expressions in Creating Realistic AI Characters</h2><p>Facial expressions are crucial for creating realistic AI characters, as they convey emotions and enhance the overall immersive experience for users. Realistic facial animations can improve user engagement by making interactions with virtual characters more relatable and lifelike. </p><p>Expressive faces help communicate non-verbal cues essential for natural and effective communication. Without accurate facial expressions, AI characters may appear robotic and fail to evoke the intended emotional responses from users.</p><figure style="max-width:1010pxpx" class="w-richtext-align-fullwidth w-richtext-figure-type-image"><div><img src="https://cdn.prod.website-files.com/63ea75dcbd94095356f1d43b/6654cdb52e3fc434f9cd5b02_Lip%20Sync%20and%20Facial%20Expression%20Demo%20Convai.gif" loading="lazy" alt=""/></div></figure><h3>Use of ARKit Blend Shapes</h3><p>ARKit blend shapes provide a powerful tool for creating dynamic facial expressions. These blend shapes are based on the Facial Action Coding System (<a href="https://imotions.com/blog/learning/research-fundamentals/facial-action-coding-system/">FACS</a>), which breaks down facial movements into individual action units (eyebrows, eyes, mouth, and cheeks). </p><p>Combining these action units can achieve various facial expressions that reflect various emotions. ARKit offers <a href="https://developer.apple.com/documentation/arkit/arfaceanchor/2928251-blendshapes/">52 distinct blend shapes</a>, such as <a href="https://developer.apple.com/documentation/arkit/arfaceanchor/blendshapelocation/2928249-mouthsmileleft">mouthSmile_L</a>, <a href="https://developer.apple.com/documentation/arkit/arfaceanchor/blendshapelocation/2928262-eyeblinkright">eyeBlink_R</a>, and <a href="https://developer.apple.com/documentation/arkit/arfaceanchor/blendshapelocation/2928264-browinnerup">browInnerUp</a>, which can be used to create nuanced emotions.</p><p>ARKit blend shapes are particularly effective because they provide a standardized way to achieve high-quality facial animations across different platforms and devices. These blend shapes are integral to generating realistic facial movements synchronized with audio for lip-syncing.</p><p>‍</p><h3>Predefined Poses for Emotions</h3><p>You use predefined poses—sets of blend shape values representing specific emotions—to simplify the creation of emotional expressions. For instance, a “happy” pose might involve raised eyebrows, a wide smile, and lifted cheeks, while a “sad” pose might include furrowed brows, a downturned mouth, and drooping eyelids.</p><p>Here are some common predefined poses:</p><ul role="list"><li><strong>Happy</strong>: Raised eyebrows, wide smile, lifted cheeks</li><li><strong>Sad:</strong> Furrowed brows, downturned mouth, drooping eyelids</li><li><strong>Angry</strong>: Lowered brows, pursed lips, flared nostrils</li><li><strong>Surprised:</strong> Wide eyes, raised eyebrows, slightly open mouth</li><li><strong>Fear: </strong>Tense facial muscles, wide eyes, slightly open mouth</li></ul><p>Using these predefined poses, you can quickly apply realistic facial expressions to AI characters to improve their emotional depth and make them more engaging.</p><figure style="max-width:1818pxpx" class="w-richtext-align-fullwidth w-richtext-figure-type-image"><div><img src="https://cdn.prod.website-files.com/63ea75dcbd94095356f1d43b/665878b0d5da08a8a1426b4e_image3.png" loading="lazy" alt=""/></div><figcaption><em>Context-aware character emotion state with Convai.</em></figcaption></figure><p>‍</p><h3>Blending Multiple Emotions</h3><p>In real-world interactions, people often experience and express mixed emotions simultaneously. To capture this complexity, your AI character must seamlessly blend multiple emotions. This involves combining different predefined poses based on the intensity of each emotion. </p><p>For example, a character might be primarily happy but also slightly surprised. The system would blend the "happy" and "surprised" poses, resulting in a nuanced expression that reflects both emotions.</p><p>The process of blending multiple emotions involves:</p><ul role="list"><li><strong>Receiving Emotional Data: </strong>The backend system analyzes the conversation's context and the <a href="https://docs.convai.com/api-docs/convai-playground/character-creator-tool/personality-and-style">character's emotional state</a>.</li><li><strong>Calculating Blend Shape Values: </strong>The system calculates each emotion's appropriate blend shape value based on the emotional data.</li><li><strong>Blending Poses: </strong>The system blends predefined poses according to the calculated values for smooth emotional transitions.</li><li><strong>Applying to the Character: </strong>The blended facial expression is then applied to the AI character in real-time to create a lifelike and dynamic emotional response.</li></ul><p>When you blend multiple emotions, AI characters can exhibit a wide range of expressive behaviors that make interactions with them more realistic and engaging. This is essential for gaming, virtual reality, and other interactive media applications where emotional authenticity improves the user experience.</p><p>‍</p><h2>Integration and Workflow for AI Character Facial Expressions in Convai</h2><p>Convai's lip sync and facial animation system is designed to provide seamless integration and an efficient workflow across various platforms. Convai ensures optimal performance and customization options for creating realistic character animations using server-side computation and client-side synchronization.</p><p>‍</p><h3>Server-Side Computation of Lip Sync and Facial Animation</h3><p>Convai handles the heavy lifting of generating lip sync and facial animation data on the server side. This approach reduces client-side performance impact by offloading the computation to the server. </p><p>Client devices, especially those with limited processing power like mobile phones or web browsers, can focus on rendering and other tasks. This ensures a smooth user experience without overburdening the client's hardware.</p><figure style="max-width:960pxpx" class="w-richtext-align-fullwidth w-richtext-figure-type-image"><div><img src="https://cdn.prod.website-files.com/63ea75dcbd94095356f1d43b/665878db43f81223fa5ecf03_image7.png" loading="lazy" alt=""/></div></figure><p>‍</p><h3>Client-Side Integration in Unreal Engine, Unity, WebGL</h3><p>Convai provides <a href="https://docs.convai.com/api-docs/plugins-and-integrations/introduction">integration support for popular game engines and platforms</a> (Unity, Unreal Engine, Roblox, WebGL, Discord). This helps if you want to incorporate lip sync and facial animation into your projects.</p><p>On the client side, platforms like Unreal Engine, Unity, and WebGL synchronize the audio playback with the lip sync frames received from the server. Convai's plugin handles this synchronization process so that the character's lips move in perfect harmony with the audio. </p><p>Each animation frame is carefully timed to correspond with the audio, making the characters' speech appear natural and fluid. Synchronization is crucial for creating a believable and immersive character performance.</p><figure style="padding-bottom:56.206088992974244%" class="w-richtext-align-fullwidth w-richtext-figure-type-video"><div><iframe allowfullscreen="true" frameborder="0" scrolling="no" src="https://www.youtube.com/embed/jqNttRE3nD4" title="Integrate AI Characters with MetaHuman Avatars in Unreal Engine 5 | Convai Unreal Engine Tutorial"></iframe></div></figure><p>‍</p><h3>Developer Customization Options</h3><p>Convai offers flexibility and customization options to fine-tune the lip sync and facial animation system to your needs as a developer.</p><p>‍</p><h4>Enabling/Disabling Features in Initial Request to Server</h4><p>When sending the initial request to the Convai server, you can control which features to enable or disable. For example, you can receive only the audio without lip sync data or request both audio and lip sync frames for comprehensive character animation. </p><p>This granular control allows you to optimize performance and tailor the system to the project's requirements.</p><p>‍</p><h4>Mapping Visemes to Custom Character Blend Shapes</h4><p>For custom characters with unique blend shapes, developers can map the visemes (visual representations of speech sounds) to their character's specific blend shapes. Convai provides a tutorial and step-by-step guidance on how to perform this mapping process.</p><p>You can ensure the lip sync looks natural and matches the character's unique facial structure by customizing the viseme-to-blend shape mapping.</p><p>‍</p><h2>Customizing Facial Animations and Lip-Sync in Unreal Engine</h2><p>In Unreal Engine, you can fine-tune lip-sync and facial animations to achieve desired results for your AI characters. Convai plugin allows developers to adjust and customize how characters' mouths and facial features move during speech.</p><p>This customization is crucial for ensuring that animations align with the project's specific visual style and requirements, whether it's a realistic or stylized character.</p><p>The video below is a tutorial on integrating custom characters with Convai in Unreal Engine and adding components like facial animations and lip sync.</p><figure style="padding-bottom:56.206088992974244%" class="w-richtext-align-fullwidth w-richtext-figure-type-video"><div><iframe allowfullscreen="true" frameborder="0" scrolling="no" src="https://www.youtube.com/embed/69ueqmUShq4" title="AI Characters with Custom Avatars in Unreal Engine | Convai Unreal Engine Tutorial"></iframe></div></figure><p>Here are the steps from that video on how to customize lip-sync and facial animations in Unreal Engine:</p><ol role="list"><li>Create a new Convai integration folder in the content folder.</li><li>Create a new blueprint for your character and name it Echo. For the parent class, select <a href="https://docs.convai.com/api-docs/plugins-and-integrations/unreal-engine/guides/creating-metahuman-characters/change-the-parent-class-for-metahuman#steps-to-change-parent-class-for-metahuman">Convai Base Character</a>.</li><li>Add your character's skeletal mesh to the blueprint.</li></ol><p>‍</p><h3>Set Up Face Animations</h3><ol role="list"><li>Download the latest Convai Reallusion animation blueprint using <a href="https://drive.google.com/drive/u/2/folders/1n_mj_CBe_i63iihkdU09LFkEJrXgPs47">this link</a>.</li><li>You will add animations in the body animation section of the animation graph.</li><li>Connect your <em>idle</em>, walking, and transition animations.</li><li>Select the Echo’s head bone in the head rotation section to allow it to look at the player.</li><li>Assign Echo's right and left eye bones in their respective sections for eye animation. If you want Echo to have facial emotions, you can add animations for each emotion in the emotions layer.</li></ol><p>‍</p><h3>Set Up Lip Sync</h3><p>Convai currently uses viseme-based animations for lip sync rather than blend shapes or ARKit. In future releases, the plugin will support ARKit blend shapes. To set up lip sync, map each visem to Echo's corresponding blend shape or bones. </p><p><strong>Here's how to do it:</strong></p><ol role="list"><li>Open <strong>Echo skeleton mesh</strong> and navigate to the morph targets section. Identify the relevant blend shapes or bones in Echo's morph targets or skeleton.</li><li>In the lips sync layer, <strong>map each viseme to Echo's corresponding blend shape or bones</strong>. For each viseme, replace the existing blend shapes node with a “modify” curve node. Right-click on the modify curve node and select add curve for each relevant blend shape. Adjust the values for Echo’s specific facial structure.</li><li><strong>Compile the animation blueprint</strong> to ensure there are no errors.</li></ol><p>Note that the main challenge in this process is the mapping from visemes to your character's specific blend shapes or bones. This step can vary significantly depending on your character's design and facial structure. </p><p>Take your time to carefully identify the relevant blend shapes or bones for each viseme and adjust as needed to achieve the desired lip sync animation.</p><figure style="max-width:1010pxpx" class="w-richtext-align-fullwidth w-richtext-figure-type-image"><div><img src="https://cdn.prod.website-files.com/63ea75dcbd94095356f1d43b/66587939a1f3fb4372c8bf15_image5.gif" loading="lazy" alt=""/></div><figcaption><em>Viseme-to-blend shape mapping for Echo in Unreal Engine with Convai plugin.</em></figcaption></figure><p>‍</p><h2>Recent Improvements to Convai’s Plugin for Face Animations and Lip Syncing</h2><p>Convai lip sync and facial animation components have undergone significant advancements over the past year. These improvements have improved the realism and accuracy of the animations, resulting in a more immersive and engaging experience for users.</p><p>‍</p><h3>Improved Viseme-Based Lip Sync Accuracy</h3><p>One of the key improvements has been the optimization of the viseme-based lip sync system. Previously, the lip movements were often inaccurate and noisy, with fast fluctuations that appeared unnatural.</p><figure style="max-width:1012pxpx" class="w-richtext-align-fullwidth w-richtext-figure-type-image"><div><img src="https://cdn.prod.website-files.com/63ea75dcbd94095356f1d43b/665879544ab0aea1a43c6d73_image4.gif" loading="lazy" alt=""/></div><figcaption><em>Lip Syncing and Facing Animations with the Convai Plugin in 2023.</em></figcaption></figure><p>However, by implementing server-side post-processing techniques, Convai has significantly improved the accuracy and smoothness of the lip sync, resulting in more realistic animations.</p><p>‍</p><h3>Introduction of Audio2Face for ARKit Blend Shapes (in Beta)</h3><p>We have also started rolling out NVIDIA's Audio2Face on the Convai server. Unlike the viseme-based system, Audio2Face outputs ARKit blend shapes directly for more advanced and expressive solution. </p><p>It generates blend shapes not only for the lips but also for the eyebrows, eyes, and the entire face, enabling a wider range of emotions and expressions. Although Audio2Face is computationally expensive and in beta, it is gradually being rolled out for high-tier plans and partners.</p><figure style="padding-bottom:56.206088992974244%" class="w-richtext-align-fullwidth w-richtext-figure-type-video"><div><iframe allowfullscreen="true" frameborder="0" scrolling="no" src="https://www.youtube.com/embed/X_2X2uLZyL4" title="AI NPCs are the future of video games - Jensen Huang | Nvidia Convai partnership"></iframe></div></figure><p>‍</p><h3>Transition to AI-Generated Facial Expressions to Match Voice</h3><p>Looking ahead, Convai is excited about the ongoing work on AI-generated facial expressions. We are training models to output emotional voice, emotional lip sync, and corresponding facial expressions. </p><p>Our users want to move beyond preset poses and achieve even greater realism. This transition will ensure that your character's voice, facial expressions, and lip sync are seamlessly synchronized to improve the overall believability of the animations.</p><p>‍</p><h3>Applications in Brand Agents, Gaming, and Education</h3><p>Advanced lip sync and facial animation technology open up many potential applications. In the gaming industry, these improvements can lead to more immersive and engaging character interactions, enhancing the overall player experience. </p><figure style="max-width:1080pxpx" class="w-richtext-align-fullwidth w-richtext-figure-type-image"><div><img src="https://cdn.prod.website-files.com/63ea75dcbd94095356f1d43b/6658798d55dbae01a16fa2a7_image2.png" loading="lazy" alt=""/></div></figure><p>Brand agents can benefit from more natural and expressive communication, making the interaction more human-like. Realistic facial animations can also help the education sector create compelling e-learning content.</p><p>‍</p><h2>Conclusion</h2><p>Great! Throughout this article, we reviewed face animation, lip-syncing, and the complexities of creating realistic and expressive animations. We reviewed how you could use the Convai plugin on a platform like Unreal Engine to integrate face animations and apply lip-syncing to your character.</p><p>The integration process involves server-side computation for performance optimization and client-side synchronization for seamless playback. You also have customization options for tailoring animations to specific characters and use cases.</p><p>Convai is actively working on integrating facial capture technology and AI-generated facial expressions for even greater realism and expressiveness. Convai empowers developers to create engaging and believable character animations with AI across various platforms.</p><figure style="max-width:1999pxpx" class="w-richtext-align-fullwidth w-richtext-figure-type-image"><div><img src="https://cdn.prod.website-files.com/63ea75dcbd94095356f1d43b/665879a7678cbcbaac4140d8_image1.png" loading="lazy" alt=""/></div><figcaption><em>Build AI-enchanced characters with Convai.</em></figcaption></figure><p>Check out our other <a href="https://convai.com/blog/lip-syncing-virtual-ai-characters-techniques-integration-and-future-trends">article on lip-syncing AI characters</a>, and sign up for our <a href="https://discord.gg/aHUYXX3gW">Discord</a> if you have questions about using Convai.</p></div><div class="video-2 w-dyn-bind-empty w-video w-embed"></div><img src="https://cdn.prod.website-files.com/plugins/Basic/assets/placeholder.60f9b1840c.svg" loading="lazy" alt="" class="image2 w-dyn-bind-empty"/><div class="rich-text-block2 w-dyn-bind-empty w-richtext"></div><img src="" loading="lazy" alt="" class="image3 w-dyn-bind-empty"/><div class="rich-text-block3 w-dyn-bind-empty w-richtext"></div><div class="div-block-27"><img src="https://cdn.prod.website-files.com/plugins/Basic/assets/placeholder.60f9b1840c.svg" loading="lazy" width="284" alt="" class="image4 w-dyn-bind-empty"/></div><div class="rich-text-4 w-dyn-bind-empty w-richtext"></div><img src="https://cdn.prod.website-files.com/plugins/Basic/assets/placeholder.60f9b1840c.svg" loading="lazy" alt="" class="image5 w-dyn-bind-empty"/><div class="rich-text-block-5 w-dyn-bind-empty w-richtext"></div><img src="https://cdn.prod.website-files.com/plugins/Basic/assets/placeholder.60f9b1840c.svg" loading="lazy" alt="" class="image-6 w-dyn-bind-empty"/><div class="rich-text-block6 w-dyn-bind-empty w-richtext"></div><img src="https://cdn.prod.website-files.com/plugins/Basic/assets/placeholder.60f9b1840c.svg" loading="lazy" alt="" class="image-7 w-dyn-bind-empty"/><div class="conclusion w-dyn-bind-empty w-richtext"></div><div id="w-node-_2a3fae0a-171c-313c-4a59-c25206ce3618-0487ce17" class="w-layout-layout wf-layout-layout"><div id="w-node-_2a3fae0a-171c-313c-4a59-c25206ce3619-0487ce17" class="w-layout-cell"></div><div id="w-node-_2a3fae0a-171c-313c-4a59-c25206ce361a-0487ce17" class="w-layout-cell"><a href="https://convai.com/auth/login" class="c-button w-button">Build a Character for Free</a></div><div id="w-node-c6bfd143-c815-1881-56a2-77a6ceaadca6-0487ce17" class="w-layout-cell"><a href="mailto:support@convai.com" class="c-button w-button">Ask a Question</a></div><div id="w-node-ea858fcd-6142-ec13-066d-37637a6854de-0487ce17" class="w-layout-cell"></div></div></div></div></div><footer class="c-footer"><div class="c-custom-code w-embed"><style> .c-footer_form-wrapper::before { content: ''; position: absolute; inset: 0; background: linear-gradient(180deg, rgba(25, 189, 85, 0) 0%, rgba(25, 189, 85, 0.37) 206.53%, #19BD55 496.43%);; width: 100%; height: 100%; pointer-events: none; opacity: 0; transition: .3s cubic-bezier(.645, .045, .355, 1); } @media screen and (max-width: 991px) { .c-footer_form-wrapper::before { height: 48px; } } .c-footer_form-wrapper:has(:focus)::before { opacity: 1; } input:-webkit-autofill, input:-webkit-autofill:hover, input:-webkit-autofill:focus, input:-webkit-autofill:active { -webkit-text-fill-color: white !important; -webkit-background-clip: text; } </style></div><img src="https://cdn.prod.website-files.com/63ea75dcbd94093302f1d430/63ebbfd37a72985157e1bd9f_decoration.svg" loading="lazy" alt="" class="image-8"/><div class="c-footer-wrapper"><div class="c-container"><div class="c-footer_form-block w-form"><form id="email-form" name="email-form" data-name="Email Form" method="get" class="c-footer_form" data-wf-page-id="659c2b6c4883f18a0487ce17" data-wf-element-id="5c3d93f3-e017-32f8-d833-5bdfa4c29965"><div class="c-footer_form-label"><label for="email" class="c-title-7">Subscribe to our newsletter:</label></div><div class="c-footer_form-wrapper"><input class="c-footer_form-field w-input" maxlength="256" name="email" data-name="Email" placeholder="enter your email here" type="email" id="footer-email" required=""/><input type="submit" data-wait="..." class="c-footer_form-submit w-button" value="Send Email"/></div></form><div class="success-message w-form-done"><div>Thank you! Your submission has been received!</div></div><div class="error-message w-form-fail"><div>Oops! Something went wrong while submitting the form.</div></div></div><div class="c-footer_socials"><div class="c-social-embed cc-footer w-embed"><a href="https://www.linkedin.com/company/convaitech/"> <svg width="42" height="43" viewBox="0 0 42 43" fill="none" xmlns="http://www.w3.org/2000/svg"> <rect y="0.5" width="42" height="42" rx="21" fill="white"/> <path fill-rule="evenodd" clip-rule="evenodd" d="M20.3139 20.2377V20.2041C20.3071 20.2154 20.2976 20.2266 20.2916 20.2377H20.3139Z" fill="black"/> <path fill-rule="evenodd" clip-rule="evenodd" d="M16.8427 29.2434H13.3429V18.7138H16.8427V29.2434ZM15.0928 17.2766H15.0697C13.8956 17.2766 13.1346 16.4678 13.1346 15.4568C13.1346 14.4236 13.9179 13.6377 15.1157 13.6377C16.3138 13.6377 17.0501 14.4236 17.073 15.4568C17.073 16.4678 16.3138 17.2766 15.0928 17.2766ZM29.466 29.2434H25.9652V23.61C25.9652 22.1951 25.4594 21.2293 24.1923 21.2293C23.2247 21.2293 22.6497 21.8796 22.3963 22.5095C22.3043 22.7345 22.2802 23.0473 22.2802 23.3628V29.2435H18.7787C18.7787 29.2435 18.8256 19.7017 18.7787 18.7138H22.2802V20.2059C22.7454 19.4901 23.5756 18.4666 25.4353 18.4666C27.7392 18.4666 29.466 19.9712 29.466 23.2052V29.2434Z" fill="black"/> </svg> </a></div><div class="c-social-embed cc-footer w-embed"><a href="https://www.youtube.com/@convai"> <svg width="42" height="42" viewBox="0 0 42 42" fill="none" xmlns="http://www.w3.org/2000/svg"> <rect width="42" height="42" rx="21" fill="white"/> <path d="M26.6772 14H15.1094C12.8399 14 11 15.8397 11 18.109V23.891C11 26.1603 12.8399 28 15.1094 28H26.6772C28.9468 28 30.7867 26.1603 30.7867 23.891V18.109C30.7867 15.8397 28.9468 14 26.6772 14ZM23.898 21.2813L18.4874 23.8616C18.3432 23.9303 18.1767 23.8252 18.1767 23.6655V18.3437C18.1767 18.1818 18.3476 18.0768 18.4921 18.15L23.9027 20.8916C24.0636 20.973 24.0608 21.2037 23.898 21.2813Z" fill="black"/> </svg> </a></div><div class="c-social-embed cc-footer w-embed"><a href="https://discord.gg/aHUYXX3gW"> <svg width="42" height="42" viewBox="0 0 42 42" fill="none" xmlns="http://www.w3.org/2000/svg"> <rect width="42" height="42" rx="21" fill="white"/> <g clip-path="url(#clip0_3492_510)"> <path d="M28.6361 14.34C27.1907 13.705 25.6648 13.2545 24.0974 13C23.8829 13.3679 23.6888 13.7465 23.5159 14.1341C21.8463 13.8927 20.1484 13.8927 18.4788 14.1341C18.3059 13.7465 18.1118 13.368 17.8974 13C16.329 13.2566 14.8021 13.7082 13.3552 14.3433C10.4828 18.4213 9.70409 22.3981 10.0934 26.3184C11.7756 27.511 13.6584 28.418 15.66 29C16.1108 28.4183 16.5096 27.8012 16.8523 27.1553C16.2014 26.922 15.5731 26.6342 14.9748 26.2951C15.1323 26.1856 15.2863 26.0726 15.4351 25.9631C17.176 26.7487 19.0762 27.156 21 27.156C22.9238 27.156 24.8239 26.7487 26.5649 25.9631C26.7154 26.0809 26.8694 26.1939 27.0252 26.2951C26.4257 26.6347 25.7963 26.9231 25.1442 27.1569C25.4865 27.8026 25.8854 28.4192 26.3365 29C28.3398 28.4204 30.2241 27.5138 31.9065 26.32C32.3634 21.7738 31.1261 17.8335 28.6361 14.34ZM17.3454 23.9074C16.2605 23.9074 15.3641 22.9626 15.3641 21.8003C15.3641 20.638 16.2293 19.6849 17.342 19.6849C18.4546 19.6849 19.344 20.638 19.325 21.8003C19.3059 22.9626 18.4511 23.9074 17.3454 23.9074ZM24.6545 23.9074C23.5679 23.9074 22.675 22.9626 22.675 21.8003C22.675 20.638 23.5402 19.6849 24.6545 19.6849C25.7689 19.6849 26.6514 20.638 26.6324 21.8003C26.6133 22.9626 25.7602 23.9074 24.6545 23.9074Z" fill="black"/> </g> <defs> <clipPath id="clip0_3492_510"> <rect width="22" height="16" fill="white" transform="translate(10 13)"/> </clipPath> </defs> </svg> </a></div><div class="c-social-embed cc-footer w-embed"><a href="https://www.reddit.com/r/Convai/"> <svg width="42" height="42" viewBox="0 0 42 42" xmlns="http://www.w3.org/2000/svg"> <rect width="42" height="42" rx="21" fill="white"/> <path transform="translate(8.75, 8.35)" d="M24 11.779c0-1.459-1.192-2.645-2.657-2.645-.715 0-1.363.286-1.84.746-1.81-1.191-4.259-1.949-6.971-2.046l1.483-4.669 4.016.941-.006.058c0 1.193.975 2.163 2.174 2.163 1.198 0 2.172-.97 2.172-2.163s-.975-2.164-2.172-2.164c-.92 0-1.704.574-2.021 1.379l-4.329-1.015c-.189-.046-.381.063-.44.249l-1.654 5.207c-2.838.034-5.409.798-7.3 2.025-.474-.438-1.103-.712-1.799-.712-1.465 0-2.656 1.187-2.656 2.646 0 .97.533 1.811 1.317 2.271-.052.282-.086.567-.086.857 0 3.911 4.808 7.093 10.719 7.093s10.72-3.182 10.72-7.093c0-.274-.029-.544-.075-.81.832-.447 1.405-1.312 1.405-2.318zm-17.224 1.816c0-.868.71-1.575 1.582-1.575.872 0 1.581.707 1.581 1.575s-.709 1.574-1.581 1.574-1.582-.706-1.582-1.574zm9.061 4.669c-.797.793-2.048 1.179-3.824 1.179l-.013-.003-.013.003c-1.777 0-3.028-.386-3.824-1.179-.145-.144-.145-.379 0-.523.145-.145.381-.145.526 0 .65.647 1.729.961 3.298.961l.013.003.013-.003c1.569 0 2.648-.315 3.298-.962.145-.145.381-.144.526 0 .145.145.145.379 0 .524zm-.189-3.095c-.872 0-1.581-.706-1.581-1.574 0-.868.709-1.575 1.581-1.575s1.581.707 1.581 1.575-.709 1.574-1.581 1.574z"/> </svg> </a></div><div class="c-social-embed cc-footer w-embed"><a href="https://twitter.com/convaitech"> <svg width="42" height="43" viewBox="0 0 42 43" fill="none" xmlns="http://www.w3.org/2000/svg"> <rect y="0.5" width="42" height="42" rx="21" fill="white"/> <path d="M30.8288 15.4023C30.0926 15.7286 29.3117 15.9427 28.5121 16.0373C29.3546 15.5335 29.9852 14.7405 30.2862 13.8061C29.4946 14.2764 28.6282 14.6075 27.7248 14.7851C27.1712 14.1957 26.4533 13.7863 25.6642 13.6099C24.8752 13.4335 24.0513 13.4982 23.2995 13.7957C22.5476 14.0932 21.9025 14.6097 21.4477 15.2782C20.993 15.9468 20.7496 16.7365 20.7491 17.5451C20.7485 17.8543 20.7831 18.1626 20.8523 18.4641C19.2489 18.3836 17.6804 17.9668 16.2485 17.2409C14.8167 16.515 13.5534 15.4961 12.5408 14.2504C12.0251 15.1383 11.867 16.1894 12.099 17.1897C12.3309 18.19 12.9353 19.0643 13.7891 19.6347C13.1482 19.6151 12.5213 19.4422 11.9609 19.1305V19.1816C11.9611 20.1129 12.2834 21.0154 12.8731 21.7361C13.4629 22.4569 14.2838 22.9514 15.1966 23.1358C14.8503 23.23 14.493 23.2773 14.1341 23.2764C13.8792 23.2765 13.6248 23.2528 13.3743 23.2056C13.6319 24.0068 14.1334 24.7076 14.8087 25.2098C15.484 25.7121 16.2994 25.9907 17.1408 26.0069C15.4497 27.3305 13.303 27.9303 11.1707 27.6749C13.0146 28.8603 15.161 29.4895 17.3531 29.4874C18.8626 29.4976 20.3591 29.2077 21.7557 28.6347C23.1523 28.0617 24.4211 27.217 25.4885 26.1495C26.5558 25.0821 27.4005 23.8132 27.9734 22.4166C28.5462 21.0199 28.8359 19.5234 28.8256 18.0139C28.8256 17.8389 28.8214 17.6653 28.8128 17.493C29.6036 16.9213 30.2863 16.2134 30.8288 15.4023Z" fill="black"/> </svg> </a></div><div class="c-social-embed cc-footer w-embed"><a href="https://www.instagram.com/conv.ai/"> <svg width="42" height="42" viewBox="0 0 42 42" fill="none" xmlns="http://www.w3.org/2000/svg"> <rect width="42" height="42" rx="21" fill="white"/> <g clip-path="url(#clip0_3492_514)"> <path d="M21 12.3926C23.8035 12.3926 24.136 12.4031 25.2438 12.4539C28.0893 12.5834 29.4184 13.9335 29.5479 16.758C29.5986 17.8649 29.6082 18.1974 29.6082 21.0009C29.6082 23.8053 29.5977 24.1369 29.5479 25.2438C29.4175 28.0656 28.0919 29.4184 25.2438 29.5479C24.136 29.5986 23.8053 29.6091 21 29.6091C18.1965 29.6091 17.864 29.5986 16.7571 29.5479C13.9046 29.4175 12.5825 28.0613 12.453 25.2429C12.4023 24.136 12.3917 23.8044 12.3917 21C12.3917 18.1965 12.4031 17.8649 12.453 16.7571C12.5834 13.9335 13.909 12.5825 16.7571 12.453C17.8649 12.4031 18.1965 12.3926 21 12.3926ZM21 10.5C18.1484 10.5 17.7914 10.5122 16.6714 10.563C12.8581 10.738 10.7389 12.8538 10.5639 16.6705C10.5122 17.7914 10.5 18.1484 10.5 21C10.5 23.8516 10.5122 24.2095 10.563 25.3295C10.738 29.1427 12.8538 31.262 16.6705 31.437C17.7914 31.4877 18.1484 31.5 21 31.5C23.8516 31.5 24.2095 31.4877 25.3295 31.437C29.1393 31.262 31.2638 29.1462 31.4361 25.3295C31.4878 24.2095 31.5 23.8516 31.5 21C31.5 18.1484 31.4877 17.7914 31.437 16.6714C31.2655 12.8616 29.1471 10.7389 25.3304 10.5639C24.2095 10.5122 23.8516 10.5 21 10.5ZM21 15.6082C18.0224 15.6082 15.6082 18.0224 15.6082 21C15.6082 23.9776 18.0224 26.3926 21 26.3926C23.9776 26.3926 26.3918 23.9785 26.3918 21C26.3918 18.0224 23.9776 15.6082 21 15.6082ZM21 24.5C19.0671 24.5 17.5 22.9338 17.5 21C17.5 19.0671 19.0671 17.5 21 17.5C22.9329 17.5 24.5 19.0671 24.5 21C24.5 22.9338 22.9329 24.5 21 24.5ZM26.6053 14.1356C25.9088 14.1356 25.3444 14.7 25.3444 15.3956C25.3444 16.0912 25.9088 16.6556 26.6053 16.6556C27.3009 16.6556 27.8644 16.0912 27.8644 15.3956C27.8644 14.7 27.3009 14.1356 26.6053 14.1356Z" fill="black"/> </g> <defs> <clipPath id="clip0_3492_514"> <rect width="21" height="21" fill="white" transform="translate(10.5 10.5)"/> </clipPath> </defs> </svg> </a></div><div class="c-social-embed cc-footer w-embed"><a href="https://www.facebook.com/ConvaiOfficial"> <svg width="42" height="42" viewBox="0 0 42 42" xmlns="http://www.w3.org/2000/svg"> <rect width="42" height="42" rx="21" fill="white"/> <path transform="translate(8.95, 8.55)" d="M9 8h-3v4h3v12h5v-12h3.642l.358-4h-4v-1.667c0-.955.192-1.333 1.115-1.333h2.885v-5h-3.808c-3.596 0-5.192 1.583-5.192 4.615v3.385z"/> </svg> </a></div><div class="c-social-embed cc-footer w-embed"><a href="https://www.tiktok.com/@conv.ai"> <svg width="42" height="42" viewBox="0 0 42 42" xmlns="http://www.w3.org/2000/svg"> <rect width="42" height="42" rx="21" fill="white"/> <g transform="translate(9.75, 7.5) scale(0.00850000,0.00800000)" fill="#000000" stroke="none"> <path xmlns="http://www.w3.org/2000/svg" d="M2081 0c55 473 319 755 778 785v532c-266 26-499-61-770-225v995c0 1264-1378 1659-1932 753-356-583-138-1606 1004-1647v561c-87 14-180 36-265 65-254 86-398 247-358 531 77 544 1075 705 992-358V1h551z"/> </g> </svg> </a></div><div class="c-social-embed cc-footer w-embed"><a href="https://www.artstation.com/convai"> <svg width="42" height="42" viewBox="0 0 42 42" xmlns="http://www.w3.org/2000/svg"> <rect width="42" height="42" rx="21" fill="white"/> <g transform="translate(8.95, 32.5) scale(0.0100000,-0.0100000)" fill="#000000" stroke="none"> <path d="M1021 2378 c37 -70 1264 -2185 1274 -2196 10 -11 35 27 129 189 128 220 148 273 130 351 -12 53 -871 1548 -917 1596 -18 19 -50 43 -72 55 -38 21 -56 22 -298 25 l-258 3 12 -23z"/> <path d="M776 2008 c-28 -45 -549 -943 -553 -955 -4 -10 114 -13 572 -13 317 0 575 4 573 8 -7 19 -570 982 -574 982 -2 0 -10 -10 -18 -22z"/> <path d="M117 473 c121 -209 154 -252 218 -286 40 -22 43 -22 794 -25 607 -2 752 0 748 10 -3 7 -68 121 -145 253 l-140 240 -795 3 -796 2 116 -197z"/> </g> </svg> </a></div></div></div><div class="c-footer_cols-wrapper"><div class="c-container"><div class="c-footer_cols"><div class="c-footer_col"><img src="https://cdn.prod.website-files.com/63ea75dcbd94093302f1d430/63ea89fe0477067e4b125124_logo-white.svg" loading="lazy" alt="Convai logo" class="c-footer_logo"/><div class="c-text-3">© 2023 - Convai Technologies Inc.</div></div><div class="c-footer_col"><div class="c-footer_heading"><div class="c-title-7">Applications</div></div><div class="c-footer_links"><a href="#" class="c-footer_link">Use Cases</a><a href="#" class="c-footer_link">Features</a><a href="#" class="c-footer_link">Product Demo</a><a href="#" class="c-footer_link">Integrations</a></div></div><div class="c-footer_col cc-large"><div class="c-footer_heading"><div class="c-title-7">product</div></div><div class="c-footer_links"><a href="https://convai.com/pipeline/convai-character" class="c-footer_link">Convai Playground</a><a href="https://convai.com/pipeline/asr" class="c-footer_link">Speech To Text</a><a href="https://convai.com/pipeline/tts" class="c-footer_link">Text to Speech</a></div></div><div class="c-footer_col"><div class="c-footer_heading"><div class="c-title-7">Explore</div></div><div class="c-footer_links"><a href="https://docs.convai.com/" class="c-footer_link">Documentation</a><a href="https://convai.com/blog" class="c-footer_link">Blog</a><a href="https://convai.com/faqs" target="_blank" class="c-footer_link">FAQ’s</a><a href="https://convai.com/pricing" class="c-footer_link">Pricing</a><a href="https://convai.com/videos" target="_blank" class="c-footer_link">Videos</a></div></div><div class="c-footer_col cc-last"><div class="c-footer_heading"><div class="c-title-7">Company</div></div><div class="c-footer_links"><a href="https://convaitech.notion.site/Careers-at-Convai-678463719200459bbb1415fd9dea9db5" class="c-footer_link">Career</a><a href="#" class="c-footer_link">Support Center</a><a href="https://convai.com/privacy-policy" target="_blank" class="c-footer_link">Privacy Policy</a><a href="https://convai.com/tos" target="_blank" class="c-footer_link">Terms & Conditions</a></div></div></div></div></div></div></footer><script src="https://d3e54v103j8qbb.cloudfront.net/js/jquery-3.5.1.min.dc5e7f18c8.js?site=63ea75dcbd94093302f1d430" type="text/javascript" integrity="sha256-9/aliU8dGd2tb6OSsuzixeV4y/faTqgFtohetphbbj0=" crossorigin="anonymous"></script><script src="https://cdn.prod.website-files.com/63ea75dcbd94093302f1d430/js/webflow.95eba7a43.js" type="text/javascript"></script> <noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-KF8NNT39" height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript> <style> b strong em { background-color: blue; color: white; font-style: normal; font-weight: 700; padding: 5px 10px; border-radius: 5px; } table, th, td { border: 1px solid black; border-collapse: collapse; } td { padding: 10px; } </style></body></html>