CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8" /> <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" /> <title>Deep Dive into Terrajet, Part II</title> <meta name="HandheldFriendly" content="True" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <link rel="stylesheet" type="text/css" href="/assets/css/style.css?v=a1348d70b3" /> <link rel="icon" href="/favicon.png" type="image/png" /> <link rel="canonical" href="https://blog.crossplane.io/deep-dive-terrajet-part-ii/" /> <meta name="referrer" content="no-referrer-when-downgrade" /> <link rel="amphtml" href="https://blog.crossplane.io/deep-dive-terrajet-part-ii/amp/" /> <meta property="og:site_name" content="The Crossplane Blog" /> <meta property="og:type" content="article" /> <meta property="og:title" content="Deep Dive into Terrajet, Part II" /> <meta property="og:description" content="In the second part of our Deep Dive into Terrajet, we'll discuss the next step of extending the Kubernetes API with custom resources is to build a controller which reconciles its CRDs which we will focus on in this part." /> <meta property="og:url" content="https://blog.crossplane.io/deep-dive-terrajet-part-ii/" /> <meta property="og:image" content="https://blog.crossplane.io/content/images/2022/02/flo.png" /> <meta property="article:published_time" content="2022-02-01T18:26:07.000Z" /> <meta property="article:modified_time" content="2022-02-08T00:15:10.000Z" /> <meta property="article:tag" content="Crossplane" /> <meta property="article:tag" content="Providers" /> <meta property="article:tag" content="Terraform" /> <meta property="article:tag" content="Cloud Native" /> <meta property="article:tag" content="Kubernetes" /> <meta name="twitter:card" content="summary_large_image" /> <meta name="twitter:title" content="Deep Dive into Terrajet, Part II" /> <meta name="twitter:description" content="In the second part of our Deep Dive into Terrajet, we'll discuss the next step of extending the Kubernetes API with custom resources is to build a controller which reconciles its CRDs which we will focus on in this part." /> <meta name="twitter:url" content="https://blog.crossplane.io/deep-dive-terrajet-part-ii/" /> <meta name="twitter:image" content="https://blog.crossplane.io/content/images/2022/02/flo.png" /> <meta name="twitter:label1" content="Written by" /> <meta name="twitter:data1" content="Hasan Türken" /> <meta name="twitter:label2" content="Filed under" /> <meta name="twitter:data2" content="Crossplane, Providers, Terraform, Cloud Native, Kubernetes" /> <meta name="twitter:site" content="@crossplane_io" /> <meta name="twitter:creator" content="@turkenh" /> <meta property="og:image:width" content="2000" /> <meta property="og:image:height" content="720" /> <script type="application/ld+json"> { "@context": "https://schema.org", "@type": "Article", "publisher": { "@type": "Organization", "name": "The Crossplane Blog", "url": "https://blog.crossplane.io/", "logo": { "@type": "ImageObject", "url": "https://blog.crossplane.io/content/images/2020/05/CrossplaneLogo_Horiz-WhiteText.png" } }, "author": { "@type": "Person", "name": "Hasan Türken", "image": { "@type": "ImageObject", "url": "https://blog.crossplane.io/content/images/2022/01/photo-c.jpeg", "width": 1269, "height": 1327 }, "url": "https://blog.crossplane.io/author/hasan/", "sameAs": [ "https://twitter.com/turkenh" ] }, "headline": "Deep Dive into Terrajet, Part II", "url": "https://blog.crossplane.io/deep-dive-terrajet-part-ii/", "datePublished": "2022-02-01T18:26:07.000Z", "dateModified": "2022-02-08T00:15:10.000Z", "image": { "@type": "ImageObject", "url": "https://blog.crossplane.io/content/images/2022/02/flo.png", "width": 2000, "height": 720 }, "keywords": "Crossplane, Providers, Terraform, Cloud Native, Kubernetes", "description": "In the second part of our Deep Dive into Terrajet, we'll discuss the next step of extending the Kubernetes API with custom resources is to build a controller which reconciles its CRDs which we will focus on in this part.", "mainEntityOfPage": { "@type": "WebPage", "@id": "https://blog.crossplane.io/" } } </script> <meta name="generator" content="Ghost 4.2" /> <link rel="alternate" type="application/rss+xml" title="The Crossplane Blog" href="https://blog.crossplane.io/rss/" /> <script defer src="https://unpkg.com/@tryghost/portal@~1.1.0/umd/portal.min.js" data-ghost="https://blog.crossplane.io/"></script><style> .gh-post-upgrade-cta-content, .gh-post-upgrade-cta { display: flex; flex-direction: column; align-items: center; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Open Sans', 'Helvetica Neue', sans-serif; text-align: center; width: 100%; color: #ffffff; font-size: 16px; } .gh-post-upgrade-cta-content { border-radius: 8px; padding: 40px 4vw; } .gh-post-upgrade-cta h2 { color: #ffffff; font-size: 28px; letter-spacing: -0.2px; margin: 0; padding: 0; } .gh-post-upgrade-cta p { margin: 20px 0 0; padding: 0; } .gh-post-upgrade-cta small { font-size: 16px; letter-spacing: -0.2px; } .gh-post-upgrade-cta a { color: #ffffff; cursor: pointer; font-weight: 500; box-shadow: none; text-decoration: underline; } .gh-post-upgrade-cta a:hover { color: #ffffff; opacity: 0.8; box-shadow: none; text-decoration: underline; } .gh-post-upgrade-cta a.gh-btn { display: block; background: #ffffff; text-decoration: none; margin: 28px 0 0; padding: 8px 18px; border-radius: 4px; font-size: 16px; font-weight: 600; } .gh-post-upgrade-cta a.gh-btn:hover { opacity: 0.92; }</style> <style> .site-logo { max-width: 8em; } </style>  <script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start': new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0], j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src= 'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f); })(window,document,'script','dataLayer','GTM-WFF2NQHG');</script> <style>:root {--ghost-accent-color: #F3807B;}</style> </head> <body class="post-template tag-crossplane tag-providers tag-terraform tag-cloud-native tag-kubernetes"> <nav id="menu"> <a class="close-button">Close</a> <div class="nav-wrapper"> <p class="nav-label">Menu</p> <ul> <li class="nav-blog-home" role="presentation"><a href="https://blog.crossplane.io/">Blog Home</a></li> <li class="nav-crossplane-io" role="presentation"><a href="https://crossplane.io/">Crossplane.io</a></li> <li class="nav-subscribe-to-the-newsletter" role="presentation"><a href="https://eepurl.com/ivy4v-/">Subscribe to the Newsletter</a></li> <li class="nav-twitter"><a href="https://twitter.com/crossplane_io" title="@crossplane_io"><i class="ic ic-twitter"></i> Twitter</a></li> <li class="nav-rss"><a href="https://blog.crossplane.io/rss/"><i class="ic ic-rss"></i> Subscribe</a></li> </ul> </div> </nav> <section id="wrapper"> <a class="hidden-close"></a> <div class="progress-container"> <span class="progress-bar"></span> </div>  <header id="post-header"> <div class="inner"> <nav id="navigation"> <span class="blog-logo"> <a href="https://blog.crossplane.io"><img src="https://blog.crossplane.io/content/images/2020/05/CrossplaneLogo_Horiz-WhiteText.png" alt="Blog Logo" /></a> </span> <span id="menu-button" class="nav-button"> <a class="menu-button"><i class="ic ic-menu"></i> Menu</a> </span> </nav> <h1 class="post-title">Deep Dive into Terrajet, Part II</h1> <span class="post-meta"><a href="/author/hasan/">Hasan Türken</a> | <time datetime="2022-02-01">01 Feb 2022</time></span>  </div> </header> <main class="content" role="main"> <article class="post tag-crossplane tag-providers tag-terraform tag-cloud-native tag-kubernetes"> <div class="inner"> <section class="post-content"> <p>This is the second part of our <em>Deep Dive into Terrajet</em> series. In <a href="https://blog.crossplane.io/deep-dive-terrajet-part-i/">the first one</a>, we have discussed how we are generating Custom Resource Definitions (CRDs) for resources of an existing Terraform provider. The next step of extending the Kubernetes API with custom resources is building a controller that reconciles CRDs, which we will focus on in this part. We will talk about how we have built a generic controller for all <a href="https://github.com/crossplane/terrajet">Terrajet</a> based resources on top of <a href="https://pkg.go.dev/github.com/crossplane/crossplane-runtime@v0.15.1/pkg/reconciler/managed#Reconciler.Reconcile">Crossplane’s managed reconciler</a> which codifies all the best practices built by the community so far. We will also discuss how we have handled differences due to running Terraform as part of an active reconciliation instead of running it as a CLI tool.</p><h2 id="one-controller-for-all">One Controller for All</h2><p>Kubernetes docs define <a href="https://kubernetes.io/docs/concepts/architecture/controller/">controllers</a> as follows:</p><blockquote>In Kubernetes, controllers are control loops that watch the state of your cluster, then make or request changes where needed. Each controller tries to move the current cluster state closer to the desired state.</blockquote><p>Applying this to our controller, it will need to watch the current state of our external resource and try to bring it to the desired state. Here comes the tricky part, how should a controller that <a href="https://crossplane.io/docs/v1.6/concepts/managed-resources.html#continuous-reconciliation">keeps running</a> can bring the external resource into the desired state with Terraform CLI? Should we simply call <code>terraform apply</code> periodically and let it do the rest? Yeah, that could be a good POC :) However, for something real, we wouldn’t want a controller that keeps calling long-running <code>terraform apply</code>’s on our infrastructure resource no matter it is already up to date or not. Following the Kubernetes controller pattern, we want our controller to watch the state of our resource before taking any action. In other words, observe the current state and try to move it to the desired state <em>only if they do not match</em>.</p><p>Terraform provides a unified interface to interact with all resources; all define a set of arguments (inputs) and attributes (outputs), some arguments are optional, some are required, some are marked as sensitive, etc. Once we configure our resource(s) in a <code>.tf</code> file, we need to run <code>terraform plan</code> to see the configuration drift and run <code>terraform apply</code> to create/update our resource with desired specifications no matter which resource we’re interacting with. This is a great opportunity to build a common controller for all resources and one of the reasons behind <a href="https://github.com/crossplane/crossplane/blob/master/design/design-doc-terrajet.md#proposal">our design choice of interacting with CLI</a> instead of importing provider code or talking to provider servers via gRPC.</p><p>Before diving into more details on how this common controller works, let’s do a quick recap on Crossplane’s managed reconciler.</p><h3 id="crossplane-managed-reconciler">Crossplane Managed Reconciler</h3><p>Crossplane aims to manage cloud resources (indeed any resource external to Kubernetes) behind a single, well-defined and consistent API. In other words, similar to how Kubernetes makes distributed workload management easier with an opinionated resource schema (e.g. apiVersion, Kind, spec, status, etc.), Crossplane aims to bring infrastructure management under control and unlock further innovations leveraging a similar consistency. </p><p>The Crossplane team did an excellent job on <a href="https://github.com/crossplane/crossplane/blob/master/design/one-pager-managed-resource-api-design.md">defining this API</a> and building all the best practices to interact with this API as a <a href="https://github.com/crossplane/crossplane-runtime">reusable piece of code</a> so that no one needs to reinvent the wheel. Here comes <a href="https://github.com/crossplane/crossplane-runtime/blob/428b7c3903756bb0dcf5330f40298e1fa0c34301/pkg/reconciler/managed/reconciler.go#L626">the managed reconciler</a>; the reconciler powering every managed resource controller in Crossplane.</p><p>To use this reconciler and leverage all the previous knowledge built by the community so far, we need the following two:</p><ol><li>Our CRD should satisfy the <a href="https://pkg.go.dev/github.com/crossplane/crossplane-runtime@v0.15.1/pkg/resource#Managed"><code>Managed</code> interface</a>.</li><li>We need to implement an <code><a href="https://pkg.go.dev/github.com/crossplane/crossplane-runtime@v0.15.1/pkg/reconciler/managed#ExternalConnecter">ExternalConnector</a></code> that builds a client interacting with our external (cloud) API satisfying the <a href="https://pkg.go.dev/github.com/crossplane/crossplane-runtime@v0.15.1/pkg/reconciler/managed#ExternalClient"><code>ExternalClient</code> interface</a>.</li></ol><p>The <code>Managed</code> interface ensures our Custom Resource has a consistent API with other resources in the Crossplane ecosystem, in other words, to be compliant with <a href="https://crossplane.io/docs/v1.6/concepts/terminology.html#crossplane-resource-model">the Crossplane Resource Model (XRM)</a>. In the first part, we have already discussed how we have generated our CRDs accordingly using the resource schema in Terraform provider, you can find more details there.</p><h3 id="terraform-as-an-external-client">Terraform as an External Client</h3><p>For Terrajet based providers, Terraform CLI is the client that we use to interact with external APIs. This means we will need to implement an <code>ExternalClient</code> that uses Terraform CLI under the hood. Once we build the 4 methods, namely <code>Observe</code>, <code>Create</code>, <code>Update</code> and <code>Delete</code>, we will have a generic controller for all Terrajet based resources which also leverages all the best practices that the Crossplane community has built so far.</p><p>However, there is one important point here. Although we use the Terraform CLI to interact with any external API, there are differences in the <a href="https://www.terraform.io/language/providers/configuration">configuration</a> required to connect to different cloud providers like credentials and keys. Terrajet expects each provider to implement a specific setup function that returns a <code>Setup</code> object which will then be used in the <code>Connect</code> method that builds our <code>ExternalClient</code>. This <code>Setup</code> object is built by translating the configuration from Crossplane’s <a href="https://crossplane.io/docs/v1.6/concepts/providers.html#configuring-providers"><code>ProviderConfig</code> CR</a> to Terraform’s <a href="https://www.terraform.io/language/providers/configuration">Provider Configuration</a>.</p><figure class="kg-card kg-image-card kg-card-hascaption"><img src="https://lh6.googleusercontent.com/ZCMyNj6eF13-_7hZj_jRSMJfqJLDc0bFStzm8yJFmS5W2b0BK_GoTH3cGBuMWtLHSMQNSTku_FJ8rhkyMdubQID5mt3UqXF8zmDzmmT-qnZk0jHr3HdVtGTnd5TyIY266XA-kf9i" class="kg-image" alt loading="lazy"><figcaption>Terrajet Provider Setup Function and Related Types</figcaption></figure><p><code>Connect</code> method is one of the first methods called in the managed reconciler to build and return an external client for a given managed resource. In Terrajet, we have a generic <code><a href="https://pkg.go.dev/github.com/crossplane/terrajet@v0.4.0-rc.0.0.20220124152600-67fddf8893a7/pkg/controller#Connector">Connector</a></code> type that builds the external client using the aforementioned <code>Setup</code> object in its <code>Connect</code> method.</p><figure class="kg-card kg-image-card kg-card-hascaption"><img src="https://lh4.googleusercontent.com/lwZ6UCrp4tBL1Zjqoq20nKoxKuQSAlAOzVslBNQvYCKnUyp0FdaXajJvDQhfS2FbPz-ecYDgUB-W07zS1JGo8ZOMjibD4ONv1Zk05Aa4pSyqUl4hFGYwo5TTRpccRHjKQYT3zWd_" class="kg-image" alt loading="lazy"><figcaption>Terrajet Connector Connect Method</figcaption></figure><p>At this step, a temporary directory on the pod file system is prepared as a dedicated Terraform workspace for the CR instance being reconciled. We will not go into further details now on how we are building and managing those workspaces since we will dive into that in the next part of this series. All we need to know for now is, after this step, we have everything set up and configured for our Terraform CLI to talk to the external API.</p><h3 id="crud-my-external-resource-implementing-the-4-methods">CRUD My External Resource: Implementing the 4 Methods</h3><p>Having configured the Terraform CLI against an external API, let’s use it in our controller. </p><figure class="kg-card kg-image-card kg-card-hascaption"><img src="https://lh3.googleusercontent.com/bhMQKFGel8XyxI18mSiWxrXw1sak-DVUDI50ZY7K2DtlKdCghc-ovP69XxW_7oZxD82DtyF01mXz6NP37Zu2wWj3HoAmM_xvUSbcFYZZDjNzYPpdCdTCLAtk3cKknhg91x7Nq3cr" class="kg-image" alt loading="lazy"><figcaption>ExternalClient Interface of the Crossplane Managed Reconciler </figcaption></figure><p>Let’s start with the easy ones. <code>Create</code> and <code>Update</code> implementations are almost the same, calling <code>terraform apply</code> behind the scenes with the desired state. The only difference is, with <code>Create</code> we also want to capture and store any sensitive data that could be returned by the external API only once during creation. AWS secret access keys are a good example of such a resource. <code>Delete</code>, on the other hand, simply calls <code>terraform destroy</code> as you might have guessed.</p><p>Both terraform <code>apply</code> and <code>destroy</code> calls are blocking which could be problematic for long-running calls. Imagine you want to create a database that takes half an hour and you get no feedback (in CR status or as a K8s event) until it either succeeds or fails because that call blocks the whole reconciliation. To deal with that, we have implemented async flavors of those calls which could be configured for such resources, i.e. resources with long timeouts (e.g. > 5 mins) according to the Terraform documentation.<br>Before moving to the more challenging <code>Observe</code> method, there is one caveat with <code>Update</code> that is worth mentioning. There are some changes in Terraform that could be <a href="https://learn.hashicorp.com/tutorials/terraform/google-cloud-platform-change?in=terraform/gcp-get-started#introduce-destructive-changes">destructive</a>, meaning an update call could cause existing infrastructure to be destroyed and a new one with the desired spec to be created. This is usually fine with a CLI tool, which also provides ways to warn the user or ask for confirmation before proceeding. With a control plane approach that Crossplane offers, this does not make sense and an external resource corresponding to a CR should never be deleted unless there is an explicit delete call. Hence, Terrajet always runs <code>terraform apply</code> calls with <a href="https://learn.hashicorp.com/tutorials/terraform/resource-lifecycle?in=terraform/state#prevent-resource-deletion">prevent_destroy</a> lifecycle hook. This means, introducing a destructive change to a managed resource will simply fail with some error indicating that change is not possible without destroying the external resource.</p><p>Observing the External Resource</p><p>Ok, now let’s move to <code>Observe</code> method, which should <a href="https://pkg.go.dev/github.com/crossplane/crossplane-runtime@v0.15.1/pkg/reconciler/managed#ExternalObservation">return</a> answers to the following questions:</p><ul><li>Does the resource exist?</li><li>Is the resource up to date?</li><li>What are the <a href="https://crossplane.io/docs/v1.6/concepts/managed-resources.html#connection-details">connection details</a>?</li><li>Is the resource <a href="https://crossplane.io/docs/v1.6/concepts/managed-resources.html#late-initialization">late initialized</a>?</li></ul><p>To answer these, we will need to watch the current state of the resource and compare it with the desired state. But wait, where is the desired state?</p><figure class="kg-card kg-image-card kg-card-hascaption"><img src="https://blog.crossplane.io/content/images/2022/01/image-9.png" class="kg-image" alt loading="lazy" width="1972" height="726" srcset="https://blog.crossplane.io/content/images/size/w600/2022/01/image-9.png 600w, https://blog.crossplane.io/content/images/size/w1000/2022/01/image-9.png 1000w, https://blog.crossplane.io/content/images/size/w1600/2022/01/image-9.png 1600w, https://blog.crossplane.io/content/images/2022/01/image-9.png 1972w" sizes="(min-width: 720px) 720px"><figcaption>State Handling in Terraform</figcaption></figure><p>Terraform keeps the last applied state in a local <code>.tfstate</code> file and does a refresh to update it with the current state prior to any operation. According to terraform documentation, this state file is used to:</p><ul><li>Map real-world resources to your configuration</li><li>Keep track of metadata</li><li>Improve performance for large infrastructures</li></ul><p>In Crossplane, there is always a one-to-one mapping between a CR and an external resource. This means that there will always be one resource in our <code>.tfstate</code> hence we can ignore the last use case. For mapping real word resources, Crossplane has the External Name concept. For tracking metadata, we can use annotations or labels of our CRs. So, we can indeed get rid of the management of a <code>.tfstate</code> file by translating pieces into Crossplane/Kubernetes realm.</p><p>Once we can uniquely identify an external resource using its External Name, we can just observe its current state by “<a href="https://www.terraform.io/cli/commands/refresh">refreshing</a>”. The desired state is already available in our CR, hence we can compare and figure out whether the resource is up to date or not by invoking a <a href="https://www.terraform.io/cli/commands/plan">plan</a> command. More details on the implementation of this coming in the next blog post. All we need to know for now is, there is no <code>.tfstate</code> file persisted somewhere, rather, we are just (re)building it using the information in our CR whenever it is needed.</p><p>OK, but, What About Sensitive Information?</p><p>There is also some sensitive information that needs to be handled properly, we can not just write them to <code>spec</code> or <code>status</code> of the managed resource. Terraform state files <a href="https://github.com/hashicorp/terraform/issues/516">could contain sensitive data</a> like private keys or there could be some input configuration like an initial password of a database.</p><p>As you might have guessed, Kubernetes has secrets exactly for this purpose and Crossplane has the concept of <a href="https://crossplane.io/docs/v1.6/concepts/managed-resources.html#connection-details">Connection Details</a>, which are special K8s secrets holding the sensitive information required to connect to external resources. Sensitive fields are already <a href="https://github.com/hashicorp/terraform-plugin-sdk/blob/3819ed23c0a6aced81f77354a7b366a0234d477f/helper/schema/schema.go#L240">marked</a> in Terraform resource schema, so we already know which fields are sensitive. We just need to connect the dots and solve this problem:</p><ul><li>All sensitive input fields converted to Secret references</li><li>All sensitive output fields (the ones coming from <code>tfstate</code> files) are only written into the Connection Details secret.</li></ul><figure class="kg-card kg-image-card kg-card-hascaption"><img src="https://lh4.googleusercontent.com/P-r_ROHcZ7LCJIOWOoVu4yUDqpL0saqfc0ijYKt9CZ-_wInGOfTKM9LfCtIU85Md5QXouamACjBM2SWCVVxk62h5AXOKmfukNsQqqm5NfCJCIoEuYjb73Seo7Wtr17pB2SBgHFC3" class="kg-image" alt loading="lazy"><figcaption>Handling Sensitive Information</figcaption></figure><p>This would also require us to collect the sensitive information distributed to secrets while building our desired state, so, we have <a href="https://github.com/crossplane/terrajet/blob/v0.3.2/pkg/resource/sensitive.go">implemented</a> a two-way conversion from CR + Secrets to Terraform configuration and vice versa.</p><p>All output configuration that is marked as sensitive by Terraform will land in the connection secret. However, there are some cases where it would be convenient to have more information in a connection details secret like host and port which were not marked as sensitive (and could arguably be considered as sensitive as well) in schema or, we may want to store them in a well-known format like a <code>kubeconfig</code>. Thanks to Terrajet’s flexible and powerful <a href="https://github.com/crossplane/terrajet/blob/main/docs/configuring-a-resource.md#configuring-a-resource">configuration mechanism</a>, all of these are possible by providing an <code><a href="https://github.com/crossplane/terrajet/blob/main/docs/configuring-a-resource.md#additional-sensitive-fields-and-custom-connection-details">AdditionalConnectionDetailsFn</a></code>. Please check <a href="https://github.com/crossplane-contrib/provider-jet-gcp/blob/8c752236fefc423ad377feb24f8f5c6f77e7f7e1/config/container/config.go#L39">this</a> and <a href="https://github.com/crossplane-contrib/provider-jet-azure/blob/0c2bbdf95aad3dd80ce9299eaf48dd68b9bd777c/config/sql/config.go#L31">this</a> as examples.</p><p>Single Source of Truth</p><p>Crossplane managed resources are authoritative on the external resources that they are managing. In other words, etcd is the single source of truth for the Crossplane managed resources. Hence, they need to have all the configurations for the resources including the optional ones that are initialized with some server-side defaults. Crossplane follows <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#late-initialization">Kubernetes conventions</a> to handle this type of configuration as well and <a href="https://crossplane.io/docs/v1.6/concepts/managed-resources.html#late-initialization">late initializes</a> them after resources are created with initially provided configuration. Similarly, terrajet runtime automatically performs <a href="https://crossplane.io/docs/v1.6/concepts/managed-resources.html#late-initialization">late initialization</a> using <code>runtime.reflection</code> for jet-based resources.</p><p>After the state is refreshed with the current state of the external resource, <a href="https://pkg.go.dev/github.com/crossplane/terrajet@v0.4.0-rc.0.0.20220131134536-ae78a0a4c438/pkg/resource#Terraformed"><code>Terraformed</code></a> resource’s <a href="https://pkg.go.dev/github.com/crossplane/terrajet@v0.4.0-rc.0.0.20220131134536-ae78a0a4c438/pkg/resource#LateInitializer">LateInitialize</a> method is called with the fresh state attributes. LateInitialize in turn employs Terrajet’s <a href="https://pkg.go.dev/github.com/crossplane/terrajet@v0.4.0-rc.0.0.20220131134536-ae78a0a4c438/pkg/resource#GenericLateInitializer">resource.GenericLateInitializer</a> to perform late-initialization using the observed attributes from the provider. This library recursively sets previously unset attributes using the state data.</p><p>In most cases, Terrajet’s late initialization simply works with no custom configuration. However, there are some cases where <a href="https://github.com/crossplane/terrajet/blob/main/docs/configuring-a-resource.md#late-initialization-configuration">customization</a> is required in the late initialization behavior.</p><p>Finally, the Observe Method</p><p>Alright, having discussed all the important points, we can now talk about how we have brought them together and implemented the <code>Observe</code> method. It is expected to return an <code><a href="https://pkg.go.dev/github.com/crossplane/crossplane-runtime@v0.15.1/pkg/reconciler/managed#ExternalObservation">ExternalObservation</a></code> without making any modifications on the external resource. Below you can see a simplified version of the Observe flow, which completes all the required methods to run as a Crossplane Managed Reconciler. In the <a href="https://github.com/crossplane/terrajet/blob/844191d953b66f8ee591e33b13d5de388a95704c/pkg/controller/external.go#L114">actual implementation</a>, we have further optimizations like handling async calls, marking resources as ready as soon as possible and, avoiding unnecessary plans, which are not shown in the flow for brevity.</p><figure class="kg-card kg-image-card kg-card-hascaption"><img src="https://blog.crossplane.io/content/images/2022/01/image-8.png" class="kg-image" alt loading="lazy" width="2000" height="720" srcset="https://blog.crossplane.io/content/images/size/w600/2022/01/image-8.png 600w, https://blog.crossplane.io/content/images/size/w1000/2022/01/image-8.png 1000w, https://blog.crossplane.io/content/images/size/w1600/2022/01/image-8.png 1600w, https://blog.crossplane.io/content/images/size/w2400/2022/01/image-8.png 2400w" sizes="(min-width: 720px) 720px"><figcaption>Simplified Observe Flow with Terrajet</figcaption></figure><h2 id="what-is-next">What is next?</h2><p>In this second part of our 3-part Terrajet deep dive series, we focused on the generic controller part and discussed how we applied the controller pattern using the Terraform CLI while leveraging Crossplane’s managed reconciler. We also briefly talked about how we are dealing with the Terraform state and how we are translating it to the Crossplane world. <a href="https://blog.crossplane.io/deep-dive-into-terrajet-part-iii/">In the next and final part,</a> we will dive into more details about working with Terraform CLI, setting up temporary workspaces, and building state files using the information in etcd.</p><p>If you have more questions about Terrajet, or want to talk more generally about Crossplane, head to our <a href="https://slack.crossplane.io/">Slack channel</a> for feedback from community experts. </p> </section> <section class="post-info"> <div class="post-share"> <a class="twitter" href="https://twitter.com/share?text=Deep Dive into Terrajet, Part II&url=https://blog.crossplane.io/deep-dive-terrajet-part-ii/" onclick="window.open(this.href, 'twitter-share', 'width=550,height=235');return false;"> <i class="ic ic-twitter"></i><span class="hidden">Twitter</span> </a> <a class="facebook" href="https://www.facebook.com/sharer/sharer.php?u=https://blog.crossplane.io/deep-dive-terrajet-part-ii/" onclick="window.open(this.href, 'facebook-share','width=580,height=296');return false;"> <i class="ic ic-facebook"></i><span class="hidden">Facebook</span> </a> <a class="googleplus" href="https://plus.google.com/share?url=https://blog.crossplane.io/deep-dive-terrajet-part-ii/" onclick="window.open(this.href, 'google-plus-share', 'width=490,height=530');return false;"> <i class="ic ic-googleplus"></i><span class="hidden">Google+</span> </a> <div class="clear"></div> </div> <aside class="post-tags"> <a href="/tag/crossplane/">Crossplane</a> <a href="/tag/providers/">Providers</a> <a href="/tag/terraform/">Terraform</a> <a href="/tag/cloud-native/">Cloud Native</a> <a href="/tag/kubernetes/">Kubernetes</a> </aside> <div class="clear"></div> <aside class="post-author"> <figure class="post-author-avatar avatar"> <img src="https://blog.crossplane.io/content/images/2022/01/photo-c.jpeg" alt="Hasan Türken" /> </figure> <div class="post-author-bio"> <h4 class="post-author-name"><a href="/author/hasan/">Hasan Türken</a></h4> <span class="post-author-twitter"><i class="ic ic-twitter"></i> <a target="_blank" href="https://twitter.com/@turkenh">Twitter</a></span> </div> <div class="clear"></div> </aside> </section>   <link href="//cdn-images.mailchimp.com/embedcode/classic-10_7.css" rel="stylesheet" type="text/css"> <style type="text/css"> #mc_embed_signup{background:#fff; clear:left; font:14px Helvetica,Arial,sans-serif; width: 600px; align-content: center; } /* Add your own Mailchimp form style overrides in your site stylesheet or in this style block. We recommend moving this block and the preceding CSS link to the HEAD of your HTML file. */ </style> <div id="mc_embed_signup"> <form action="https://upbound.us17.list-manage.com/subscribe/post?u=b9f6c1840c97ee09ae739fdb0&id=4f555f7090" method="post" id="mc-embedded-subscribe-form" name="mc-embedded-subscribe-form" class="validate" target="_blank" novalidate> <div id="mc_embed_signup_scroll"> <h2>Keep up with Upbound</h2> <div class="indicates-required"><span class="asterisk">*</span> indicates required</div> <div class="mc-field-group"> <label for="mce-LNAME">Name <span class="asterisk">*</span> </label> <input type="text" value="" name="LNAME" class="required" id="mce-LNAME"> </div> <div class="mc-field-group"> <label for="mce-EMAIL">Email Address <span class="asterisk">*</span> </label> <input type="email" value="" name="EMAIL" class="required email" id="mce-EMAIL"> </div> <div id="mce-responses" class="clear"> <div class="response" id="mce-error-response" style="display:none"></div> <div class="response" id="mce-success-response" style="display:none"></div> </div>  <div style="position: absolute; left: -5000px;" aria-hidden="true"><input type="text" name="b_b9f6c1840c97ee09ae739fdb0_4f555f7090" tabindex="-1" value=""></div> <div class="clear"><input type="submit" value="Subscribe" name="subscribe" id="mc-embedded-subscribe" class="button"></div> </div> </form> </div> <script type='text/javascript' src='//s3.amazonaws.com/downloads.mailchimp.com/js/mc-validate.js'></script><script type='text/javascript'>(function($) {window.fnames = new Array(); window.ftypes = new Array();fnames[2]='LNAME';ftypes[2]='text';fnames[0]='EMAIL';ftypes[0]='email';}(jQuery));var $mcj = jQuery.noConflict(true);</script>  <aside class="post-nav"> <a class="post-nav-next" href="/deep-dive-into-terrajet-part-iii/"> <section class="post-nav-teaser"> <i class="ic ic-arrow-left"></i> <h2 class="post-nav-title">Deep Dive into Terrajet, Part III</h2> <p class="post-nav-excerpt">In this concluding post of the "Deep Dive into Terrajet" series, we will delve into the details of how we set up Terraform workspaces and how we interact with the Terraform CLI.…</p> </section> </a> <a class="post-nav-prev" href="/deep-dive-terrajet-part-i/"> <section class="post-nav-teaser"> <i class="ic ic-arrow-right"></i> <h2 class="post-nav-title">Deep Dive into Terrajet, Part I</h2> <p class="post-nav-excerpt">In Crossplane, the low level primitive for provisioning cloud infrastructure is called…</p> </section> </a> <div class="clear"></div> </aside> </div> </article> </main> <div id="body-class" style="display: none;" class="post-template tag-crossplane tag-providers tag-terraform tag-cloud-native tag-kubernetes"></div> <footer id="footer"> <div class="inner"> <section class="credits"> <span class="credits-theme">Theme <a href="https://github.com/zutrinken/attila">Attila</a> by <a href="http://zutrinken.com" rel="nofollow">zutrinken</a></span> <span class="credits-software">Published with <a href="http://ghost.org">Ghost</a></span> </section> </div> </footer> </section> <script type="text/javascript" src="/assets/js/script.js?v=a1348d70b3"></script>  <noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-WFF2NQHG" height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>  </body> </html>