<?xml version="1.0" encoding="utf-8"?> <rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" version="2.0" xmlns:media="http://search.yahoo.com/mrss/"><channel><title>Cloud Blog</title><link>https://cloud.google.com/blog/</link><description>Cloud Blog</description><atom:link href="https://cloudblog.withgoogle.com/rss/" rel="self"></atom:link><language>en</language><lastBuildDate>Wed, 11 Aug 2021 22:54:00 -0000</lastBuildDate><image><url>https://gweb-cloudblog-publish.appspot.com/static/blog/images/google.a51985becaa6.png</url><title>Cloud Blog</title><link>https://cloud.google.com/blog/</link></image><item><title>Extend your Dataflow template with UDFs</title><link>https://cloud.google.com/blog/topics/developers-practitioners/extend-your-dataflow-template-with-udfs/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Google provides a set of &lt;a href="https://cloud.google.com/dataflow/docs/guides/templates/provided-templates"&gt;Dataflow templates&lt;/a&gt; that customers commonly use for frequent data tasks, but also as reference data pipelines that developers can extend. But what if you want to customize a Dataflow template without modifying or maintaining the Dataflow template code itself? With user-defined functions (UDFs), customers can extend certain Dataflow templates with their custom logic to transform records on the fly.&lt;/p&gt;&lt;p&gt;A UDF is a JavaScript snippet that implements a simple element processing logic, and is provided as an input parameter to the Dataflow pipeline. This is especially helpful for users who want to customize the pipeline’s output format without having to re-compile or to maintain the template code itself. Example use cases include enriching records with additional fields, redacting some sensitive fields, or filtering out undesired records - we’ll dive into each of those. That means you do not have to be an Apache Beam developer or even have a developer environment setup in order to tweak the output of these Dataflow templates!&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;As the time of this writing, the following Google-provided Dataflow templates support UDF:&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://github.com/GoogleCloudPlatform/DataflowTemplates/blob/master/src/main/java/com/google/cloud/teleport/templates/PubSubToBigQuery.java" target="_blank"&gt;Pub/Sub to BigQuery&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://github.com/GoogleCloudPlatform/DataflowTemplates/blob/master/src/main/java/com/google/cloud/teleport/templates/PubsubToDatastore.java" target="_blank"&gt;Pub/Sub to Datastore&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/dataflow/docs/guides/templates/provided-streaming#pubsub-to-splunk"&gt;Pub/Sub to Splunk&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/dataflow/docs/guides/templates/provided-streaming#pubsub-to-mongodb"&gt;Pub/Sub to MongoDB&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/dataflow/docs/guides/templates/provided-batch#datastore-to-cloud-storage-text"&gt;Datastore to GCS Text&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://github.com/GoogleCloudPlatform/DataflowTemplates/blob/master/src/main/java/com/google/cloud/teleport/templates/DatastoreToPubsub.java" target="_blank"&gt;Datastore to Pub/Sub&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/dataflow/docs/guides/templates/provided-batch#cloud-spanner-to-cloud-storage-text"&gt;Cloud Spanner to Cloud Storage Text&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/dataflow/docs/guides/templates/provided-batch#cloud-storage-text-to-datastore"&gt;GCS Text to Datastore&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/dataflow/docs/guides/templates/provided-streaming#text-files-on-cloud-storage-to-bigquery-stream"&gt;GCS Text to BigQuery&lt;/a&gt; (&lt;a href="https://cloud.google.com/dataflow/docs/guides/templates/provided-batch#cloud-storage-text-to-bigquery"&gt;Batch&lt;/a&gt; and &lt;a href="https://cloud.google.com/dataflow/docs/guides/templates/provided-streaming#text-files-on-cloud-storage-to-bigquery-stream"&gt;Stream&lt;/a&gt;)&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/dataflow/docs/guides/templates/provided-streaming#apache-kafka-to-bigquery"&gt;Apache Kafka to BigQuery&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/dataflow/docs/guides/templates/provided-utilities#datastore-bulk-delete"&gt;Datastore Bulk Delete&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;b&gt;Note&lt;/b&gt;: While the UDF concepts described here apply to any &lt;a href="https://github.com/GoogleCloudPlatform/DataflowTemplates#template-pipelines" target="_blank"&gt;Dataflow template that supports UDF&lt;/a&gt;, the utility UDF samples below are from real-world use cases using the &lt;a href="https://cloud.google.com/dataflow/docs/guides/templates/provided-streaming#cloudpubsubtosplunk"&gt;Pub/Sub to Splunk Dataflow&lt;/a&gt; template, but you can re-use those as starting point for this or other Dataflow templates. &lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h2&gt;How UDF works with templates&lt;/h2&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;When a UDF is provided, the UDF JavaScript code runs in the &lt;a href="https://developer.oracle.com/databases/nashorn-javascript-part1.html" target="_blank"&gt;Nashorn JavaScript engine&lt;/a&gt; included in the Dataflow worker’s Java runtime (applicable for Java pipelines such as Google-provided Dataflow templates). The code is invoked locally by a Dataflow worker for each element separately. Element payloads are serialized and passed as JSON strings back and forth.&lt;/p&gt;&lt;p&gt;Here’s the format of a Dataflow UDF function called &lt;code&gt;process&lt;/code&gt; which you can reuse and insert your own custom transformation logic into:&lt;br/&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Using JavaScript’s standard built-in JSON object, the UDF first parses the stringified element &lt;code&gt;inJson&lt;/code&gt; into a variable &lt;code&gt;obj&lt;/code&gt;, and, at the end, it must return a stringified version &lt;code&gt;outJson&lt;/code&gt; of the modified element &lt;code&gt;obj&lt;/code&gt;. Where highlighted, you add your custom element transformation logic depending on your use case. In the next section, we provide you with utility UDF samples from real-world use cases. &lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Note&lt;/b&gt;: The variable &lt;code&gt;includePubsubMessage&lt;/code&gt; is required if the UDF is applied to Pub/Sub to Splunk Dataflow template since it supports two possible element formats: that specific template can be configured to process the full &lt;a href="https://cloud.google.com/pubsub/docs/reference/rest/v1/PubsubMessage"&gt;Pub/Sub message payload&lt;/a&gt; or only the underlying Pub/Sub message data payload (default behavior). The statement setting &lt;code&gt;data&lt;/code&gt; variable is needed to normalize the UDF input payload in order to simplify your subsequent transformation logic in the UDF, consistent with the examples below. For more context, see &lt;code&gt;includePubsubMessage&lt;/code&gt; parameter in &lt;a href="https://cloud.google.com/dataflow/docs/guides/templates/provided-streaming#cloudpubsubtosplunk"&gt;Pub/Sub to Splunk template documentation&lt;/a&gt;.&lt;/p&gt;&lt;h2&gt;Common UDF patterns&lt;/h2&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;The following code snippets are example transformation logic to be inserted in the above UDF &lt;code&gt;process&lt;/code&gt; function. They are grouped below by common patterns.&lt;/p&gt;&lt;h3&gt;Pattern 1: Enrich events&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--small h-c-grid__col h-c-grid__col--2 h-c-grid__col--offset-5 "&gt;&lt;img alt="UDF enrich events illustration" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/1-enrich-events_k4t3CtR.max-1000x1000.jpg"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Follow this pattern to enrich events with new fields for more contextual information.&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Example 1.1: &lt;p&gt;Add a new field as metadata to track pipeline’s input Pub/Sub subscription&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Example 1.2*: &lt;p&gt;Set Splunk HEC metadata source field to track pipeline’s input Pub/Sub subscription&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;ul&gt;&lt;li&gt;Example 1.3: &lt;p&gt;Add new fields based on a user-defined local function e.g. &lt;code&gt;callerToAppIdLookup()&lt;/code&gt; acting as a static mapping or lookup table&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Pattern 2: Transform events&lt;/h3&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--small h-c-grid__col h-c-grid__col--2 h-c-grid__col--offset-5 "&gt;&lt;img alt="UDF transform events illustration" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/2-transform-events_xYwWrYg.max-1000x1000.jpg"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Follow this pattern to transform the entire event format depending on what your destination expects.&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;ul&gt;&lt;li&gt;Example 2.1: &lt;p&gt;Revert logs from Cloud Logging log payload (&lt;a href="https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry"&gt;LogEntry&lt;/a&gt;) to original raw log string. You may use this pattern with VM application or system logs (e.g. syslog or Windows Event Logs) to send source raw logs (instead of JSON payloads):&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;ul&gt;&lt;li&gt;Example 2.2*: &lt;p&gt;Transform logs from Cloud Logging log payload (&lt;a href="https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry"&gt;LogEntry&lt;/a&gt;) to original raw log string by setting Splunk HEC event metadata. Use this pattern with application or VM logs (e.g. syslog or Windows Event Logs) to index original raw logs (instead of JSON payload) for compatibility with downstream analytics. This example also enriches logs by setting HEC fields metadata to incoming resource labels metadata:&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;&lt;/h3&gt;&lt;h3&gt;Pattern 3: Redact events&lt;/h3&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--small h-c-grid__col h-c-grid__col--2 h-c-grid__col--offset-5 "&gt;&lt;img alt="UDF redact events illustration" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/3-redact-events_FUxgh4E.max-1000x1000.jpg"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Follow this pattern to redact or remove a part of the event.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;ul&gt;&lt;li&gt;Example 3.1: &lt;p&gt;Delete or redact &lt;a href="https://cloud.google.com/bigquery/docs/reference/auditlogs#visibility_and_access_control"&gt;sensitive SQL query&lt;/a&gt; field from BigQuery &lt;a href="https://cloud.google.com/bigquery/docs/reference/auditlogs/rest/Shared.Types/AuditData"&gt;AuditData&lt;/a&gt; data access logs:&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Pattern 4: Route events&lt;br/&gt;&lt;/h3&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--small h-c-grid__col h-c-grid__col--2 h-c-grid__col--offset-5 "&gt;&lt;img alt="UDF route events illustration" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/4-route-events.max-1000x1000.jpg"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Follow this pattern to programmatically route events to separate destinations.&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Example 4.1*: &lt;p&gt;Route event to the correct Splunk index per used-defined local function e.g. &lt;code&gt;splunkIndexLookup()&lt;/code&gt; acting as a static mapping or lookup table:&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;ul&gt;&lt;li&gt;Example 4.2: &lt;p&gt;Route unrecognized or unsupported events to Pub/Sub deadletter topic (if configured) in order to avoid invalid data or unnecessary consumption of downstream sinks such as BigQuery or Splunk:&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Pattern 5: Filter events&lt;/h3&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--small h-c-grid__col h-c-grid__col--2 h-c-grid__col--offset-5 "&gt;&lt;img alt="UDF filter events illustration" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/5-filter-events.max-1000x1000.jpg"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Follow this pattern to filter out undesired or unrecognized events.&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Example 5.1: &lt;p&gt;Drop events from a particular resource type or log type, e.g. filter out verbose Dataflow operational logs such as worker &amp;amp; system logs:&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;ul&gt;&lt;li&gt;Example 5.2: &lt;p&gt;Drop events from a particular log type, e.g. Cloud Run application stdout:&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;* Example applicable to &lt;a href="https://cloud.google.com/dataflow/docs/guides/templates/provided-streaming#pubsub-to-splunk"&gt;Pub/Sub to Splunk Dataflow&lt;/a&gt; template only&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h2&gt;Testing UDFs&lt;/h2&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Besides ensuring functional correctness, you must verify your UDF code is syntactically correct JavaScript on Nashorn JavaScript engine which is shipped as part of JDK (8 through 14) pre-installed in Dataflow workers. That’s where your UDF ultimately runs. Before pipeline deployment, it is highly recommended to test your UDF on Nashorn engine: any JavaScript syntax error will throw an exception, potentially on every message. This will cause a pipeline outage as the UDF is unable to process those messages in-flight.&lt;/p&gt;&lt;p&gt;At the time of this writing, Google-provided Dataflow templates run on JDK 11 environment with the corresponding Nashorn engine v11 release. By default, Nashorn engine is only ECMAScript 5.1 (ES5) compliant so a lot of newer ES6 JavaScript keywords like &lt;code&gt;let&lt;/code&gt; or &lt;code&gt;const&lt;/code&gt; will cause syntax errors. In addition, it’s important to note that Oracle Nashorn engine is a slightly different JavaScript implementation than Node.js. A common pitfall is using &lt;code&gt;console.log()&lt;/code&gt; or &lt;code&gt;Number.isNaN()&lt;/code&gt; for example, neither of which are defined in the Nashorn engine. For more details, see this &lt;a href="https://www.oracle.com/technical-resources/articles/java/jf14-nashorn.html" target="_blank"&gt;introduction to using Oracle Nashorn&lt;/a&gt;. That said, using the utility UDFs provided above without major code changes should be sufficient for most use cases.&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;An easy way to test your UDF on Nashorn engine is by launching &lt;a href="https://console.cloud.google.com/home/dashboard?cloudshell=true"&gt;Cloud Shell&lt;/a&gt; where JDK 11 is pre-installed, including jjs command-line tool to invoke Nashorn engine.&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;Let’s assume your UDF is saved in &lt;code&gt;dataflow_udf_transform.js&lt;/code&gt; JavaScript file and that you’re using UDF example 1.1 above which appends new &lt;code&gt;inputSubscription&lt;/code&gt; field.&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;In Cloud Shell, you can launch Nashorn in interactive mode as follows:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;In Nashorn interactive shell, first load your UDF JavaScript file which will load the UDF ‘process’ function in global scope:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;To test your UDF, define an arbitrary input JSON object depending on your pipeline’s expected in-flight messages. In this example, we’re using a snippet of a Dataflow job log message to be processed by our pipeline:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;You can now invoke your UDF function to process that input object as follows:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Notice how the input object is serialized first before being passed to UDF which expects an input string as noted in the previous section.&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;Print the UDF output to view the transformed log with the appended &lt;code&gt;inputSubscription&lt;/code&gt; field as expected:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Finally exit the interactive shell:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h2&gt;Deploying UDFs&lt;/h2&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--medium h-c-grid__col h-c-grid__col--4 h-c-grid__col--offset-4 "&gt;&lt;img alt="Processing elements in Dataflow pipeline" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/dataflow-pipeline_TU8Or6y.max-1000x1000.jpg"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;You deploy a UDF when you run a Dataflow job by referencing a GCS file containing the UDF JavaScript file. Here’s an example using gcloud CLI to run a job using the Pub/Sub to Splunk Dataflow template:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;The relevant parameters to configure:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;&lt;code&gt;gcs-location&lt;/code&gt;: GCS location path to the Dataflow template&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;code&gt;javascriptTextTransformGcsPath&lt;/code&gt;: GCS location path to JavaScript file with your UDF code&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;code&gt;javascriptTextTransformFunctionName&lt;/code&gt;: Name of JavaScript function to call as your UDF&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;As a Dataflow user or operator, you simply reference a pre-existing template URL (Google-hosted), and your custom UDF (Customer-hosted) without the requirement to have a Beam developer environment setup or to maintain the template code itself.&lt;/p&gt;&lt;p&gt;&lt;b&gt;Note&lt;/b&gt;: The Dataflow worker service account used must have access to the GCS object (JavaScript file) containing your UDF function. Refer to &lt;a href="https://cloud.google.com/dataflow/docs/concepts/security-and-permissions#worker_service_account"&gt;Dataflow user docs&lt;/a&gt; to learn more about Dataflow worker service account.&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;h2&gt;What’s Next?&lt;/h2&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;We hope this helps you get started with customizing some of the off-the-shelf &lt;a href="https://cloud.google.com/dataflow/docs/guides/templates/provided-streaming"&gt;Google-provided Dataflow templates&lt;/a&gt; using one of the above utility UDFs or writing your own UDF function. As a technical artifact of your pipeline deployment, the UDF is a component of your infrastructure, and so we recommend you follow Infrastructure-as-Code (IaC) best practices including version-controlling your UDF. If you have questions or suggestions for other utility UDFs, we’d like to hear from you: &lt;a href="https://github.com/GoogleCloudPlatform/DataflowTemplates/issues" target="_blank"&gt;create an issue&lt;/a&gt; directly in GitHub repo, or ask away in our &lt;a href="https://stackoverflow.com/questions/tagged/google-cloud-dataflow" target="_blank"&gt;Stack Overflow forum&lt;/a&gt;.&lt;/p&gt;&lt;p&gt;In a follow-up blog post, we’ll dive deeper into testing UDFs (unit tests and end-to-end pipeline tests) as well as setting up a CI/CD pipeline (for your pipelines!) including triggering new deployment every time you update your UDFs - all without maintaining any Apache Beam code.&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Wed, 11 Aug 2021 22:54:00 -0000</pubDate><guid>https://cloud.google.com/blog/topics/developers-practitioners/extend-your-dataflow-template-with-udfs/</guid><category>Data Analytics</category><category>Developers &amp; Practitioners</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/original_images/dataflow-hero.gif" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>Extend your Dataflow template with UDFs</title><description>Learn how to easily extend a Cloud Dataflow template with user-defined functions (UDFs) to transform messages in-flight, without modifying or maintaining Apache Beam code</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/original_images/dataflow-hero.gif</image><site_name>Google</site_name><url>https://cloud.google.com/blog/topics/developers-practitioners/extend-your-dataflow-template-with-udfs/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Roy Arsan</name><title>Solutions Architect</title><department></department><company></company></author></item><item><title>What’s new with Google Cloud</title><link>https://cloud.google.com/blog/topics/inside-google-cloud/whats-new-google-cloud/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Want to know the latest from Google Cloud? Find it here in one handy location. Check back regularly for our newest updates, announcements, resources, events, learning opportunities, and more. &lt;br/&gt;&lt;/p&gt;&lt;hr/&gt;&lt;p&gt;&lt;b&gt;Tip&lt;/b&gt;: Not sure where to find what you’re looking for on the Google Cloud blog? Start here: &lt;a href="https://cloud.google.com/blog/topics/inside-google-cloud/complete-list-google-cloud-blog-links-2021"&gt;Google Cloud blog 101: Full list of topics, links, and resources&lt;/a&gt;.&lt;/p&gt;&lt;hr/&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Aug 9 -Aug 13&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Google Cloud and NVIDIA&lt;/b&gt; have teamed up to make VR/AR workloads easier, faster to create and tetherless! &lt;a href="https://cloud.google.com/blog/products/gaming/streaming-vr-and-ar-content-google-cloud-nvidia-cloudxr"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Register for the Google Cloud Startup Summit, September 9, 2021&lt;/b&gt; at &lt;a href="http://goo.gle/StartupSummit" target="_blank"&gt;goo.gle/StartupSummit&lt;/a&gt; for a digital event filled with inspiration, learning, and discussion. This event will bring together our startup and VC community to discuss the latest trends and insights, headlined by a keynote by Astro Teller, Captain of Moonshots at X the moonshot factory. Additionally, learn from a variety of technical and business sessions to help take your startup to the next level.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Google Cloud and Harris Poll healthcare research reveals COVID-19 impacts on healthcare technology&lt;/b&gt;. &lt;a href="https://cloud.google.com/blog/topics/healthcare-life-sciences/google-cloud-covid-19-research"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/gcp/google-cloud-a-leader-in-2021-gartner-iaas-mq/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/Google_Cloud_GCP.max-500x500.png')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Google named a Leader in 2021 Gartner Magic Quadrant for Cloud Infrastructure and Platform Services again&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Gartner named Google Cloud a Leader in the 2021 Magic Quadrant for Cloud Infrastructure and Platform Services, formerly Infrastructure as...&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Aug 2-Aug 6&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;&lt;br/&gt;&lt;/b&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Gartner named Google Cloud a Leader in the 2021 Magic Quadrant for Cloud Infrastructure and Platform Services, formerly Infrastructure as a Service. &lt;a href="https://cloud.google.com/blog/products/gcp/google-cloud-a-leader-in-2021-gartner-iaas-mq"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/b&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Private Service Connect is now generally available&lt;/b&gt;. Private Service Connect lets you create private and secure connections to Google Cloud and third-party services with service endpoints in your VPCs. &lt;a href="https://cloud.google.com/blog/products/networking/private-service-connect-is-now-generally-available"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;30 migration guides&lt;/b&gt; designed to help you identify the best ways to migrate, which include meeting common organizational goals like minimizing time and risk during your migration, identifying the most enterprise-grade infrastructure for your workloads, picking a cloud that aligns with your organization’s sustainability goals, and more. &lt;a href="https://cloud.google.com/blog/products/cloud-migration/30-guides-to-kickstart-your-migration-to-google-cloud"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/infrastructure/the-google-cloud-region-in-melbourne-is-now-open/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/google_cloud_melbourne_region.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;The new Google Cloud region in Melbourne is now open&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;The new Google Cloud region in Melbourne adds a second region to Australia, supporting economic growth in the region.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Jul 26-Jul 30&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;This week we hosting our &lt;a href="https://cloudonair.withgoogle.com/events/summit-retail" target="_blank"&gt;&lt;b&gt;Retail &amp;amp; Consumer Goods Summit&lt;/b&gt;&lt;/a&gt;, a digital event dedicated to helping leading retailers and brands digitally transform their business. &lt;a href="https://cloud.google.com/blog/topics/consumer-packaged-goods/retail-cpg-summit-2021-a-new-era-for-consumer-brands"&gt;Read more&lt;/a&gt; about our consumer packaged goods strategy and a guide to key summit content for brands in this blog from Giusy Buonfantino, Google Cloud’s Vice President of CPG.&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;We’re hosting our &lt;a href="https://cloudonair.withgoogle.com/events/summit-retail" target="_blank"&gt;&lt;b&gt;Retail &amp;amp; Consumer Goods Summit&lt;/b&gt;&lt;/a&gt;, a digital event dedicated to helping leading retailers and brands digitally transform their business. &lt;a href="https://cloud.google.com/blog/topics/retail/the-three-pillars-of-the-future-of-retail"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;See how &lt;b&gt;IKEA uses Recommendations AI&lt;/b&gt; to provide customers with more relevant product information. &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/ikea-uses-google-cloud-recommendations-ai"&gt;Read more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;​​Google Cloud launches a &lt;b&gt;career program for people with autism&lt;/b&gt; designed to hire and support more talented people with autism in the rapidly growing cloud industry. &lt;a href="https://cloud.google.com/blog/topics/inside-google-cloud/google-cloud-launches-a-career-program-for-people-with-autism"&gt;Learn more&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Google Cloud follows &lt;b&gt;new API stability tenets&lt;/b&gt; that work to minimize unexpected deprecations to our Enterprise APIs. &lt;a href="https://cloud.google.com/blog/topics/inside-google-cloud/new-api-stability-tenets-govern-google-enterprise-apis"&gt;Read more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/google-cloud-next/register-now-for-google-cloud-next21/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/CloudNext21.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Registration is open for Google Cloud Next: October 12–14&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Register now for Google Cloud Next on October 12–14, 2021&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Jul 19-Jul 23&lt;br/&gt;&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Register and join us for &lt;b&gt;Google Cloud Next, October 12-14, 2021&lt;/b&gt; at &lt;a href="http://g.co/CloudNext" target="_blank"&gt;g.co/CloudNext&lt;/a&gt; for a fresh approach to digital transformation, as well as a few surprises. &lt;b&gt;Next ‘21&lt;/b&gt; will be a fully customizable digital adventure for a more personalized learning journey. Find the tools and training you need to succeed. From live, interactive Q&amp;amp;As and informative breakout sessions to educational demos and real-life applications of the latest tech from Google Cloud. Get ready to plug into your cloud community, get informed, and be inspired. Together we can tackle today’s greatest business challenges, and start solving for what’s next.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;"Application Innovation" takes a front row seat this year&lt;/b&gt;– To stay ahead of rising customer expectations and the digital and in-person hybrid landscape, enterprises must know what application innovation means and how to deliver this type of innovation with a small piece of technology that might surprise you. &lt;a href="https://cloud.google.com/blog/products/api-management/accelerating-app-innovation-with-api-management"&gt;Learn more about the three pillars of app innovation here.&lt;/a&gt;&lt;br/&gt;&lt;/li&gt;&lt;li&gt;We &lt;a href="https://cloud.google.com/blog/products/identity-security/detect-complex-network-threats-with-cloud-ids"&gt;&lt;b&gt;announced&lt;/b&gt;&lt;/a&gt; Cloud IDS, our new network security offering, which is now available in preview. Cloud IDS delivers easy-to-use, cloud-native, managed, &lt;b&gt;network-based threat detection&lt;/b&gt;. With Cloud IDS, customers can enjoy a Google Cloud-integrated experience, built with Palo Alto Networks’ industry-leading threat detection technologies to provide high levels of security efficacy. &lt;a href="https://cloud.google.com/ids/"&gt;Learn more&lt;/a&gt;. &lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Key Visualizer for Cloud Spanner is now generally available&lt;/b&gt;. Key Visualizer is a new interactive monitoring tool that lets developers and administrators analyze usage patterns in Spanner. It reveals trends and outliers in key performance and resource metrics for databases of any size, helping to optimize queries and reduce infrastructure costs. &lt;a href="https://youtu.be/P3hiF3iNQew" target="_blank"&gt;See it in action&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;The market for &lt;b&gt;healthcare cloud is projected to grow 43%&lt;/b&gt;. This means a need for better tech infrastructure, digital transformation &amp;amp; Cloud tools. &lt;a href="https://cloud.google.com/blog/topics/healthcare-life-sciences/google-cloud-partner-advantage-for-healthcare"&gt;Learn how&lt;/a&gt; Google Cloud Partner Advantage partners help customers solve business challenges in healthcare.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/infrastructure/google-cloud-region-in-delhi-ncr-is-now-open/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/dehli_google_cloud.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;The new Google Cloud region in Delhi NCR is now open&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;The Google Cloud region in Delhi NCR is now open for business, ready to host your workloads.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Jul 12-Jul 16&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Simplify VM migrations with Migrate for Compute Engine as a Service&lt;/b&gt;: delivers a Google-managed cloud service that enables simple, frictionless, and large-scale enterprise migrations of virtual machines to Google Compute Engine with minimal downtime and risk. API-driven and integrated into your Google Cloud console for ease of use, this service uses agent-less replication to copy data without manual intervention and without VPN requirements. It also enables you to launch non-disruptive validations of your VMs prior to cutover.  Rapidly migrate a single application or execute a sprint with hundred systems using migration groups with confidence. Read more &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/migrate-for-compute-engine-as-a-service"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;The &lt;b&gt;Google Cloud region in Delhi NCR is now open&lt;/b&gt; for business, ready to host your workloads. &lt;a href="https://cloud.google.com/blog/products/infrastructure/google-cloud-region-in-delhi-ncr-is-now-open"&gt;Learn more&lt;/a&gt; and watch the region launch event &lt;a href="https://cloudonair.withgoogle.com/events/india-region-launch" target="_blank"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Introducing Quilkin: the open-source game server proxy&lt;/b&gt;. Developed in collaboration with Embark Studios, Quilkin is an open source UDP proxy, tailor-made for high performance real-time multiplayer games. &lt;a href="https://cloud.google.com/blog/products/gaming/introducing-quilkin"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;We’re making &lt;b&gt;Google Glass on Meet available to a wider network&lt;/b&gt; of global customers. &lt;a href="https://cloud.google.com/blog/products/workspace/open-beta-now-available-to-all-workspace-customers"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Transfer Appliance supports Google Managed Encryption Keys&lt;/b&gt; — We’re announcing the support for Google Managed Encryption Keys with Transfer Appliance, this is in addition to the currently available Customer Managed Encryption Keys feature. Customers have asked for the Transfer Appliance service to create and manage encryption keys for transfer sessions to improve usability and maintain security. The Transfer Appliance Service can now manage the encryption keys for the customers who do not wish to handle a key themselves. Learn more about &lt;a href="https://cloud.google.com/transfer-appliance/docs/4.0/permissions-step-by-step#prepare-keys"&gt;Using Google Managed Encryption Keys&lt;/a&gt;. &lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;UCLA builds a campus-wide API program&lt;/b&gt;– With Google Cloud's API management platform, Apigee, UCLA created a unified and strong API foundation that removes data friction that students, faculty, and administrators alike face. This foundation not only simplifies how various personas connect to data, but also encourages more innovations in the future. &lt;a href="https://cloud.google.com/blog/products/api-management/ucla-accelerates-innovation-with-google-cloud"&gt;Learn their story.&lt;/a&gt;&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;An enhanced region picker makes it easy to &lt;b&gt;choose a Google Cloud region with the &lt;/b&gt;&lt;b&gt;lowest CO2 &lt;/b&gt;&lt;b&gt;output&lt;/b&gt;. &lt;a href="https://cloud.google.com/blog/topics/sustainability/pick-the-google-cloud-region-with-the-lowest-co2"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;Amwell and Google Cloud explore &lt;b&gt;five ways telehealth can help democratize access to healthcare&lt;/b&gt;. &lt;a href="https://cloud.google.com/blog/topics/healthcare-life-sciences/how-telehealth-improves-access-to-healthcare"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;Major League Baseball and Kaggle &lt;b&gt;launch ML competition&lt;/b&gt; to learn about fan engagement. &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/using-machine-learning-to-deconstruct-baseball-fandom"&gt;Batter up!&lt;/a&gt;&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;We’re rolling out &lt;b&gt;general support&lt;/b&gt; of &lt;b&gt;Brand Indicators for Message Identification (BIMI)&lt;/b&gt; in Gmail within Google Workspace. &lt;a href="https://cloud.google.com/blog/products/identity-security/bringing-bimi-to-gmail-in-google-workspace"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/blog/products/data-analytics/dena-sports-business-helps-fight-covid-19-with-device-data-app"&gt;Learn how&lt;/a&gt; DeNA Sports Business created an operational status visualization system that helps determine whether live event attendees have correctly installed Japan’s coronavirus contact tracing app COCOA.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Google Cloud CAS&lt;/b&gt; provides a highly scalable and available private CA to address the unprecedented growth in certificates in the digital world. &lt;a href="https://cloud.google.com/blog/products/identity-security/google-cloud-certificate-authority-service-is-now-ga"&gt;Read more&lt;/a&gt; about CAS.&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/gaming/google-cloud-and-call-of-duty-league-power-competitive-statistics/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/gaming_JSnCYSg.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Closer to the action: Call of Duty League and Google Cloud deliver new feature for esports fans&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Google Cloud and Call of Duty League launch ActivStat to bring fans, players, and commentators the power of competitive statistics in rea...&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Jul 5-Jul 9 2021&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Google Cloud and Call of Duty League launch ActivStat&lt;/b&gt; to bring fans, players, and commentators the power of competitive statistics in real-time. &lt;a href="https://cloud.google.com/blog/products/gaming/google-cloud-and-call-of-duty-league-power-competitive-statistics"&gt;Read more.&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Building applications is a heavy lift&lt;/b&gt; due to the technical complexity, which includes the complexity of backend services that are used to manage and store data. Firestore alters this by having Google Cloud manage your backend complexity through a complete backend-as-a-service! &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/all-you-need-know-about-firestore-cheatsheet"&gt;Learn more.&lt;/a&gt;&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Google Cloud’s new Native App Development&lt;/b&gt; skills challenge lets you earn badges that demonstrate your ability to create cloud-native apps. &lt;a href="https://cloud.google.com/blog/topics/training-certifications/introducing-new-native-app-development-skills-challenge"&gt;Read more&lt;/a&gt; and &lt;a href="https://inthecloud.withgoogle.com/google-cloud-skills/register.html?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=FY21-Q1-global-trainingandenablement-website-other-skills_challenge&amp;amp;utm_content=nativeappdev" target="_blank"&gt;sign up&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/workspace/att-android-customers-to-have-messages-app-by-default/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/messages.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;AT&amp;amp;T Android customers to have Messages app by default&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Messages by Google is now the default messaging app for all AT&amp;amp;T customers using Android phones in the United States.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Jun 28-Jul 2 2021&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Storage Transfer Service&lt;/b&gt; now offers preview support for Integration with AWS Security Token Service. Security conscious customers can now use Storage Transfer Service to perform transfers from AWS S3 without passing any security credentials. This release will alleviate the security burden associated with passing long-term AWS S3 credentials, which have to be rotated or explicitly revoked when they are no longer needed. &lt;a href="https://cloud.google.com/storage-transfer/docs/configure-access#federated-identity"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;The most popular and surging &lt;b&gt;Google Search terms are now available in BigQuery&lt;/b&gt; as a public dataset. View the Top 25 and Top 25 rising queries from Google Trends from the past 30-days, including 5 years of historical data across the 210 Designated Market Areas (DMAs) in the US. &lt;a href="https://cloud.google.com/blog/products/data-analytics/top-25-google-search-terms-now-in-bigquery"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;A &lt;b&gt;new predictive autoscaling capability lets you add additional Compute Engine VMs in anticipation of forecasted demand&lt;/b&gt;. Predictive autoscaling is generally available across all Google Cloud regions. &lt;a href="https://cloud.google.com/blog/products/compute/introducing-compute-engine-predictive-autoscaling"&gt;Read more&lt;/a&gt; or consult the &lt;a href="https://cloud.google.com/compute/docs/autoscaler/predictive-autoscaling"&gt;documentation&lt;/a&gt; for more information on how to configure, simulate and monitor predictive autoscaling.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Messages by Google&lt;/b&gt; is now the default messaging app for all AT&amp;amp;T customers using Android phones in the United States. &lt;a href="https://cloud.google.com/blog/products/workspace/att-android-customers-to-have-messages-app-by-default"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;TPU v4 Pods will soon be available on Google Cloud&lt;/b&gt;, providing the most powerful publicly available computing platform for machine learning training. &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/google-wins-mlperf-benchmarks-with-tpu-v4"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;Cloud SQL for SQL Server has addressed multiple enterprise customer asks with the &lt;b&gt;GA releases of both SQL Server 2019 and Active Directory integration&lt;/b&gt;, as well as the Preview release of Cross Region Replicas.  This set of releases work in concert to allow customers to set up a more scalable and secure managed SQL Server environment to address their workloads’ needs. &lt;a href="https://cloud.google.com/blog/products/databases/improving-cloud-sql-for-sql-server-security-and-scalability"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/identity-security/how-hbo-uses-recaptcha-enterprise-to-secure-signup/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/reCAPTCHA.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;How HBO Max uses reCAPTCHA Enterprise to make its customer experience frictionless&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Balancing product, marketing, customer and security needs without slowing down signups.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Jun 21-Jun 25 2021&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Simplified return-to-office with no-code technology&lt;/b&gt;–&lt;b&gt;&lt;/b&gt;We've just released a solution to your most common return-to-office headaches: make a no-code app customized to solve your business-specific challenges. Learn how to create an automated app where employees can see office room occupancy, check what desks are reserved or open, review disinfection schedules, and more in this &lt;a href="https://cloud.google.com/blog/products/no-code-development/build-no-code-apps-with-appsheet-for-hybrid-work-and-more"&gt;blog tutorial.&lt;/a&gt;&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;New technical validation whitepaper for running ecommerce applications&lt;/b&gt;—Enterprise Strategy Group's analyst outlines the challenges of organizations running ecommerce applications and how Google Cloud helps to mitigate those challenges and handle changing demands with global infrastructure solutions. &lt;a href="https://cloud.google.com/resources/esg-technical-ecommerce-whitepaper"&gt;Download the whitepaper.&lt;/a&gt;&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;The full&lt;/b&gt;&lt;a href="https://developersonair.withgoogle.com/events/game-dev-summit-2021" target="_blank"&gt;&lt;b&gt;agenda&lt;/b&gt;&lt;/a&gt;&lt;b&gt;for Google for Games Developer Summit on July 12th-13th, 2021 is now available&lt;/b&gt;. A free digital event with announcements from teams including Stadia, Google Ads, AdMob, Android, Google Play, Firebase, Chrome, YouTube, and Google Cloud. Hear more about how Google Cloud technology creates opportunities for gaming companies to make lasting enhancements for players and creatives. Register at &lt;a href="https://developersonair.withgoogle.com/events/game-dev-summit-2021" target="_blank"&gt;g.co/gamedevsummit&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://cloud.google.com/blog/products/data-analytics/bigquery-provides-tighter-controls-over-data-access"&gt;&lt;b&gt;BigQuery row-level security&lt;/b&gt;&lt;/a&gt; is now generally available, giving customers a way to control access to subsets of data in the same table for different groups of users. Row-level security (RLS) extends the &lt;a href="https://en.wikipedia.org/wiki/Principle_of_least_privilege" target="_blank"&gt;principle of least privilege&lt;/a&gt; access and enables fine-grained access control policies in BigQuery tables. BigQuery currently supports access controls at the &lt;a href="https://cloud.google.com/bigquery/docs/access-control"&gt;project-&lt;/a&gt;, &lt;a href="https://cloud.google.com/bigquery/docs/dataset-access-controls"&gt;dataset-&lt;/a&gt;, &lt;a href="https://cloud.google.com/bigquery/docs/table-access-controls-intro"&gt;table-&lt;/a&gt; and &lt;a href="https://cloud.google.com/bigquery/docs/column-level-security"&gt;column-level&lt;/a&gt;. Adding RLS to the portfolio of access controls now enables customers to filter and define access to specific rows in a table based on qualifying user conditions—providing much needed peace of mind for data professionals.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Transfer from Azure ADLS Gen 2&lt;/b&gt;: &lt;a href="https://cloud.google.com/storage-transfer-service"&gt;Storage Transfer Service&lt;/a&gt; offers Preview support for transferring data from Azure ADLS Gen 2 to Google Cloud Storage. Take advantage of a scalable, serverless service to handle data transfer. &lt;a href="https://cloud.google.com/storage-transfer/docs/release-notes#June_18_2021"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;reCAPTCHA V2 and V3 customers can now migrate site keys to reCAPTCHA Enterprise&lt;/b&gt; in under 10 minutes and without making any code changes. Watch our &lt;a href="https://www.brighttalk.com/webcast/18282/490567?utm_source=GoogleCloudSecurity&amp;amp;utm_medium=brighttalk&amp;amp;utm_campaign=490567" target="_blank"&gt;Webinar to learn more. &lt;/a&gt;&lt;br/&gt;&lt;/li&gt;&lt;li&gt;Bot attacks are the biggest threat to your business that you probably haven’t addressed yet. &lt;b&gt;Check out our&lt;/b&gt; &lt;a href="https://www.forbes.com/sites/googlecloud/2021/04/01/bot-attacks-are-the-biggest-online-risk-you-havent-addressed/?sh=4cec49d46dda" target="_blank"&gt;&lt;b&gt;Forbes article&lt;/b&gt;&lt;/a&gt; to see what you can do about it.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/compute/google-cloud-introduces-tau-vms/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/Tau_VM.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;New Tau VMs deliver leading price-performance for scale-out workloads&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Compute Engine’s new Tau VMs based on AMD EPYC processors provide leading price/performance for scale-out workloads on an x86-based archi...&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Jun 14-Jun 18 2021&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;A new VM family for scale-out workloads&lt;/b&gt;—New AMD-based Tau VMs offer 56% higher absolute performance and 42% higher price-performance compared to general-purpose VMs from any of the leading public cloud vendors. &lt;a href="https://cloud.google.com/blog/products/compute/google-cloud-introduces-tau-vms"&gt;Learn more&lt;/a&gt;. &lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;New whitepaper helps customers plot their cloud migrations&lt;/b&gt;—Our new whitepaper distills the conversations we’ve had with CIOs, CTOs, and their technical staff into several frameworks that can help cut through the hype and the technical complexity to help devise the strategy that empowers both the business and IT. &lt;a href="https://cloud.google.com/blog/products/cloud-migration/new-cloud-migration-whitepaper-asks-up-or-out"&gt;Read more&lt;/a&gt; or &lt;a href="https://cloud.google.com/resources/understanding-cloud-migration-frameworks-whitepaper"&gt;download the whitepaper&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Ubuntu Pro lands on Google Cloud&lt;/b&gt;—The general availability of Ubuntu Pro images on Google Cloud gives customers an improved Ubuntu experience, expanded security coverage, and integration with critical Google Cloud features. &lt;a href="https://cloud.google.com/blog/products/compute/ubuntu-pro-available-on-google-cloud"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Navigating hybrid work with a single, connected experience in Google Workspace&lt;/b&gt;—New additions to Google Workspace help businesses navigate the challenges of hybrid work, such as Companion Mode for Google Meet calls. &lt;a href="https://cloud.google.com/blog/products/workspace/helping-business-with-new-additions-to-google-workspace"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Arab Bank embraces Google Cloud technology&lt;/b&gt;—This Middle Eastern bank now offers innovative apps and services to their customers and employees with Apigee and Anthos. In fact, Arab Bank reports over 90% of their new-to-bank customers are using their mobile apps. &lt;a href="https://cloud.google.com/blog/products/apigee/arab-bank-accelerates-app-innovation-with-google-cloud"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Google Workspace for the Public Sector Sector events&lt;/b&gt;—This June, learn about Google Workspace tips and tricks to help you get things done. Join us for one or more of our learning events tailored for government and higher education users. &lt;a href="https://cloud.google.com/blog/topics/public-sector/join-us-google-workspace-public-sector-sector-events"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/infrastructure/learn-about-googles-subsea-cables/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/world_map1_54SFPZM.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;All about cables: A guide to posts on our infrastructure under the sea&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;All our posts on Google’s global subsea cable system in one handy location.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Jun 7-Jun 11 2021&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;The top cloud capabilities industry leaders want for sustained innovation&lt;/b&gt;—Multicloud and hybrid cloud approaches, coupled with open-source technology adoption, enable IT teams to take full advantage of the best cloud has to offer. Our recent study with IDG shows just how much of a priority this has become for business leaders. &lt;a href="https://cloud.google.com/blog/topics/hybrid-cloud/multicloud-and-hybrid-cloud-are-must-haves-from-cloud-providers"&gt;Read more&lt;/a&gt; or &lt;a href="https://cloud.google.com/resources/5-ways-google-hybrid-multicloud-success-whitepaper?skip_cache=true&amp;amp;hl=en"&gt;download the report&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Announcing the Firmina subsea cable&lt;/b&gt;—Planned to run from the East Coast of the United States to Las Toninas, Argentina, with additional landings in Praia Grande, Brazil, and Punta del Este, Uruguay, Firmina will be the longest open subsea cable in the world capable of running entirely from a single power source at one end of the cable if its other power source(s) become temporarily unavailable—a resilience boost at a time when reliable connectivity is more important than ever. &lt;a href="https://cloud.google.com/blog/products/infrastructure/announcing-the-firmina-subsea-cable"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;New research reveals what’s needed for AI acceleration in manufacturing&lt;/b&gt;—According to our data, which polled more than 1,000 senior manufacturing executives across seven countries, 76% have turned to digital enablers and disruptive technologies due to the pandemic such as data and analytics, cloud, and artificial intelligence (AI). And 66% of manufacturers who use AI in their day-to-day operations report that their reliance on AI is increasing. &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/research-on-ai-trends-in-manufacturing"&gt;Read more&lt;/a&gt; or &lt;a href="https://services.google.com/fh/files/blogs/google_cloud_manufacturing_report_2021.pdf" target="_blank"&gt;download the report&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Cloud SQL offers even faster maintenance&lt;/b&gt;—Cloud SQL maintenance is zippier than ever. MySQL and PostgreSQL planned maintenance typically lasts less than 60 seconds and SQL Server maintenance typically lasts less than 120 seconds. You can learn more about maintenance &lt;a href="https://cloud.google.com/sql/docs/mysql/maintenance"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Simplifying Transfer Appliance configuration with Cloud Setup Application&lt;/b&gt;—We’re announcing the availability of the &lt;a href="https://cloud.google.com/transfer-appliance/docs/4.0/permissions-application"&gt;Transfer Appliance Cloud Setup Application&lt;/a&gt;. This will use the information you provide through simple prompts and configure your Google Cloud permissions, preferred Cloud Storage bucket, and Cloud KMS key for your transfer. Several cloud console based manual steps are now simplified with a command line experience. &lt;a href="https://cloud.google.com/transfer-appliance/docs/4.0/permissions-application"&gt;Read more&lt;/a&gt;. &lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Google Cloud VMware Engine is now HIPAA compliant&lt;/b&gt;—As of April 1, 2021, Google Cloud VMware Engine is covered under the Google Cloud Business Associate Agreement (BAA), meaning it has achieved HIPAA compliance. Healthcare organizations can now migrate and run their HIPAA-compliant VMware workloads in a fully compatible VMware Cloud Verified stack running natively in Google Cloud with Google Cloud VMware Engine, without changes or re-architecture to tools, processes, or applications. &lt;a href="https://cloud.google.com/blog/products/cloud-migration/google-cloud-vmware-engine-has-achieved-hipaa-compliance"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Introducing container-native Cloud DNS&lt;/b&gt;—Kubernetes networking almost always starts with a DNS request. DNS has broad impacts on your application and cluster performance, scalability, and resilience. That is why we are excited to announce the release of &lt;a href="https://cloud.google.com/kubernetes-engine/docs/how-to/cloud-dns"&gt;container-native Cloud DNS&lt;/a&gt;—the native integration of Cloud DNS with Google Kubernetes Engine (GKE) to provide in-cluster Service DNS resolution with &lt;a href="https://cloud.google.com/dns"&gt;Cloud DNS&lt;/a&gt;, our scalable and full-featured DNS service. &lt;a href="https://cloud.google.com/blog/products/containers-kubernetes/integrating-cloud-dns-with-gke"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Welcoming the EU’s new Standard Contractual Clauses for cross-border data transfers&lt;/b&gt;—Learn how we’re incorporating the new Standard Contractual Clauses (SCCs) into our contracts to help protect our customers’ data and meet the requirements of European privacy legislation. &lt;a href="https://cloud.google.com/blog/products/compliance/eus-new-standard-contractual-clauses-to-protect-user-privacy"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Lowe’s meets customer demand with Google SRE practices&lt;/b&gt;—Learn how Low’s has been able to increase the number of releases they can support by adopting Google’s &lt;a href="https://sre.google/" target="_blank"&gt;Site Reliability Engineering&lt;/a&gt; (SRE) framework and leveraging their partnership with Google Cloud. &lt;a href="https://cloud.google.com/blog/products/devops-sre/how-lowes-leverages-google-sre-practices"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;What’s next for SAP on Google Cloud at SAPPHIRE NOW and beyond&lt;/b&gt;—As &lt;a href="https://reg.sapevents.sap.com/flow/sap/sapphirenow2021/portal/page/home" target="_blank"&gt;SAP’s SAPPHIRE conference&lt;/a&gt; begins this week, we believe businesses have a more significant opportunity than ever to build for their next decade of growth and beyond. Learn more on how we’re working together with our customers, SAP, and our partners to support this transformation. &lt;a href="https://cloud.google.com/blog/products/sap-google-cloud/sap-on-google-cloud-updates-at-sapphire-now-2021"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Support for Node.js, Python and Java repositories for &lt;a href="https://cloud.google.com/artifact-registry"&gt;Artifact Registry&lt;/a&gt;&lt;/b&gt;&lt;b&gt;now in Preview&lt;/b&gt;–With today’s announcement, you can not only use Artifact Registry to secure and distribute container images, but also manage and secure your other software artifacts. &lt;a href="https://cloud.google.com/blog/products/application-development/artifact-registry-adds-node-python-and-java-repositories"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;What’s next for SAP on Google Cloud at SAPPHIRE NOW and beyond&lt;/b&gt;—As &lt;a href="https://reg.sapevents.sap.com/flow/sap/sapphirenow2021/portal/page/home" target="_blank"&gt;SAP’s SAPPHIRE conference&lt;/a&gt; begins this week, we believe businesses have a more significant opportunity than ever to build for their next decade of growth and beyond. Learn more on how we’re working together with our customers, SAP, and our partners to support this transformation. &lt;a href="https://cloud.google.com/blog/products/sap-google-cloud/sap-on-google-cloud-updates-at-sapphire-now-2021"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Google named a Leader in The Forrester Wave: Streaming Analytics, Q2 2021 report&lt;/b&gt;–Learn about the criteria where Google Dataflow was rated 5 out 5 and why this matters for our customers &lt;a href="https://cloud.google.com/blog/products/data-analytics/dataflow-leader-forrester-wave-streaming-analytics-report-2021"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Applied ML Summit this Thursday, June 10&lt;/b&gt;–Watch our keynote to learn about predictions for machine learning over the next decade. Engage with distinguished researchers, leading practitioners, and Kaggle Grandmasters during our live Ask Me Anything session. Take part in our modeling workshops to learn how you can iterate faster, and deploy and manage your models with confidence–no matter your level of formal computer science training. Learn how to develop and apply your professional skills, grow your abilities at the pace of innovation, and take your career to the next level. &lt;a href="https://cloudonair.withgoogle.com/events/summit-ml-practitioners?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=FY21-Q2-northam-GEM912-onlineevent-er-gc-ml-practitioners-summit&amp;amp;utm_content=google-cloud-blog-appliedml&amp;amp;utm_term=-" target="_blank"&gt;Register now&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/storage-data-transfer/a-peek-behind-colossus-googles-file-system/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/GCP_Storage_and_Data_Transfer.max-2800x2800.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Colossus under the hood: a peek into Google’s scalable storage system&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;An overview of Colossus, the file system that underpins Google Cloud’s storage offerings.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of May 31-Jun 4 2021&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Security Command Center now supports CIS 1.1 benchmarks and granular access control&lt;/b&gt;–&lt;a href="https://cloud.google.com/security-command-center"&gt;Security Command Center&lt;/a&gt; (SCC) now supports &lt;a href="https://www.cisecurity.org/benchmark/google_cloud_computing_platform/" target="_blank"&gt;CIS benchmarks for Google Cloud Platform Foundation v1.1&lt;/a&gt;, enabling you to monitor and address compliance violations against industry best practices in your Google Cloud environment. Additionally, SCC now supports fine-grained access control for administrators that allows you to easily adhere to the principles of least privilege—restricting access based on roles and responsibilities to reduce risk and enabling broader team engagement to address security. &lt;a href="https://cloud.google.com/blog/products/identity-security/new-capabilities-in-security-command-center-help-manage-risk"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Zero-trust managed security for services with Traffic Director&lt;/b&gt;–We created &lt;a href="https://cloud.google.com/traffic-director"&gt;Traffic Director&lt;/a&gt; to bring to you a fully managed service mesh product that includes load balancing, traffic management and service discovery. And now, we’re happy to announce the availability of a fully-managed zero-trust security solution using Traffic Director with Google Kubernetes Engine (GKE) and &lt;a href="https://cloud.google.com/certificate-authority-service"&gt;Certificate Authority (CA) Service&lt;/a&gt;. &lt;a href="https://cloud.google.com/blog/products/networking/traffic-director-integrates-with-ca-service"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;How one business modernized their data warehouse for customer success&lt;/b&gt;–PedidosYa migrated from their old data warehouse to Google Cloud's BigQuery. Now with BigQuery, the Latin American online food ordering company has reduced the total cost per query by 5x. &lt;a href="https://cloud.google.com/blog/products/data-analytics/pedidosya-cuts-cost-per-query-by-5x-using-bigquery"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Announcing new Cloud TPU VMs&lt;/b&gt;–New Cloud TPU VMs make it easier to use our &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/google-breaks-ai-performance-records-in-mlperf-with-worlds-fastest-training-supercomputer"&gt;industry-leading TPU hardware&lt;/a&gt; by providing direct access to TPU host machines, offering a new and improved user experience to develop and deploy TensorFlow, PyTorch, and JAX on &lt;a href="https://cloud.google.com/tpu"&gt;Cloud TPUs&lt;/a&gt;. &lt;a href="https://cloud.google.com/blog/products/compute/introducing-cloud-tpu-vms"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Introducing logical replication and decoding for Cloud SQL for PostgreSQL&lt;/b&gt;–We’re announcing the public preview of &lt;a href="https://cloud.google.com/sql/docs/postgres/replication/configure-logical-replication"&gt;logical replication and decoding for Cloud SQL for PostgreSQL.&lt;/a&gt; By releasing those capabilities and enabling change data capture (CDC) from Cloud SQL for PostgreSQL, we strengthen our commitment to building an open database platform that meets critical application requirements and integrates seamlessly with &lt;a href="https://cloud.google.com/sql/docs/postgres"&gt;the PostgreSQL ecosystem&lt;/a&gt;. &lt;a href="https://cloud.google.com/blog/products/databases/you-can-now-use-cdc-from-cloudsql-for-postgresql"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;How 6 businesses are transforming with SAP on Google Cloud&lt;/b&gt;–Thousands of organizations globally rely on SAP for their most mission critical workloads. And for many Google Cloud customers, part of a broader digital transformation journey has included accelerating the migration of these essential SAP workloads to Google Cloud for greater agility, elasticity, and uptime. &lt;a href="https://cloud.google.com/blog/products/sap-google-cloud/6-businesses-transforming-sap-google-cloud"&gt;Read six of their stories&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/sap-google-cloud/6-businesses-transforming-sap-google-cloud/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/sap.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;6 businesses transforming with SAP on Google Cloud&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Businesses globally are running SAP on Google Cloud to take advantage of greater agility, uptime, and access to cutting edge smart analyt...&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of May 24-May 28 2021&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Google Cloud for financial services: driving your transformation cloud journey&lt;/b&gt;–As we welcome the industry to our Financial Services Summit, we’re sharing more on how Google Cloud accelerates a financial organization’s digital transformation through app and infrastructure modernization, data democratization, people connections, and trusted transactions. &lt;a href="https://cloud.google.com/blog/topics/financial-services/helping-the-financial-services-industry-transform"&gt;Read more&lt;/a&gt; or &lt;a href="https://cloudonair.withgoogle.com/events/summit-finserv?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=FY21-Q2-NORTHAM-summit-onlineevent-er-gc-finserve&amp;amp;utm_content=finserv_lp&amp;amp;utm_term=-" target="_blank"&gt;watch the summit on demand&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Introducing Datashare solution for financial services&lt;/b&gt;–We announced the general availability of &lt;a href="https://cloud.google.com/solutions/financial-services/datashare"&gt;Datashare&lt;/a&gt; for financial services, a new Google Cloud solution that brings together the entire capital markets ecosystem—data publishers and data consumers—to exchange market data securely and easily. &lt;a href="https://cloud.google.com/blog/topics/financial-services/announcing-the-general-availability-of-datashare"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Announcing Datastream in Preview&lt;/b&gt;–&lt;a href="http://cloud.google.com/datastream"&gt;Datastream&lt;/a&gt;, a serverless change data capture (CDC) and replication service, allows enterprises to synchronize data across heterogeneous databases, storage systems, and applications reliably and with minimal latency to support real-time analytics, database replication, and event-driven architectures. &lt;a href="https://cloud.google.com/blog/products/databases/new-cloud-based-cdc-replication-across-databases"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Introducing Dataplex: An intelligent data fabric for analytics at scale&lt;/b&gt;–&lt;a href="http://cloud.google.com/dataplex"&gt;Dataplex&lt;/a&gt; provides a way to centrally manage, monitor, and govern your data across data lakes, data warehouses and data marts, and make this data securely accessible to a variety of analytics and data science tools. &lt;a href="https://cloud.google.com/blog/products/data-analytics/introducing-google-cloud-dataplex"&gt;Read more&lt;/a&gt;. &lt;/li&gt;&lt;li&gt;&lt;b&gt;Announcing Dataflow Prime&lt;/b&gt;–Available in Preview in Q3 2021, Dataflow Prime is a new platform based on a serverless, no-ops, auto-tuning architecture built to bring unparalleled resource utilization and radical operational simplicity to big data processing. Dataflow Prime builds on Dataflow and brings new user benefits with innovations in resource utilization and distributed diagnostics. The new capabilities in Dataflow significantly reduce the time spent on infrastructure sizing and tuning tasks, as well as time spent diagnosing data freshness problems. &lt;a href="https://cloud.google.com/blog/products/data-analytics/simplify-and-automate-data-processing-with-dataflow-prime"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Secure and scalable sharing for data and analytics with Analytics Hub&lt;/b&gt;–With &lt;a href="http://cloud.google.com/analytics-hub"&gt;Analytics Hub&lt;/a&gt;, available in Preview in Q3, organizations get a rich data ecosystem by publishing and subscribing to analytics-ready datasets; control and monitoring over how their data is being used; a self-service way to access valuable and trusted data assets; and an easy way to monetize their data assets without the overhead of building and managing the infrastructure. &lt;a href="https://cloud.google.com/blog/products/data-analytics/introducing-analytics-hub-for-data-analytics-exchanges"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Cloud Spanner trims entry cost by 90%&lt;/b&gt;–Coming soon to Preview, granular instance sizing in Spanner lets organizations run workloads at as low as 1/10th the cost of regular instances, equating to approximately $65/month. &lt;a href="https://cloud.google.com/blog/products/databases/get-more-out-of-spanner-with-granular-instance-sizing"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Cloud Bigtable lifts SLA and adds new security features for regulated industries&lt;/b&gt;–Bigtable instances with a multi-cluster routing policy across 3 or more regions are now covered by a 99.999% monthly uptime percentage under the new SLA. In addition, new Data Access audit logs can help determine whether sensitive customer information has been accessed in the event of a security incident, and if so, when, and by whom. &lt;a href="https://cloud.google.com/blog/products/databases/bigtable-lifts-sla-and-adds-new-security-features"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Build a no-code journaling app&lt;/b&gt;–In honor of Mental Health Awareness Month, Google Cloud's no-code application development platform, AppSheet, demonstrates how you can build a journaling app complete with titles, time stamps, mood entries, and more. Learn how with this blog and video &lt;a href="https://blog.google/products/google-cloud/create-your-own-journaling-app-without-writing-code/" target="_blank"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;New features in Security Command Center&lt;/b&gt;—On May 24th, Security Command Center Premium launched the general availability of granular &lt;a href="https://youtu.be/WlsAMwrHCFA" target="_blank"&gt;access controls at project- and folder-level&lt;/a&gt; and Center for Internet Security (CIS) 1.1 benchmarks for Google Cloud Platform Foundation. These new capabilities enable organizations to improve their security posture and efficiently manage risk for their Google Cloud environment. &lt;a href="https://cloud.google.com/security-command-center/docs/release-notes#May_24_2021"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Simplified API operations with AI&lt;/b&gt;–Google Cloud's API management platform Apigee applies Google's industry leading ML and AI to your API metadata. Understand how it works with anomaly detection &lt;a href="https://cloud.google.com/blog/products/api-management/apigee-x-simplifies-api-management-with-ai"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;This week: Data Cloud and Financial Services Summits&lt;/b&gt;–Our &lt;a href="https://cloudonair.withgoogle.com/summit_series_21?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=-&amp;amp;utm_content=collections-page&amp;amp;utm_term=-" target="_blank"&gt;Google Cloud Summit series&lt;/a&gt; begins this week with the &lt;b&gt;Data Cloud Summit&lt;/b&gt; on Wednesday May 26 (&lt;a href="https://cloudonair.withgoogle.com/events/summit-data-cloud?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=FY21-Q2-NORTHAM-summit-onlineevent-er-gc-data-cloud-summit&amp;amp;utm_content=data_cloud_lp&amp;amp;utm_term=-" target="_blank"&gt;Global&lt;/a&gt;). At this half-day event, you’ll learn how leading companies like PayPal, Workday, Equifax, and many others are driving competitive differentiation using Google Cloud technologies to build their data clouds and transform data into value that drives innovation. The following day, Thursday May 27 (&lt;a href="https://cloudonair.withgoogle.com/events/summit-finserv?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=FY21-Q2-NORTHAM-summit-onlineevent-er-gc-finserve&amp;amp;utm_content=finserv_lp&amp;amp;utm_term=-" target="_blank"&gt;Global&lt;/a&gt; &amp;amp; &lt;a href="https://cloudonair.withgoogle.com/events/summit-emea-finserv?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=FY21-Q2-emea-summit-onlineevent-er-gc-finserve-emea&amp;amp;utm_content=finserv_emea_lp&amp;amp;utm_term=-" target="_blank"&gt;EMEA&lt;/a&gt;) at the &lt;b&gt;Financial Services Summit&lt;/b&gt;, discover how Google Cloud is helping financial institutions such as PayPal, Global Payments, HSBC, Credit Suisse, AXA Switzerland and more unlock new possibilities and accelerate business through innovation. &lt;a href="https://cloud.google.com/blog/topics/events/news-updates-on-the-google-cloud-summit-digital-event-series-2021"&gt;Read more&lt;/a&gt; and &lt;a href="https://cloudonair.withgoogle.com/summit_series_21?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=-&amp;amp;utm_content=collections-page&amp;amp;utm_term=-" target="_blank"&gt;explore the entire summit series&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Announcing the Google for Games Developer Summit 2021 on July 12th-13th&lt;/b&gt;–With a surge of new gamers and an increase in time spent playing games in the last year, it’s more important than ever for game developers to delight and engage players. To help developers with this opportunity, the games teams at Google are back to announce the return of the Google for Games Developer Summit 2021 on July 12th-13th. Hear from experts across Google about new game solutions they’re building to make it easier for you to continue creating great games, connecting with players and scaling your business. Registration is free and open to all game developers. Register for the free online event at &lt;a href="https://developersonair.withgoogle.com/events/game-dev-summit-2021" target="_blank"&gt;g.co/gamedevsummit&lt;/a&gt; to get more details in the coming weeks. We can’t wait to share our latest innovations with the developer community. &lt;a href="https://developersonair.withgoogle.com/events/game-dev-summit-2021" target="_blank"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/developers-practitioners/handy-new-google-cloud-aws-and-azure-product-map/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/727-GC-Cloud_Compare_Blog_Header-AEO-01.max-500x500.png')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;A handy new Google Cloud, AWS, and Azure product map&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;To help developers translate their prior experience with other cloud providers to Google Cloud, we have created a table showing how gener...&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of May 17-May 21 2021&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Best practices to protect your organization against ransomware threats&lt;/b&gt;–For more than 20 years Google has been operating securely in the cloud, using our modern technology stack to provide a more defensible environment that we can protect at scale. While the threat of ransomware isn’t new, our responsibility to help protect you from existing or emerging threats never changes. In our recent blog post, we shared guidance on how organizations can increase their resilience to ransomware and how some of our Cloud products and services can help. &lt;a href="https://cloud.google.com/blog/products/identity-security/5-pillars-of-protection-to-prevent-ransomware-attacks"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Forrester names Google Cloud a Leader in Unstructured Data Security Platforms&lt;/b&gt;–Forrester Research has named Google Cloud a Leader in The Forrester Wave: Unstructured Data Security Platforms, Q2 2021 report, and rated Google Cloud highest in the current offering category among the providers evaluated. &lt;a href="https://cloud.google.com/blog/products/identity-security/google-a-leader-in-unstructured-data-security-platforms"&gt;Read more&lt;/a&gt; or &lt;a href="https://cloud.google.com/resources/forrester-unstructured-data-security-2021-report"&gt;download the report&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Introducing Vertex AI: One platform, every ML tool you need&lt;/b&gt;–&lt;a href="https://cloud.google.com/vertex-ai"&gt;Vertex AI&lt;/a&gt; is a managed machine learning (ML) platform that allows companies to accelerate the deployment and maintenance of artificial intelligence (AI) models. &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/google-cloud-launches-vertex-ai-unified-platform-for-mlops"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Transforming collaboration in Google Workspace&lt;/b&gt;–We’re launching smart canvas, a new product experience that delivers the next evolution of collaboration for Google Workspace. Between now and the end of the year, we’re rolling out innovations that make it easier for people to stay connected, focus their time and attention, and transform their ideas into impact. &lt;a href="https://cloud.google.com/blog/products/workspace/next-evolution-of-collaboration-for-google-workspace"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Developing next-generation geothermal power&lt;/b&gt;–At I/O this week, we announced a first-of-its-kind, next-generation geothermal project with clean-energy startup Fervo that will soon begin adding carbon-free energy to the electric grid that serves our data centers and infrastructure throughout Nevada, including our Cloud region in Las Vegas. &lt;a href="https://cloud.google.com/blog/products/infrastructure/google-fervo-geothermal-project-creates-carbon-free-energy"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Contributing to an environment of trust and transparency in Europe&lt;/b&gt;–Google Cloud was one of the first cloud providers to support and adopt the &lt;a href="https://scope-europe.eu/en/projects/eu-cloud-code-of-conduct/" target="_blank"&gt;EU GDPR Cloud Code of Conduct &lt;/a&gt;(CoC). The CoC is a mechanism for cloud providers to demonstrate how they offer sufficient guarantees to implement appropriate technical and organizational measures as data processors under the GDPR. This week, the Belgian Data Protection Authority, based on a positive opinion by the &lt;a href="https://edpb.europa.eu/about-edpb/about-edpb/who-we-are_en" target="_blank"&gt;European Data Protection Board&lt;/a&gt; (EDPB), &lt;a href="https://www.autoriteprotectiondonnees.be/lautorite-de-protection-des-donnees-approuve-son-premier-code-de-conduite-europeen" target="_blank"&gt;approved&lt;/a&gt; the CoC, a product of years of constructive collaboration between the cloud computing community, the European Commission, and European data protection authorities. We are proud to say that &lt;a href="https://cloud.google.com/terms/services"&gt;Google Cloud Platform&lt;/a&gt; and &lt;a href="https://workspace.google.com/intl/en/terms/user_features.html" target="_blank"&gt;Google Workspace&lt;/a&gt; already &lt;a href="https://cloud.google.com/security/compliance/eu-cloud-code-of-conduct"&gt;adhere&lt;/a&gt; to these provisions. &lt;a href="https://cloud.google.com/blog/products/compliance/google-cloud-adopts-eu-gdpr-cloud-code-of-conduct"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Announcing Google Cloud datasets solutions&lt;/b&gt;–We're adding commercial, synthetic, and first-party data to our &lt;a href="https://cloud.google.com/solutions/datasets"&gt;Google Cloud Public Datasets Program&lt;/a&gt; to help organizations increase the value of their analytics and AI initiatives, and we're making available an &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/datasets-google-cloud-introducing-our-new-reference-architecture"&gt;open source reference architecture&lt;/a&gt; for a more streamlined data onboarding process to the program. &lt;a href="https://cloud.google.com/blog/products/data-analytics/datasets-for-enriching-analytics-ai-initiatives"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Introducing custom samples in Cloud Code&lt;/b&gt;–With new &lt;a href="https://cloud.google.com/code/docs/intellij/set-up-template-repo"&gt;custom samples&lt;/a&gt; in &lt;a href="https://cloud.google.com/code"&gt;Cloud Code&lt;/a&gt;, developers can quickly access your enterprise’s best code samples via a versioned Git repository directly from their IDEs. &lt;a href="https://cloud.google.com/blog/products/application-development/access-an-orgs-custom-code-repo-from-cloud-code-ides"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Retention settings for Cloud SQL&lt;/b&gt;–Cloud SQL now allows you to configure backup retention settings to protect against data loss. You can retain between 1 and 365 days’ worth of automated backups and between 1 and 7 days’ worth of transaction logs for point-in-time recovery. See the details &lt;a href="https://cloud.google.com/sql/docs/postgres/backup-recovery/backups#retention"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Cloud developer’s guide to Google I/O 2021&lt;/b&gt;–&lt;a href="https://events.google.com/io/?lng=en" target="_blank"&gt;Google I/O&lt;/a&gt; may look a little different this year, but don’t worry, you’ll still get the same first-hand look at the newest launches and projects coming from Google. Best of all, it’s free and available to all (virtually) on May 18-20. &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/cloud-developers-guide-google-io-2021"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/anthos/anthos-101-learning-series-all-the-videos-in-one-place/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/Google_Cloud_Anthos_A.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Anthos 101 learning series: All the videos in one place&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;In under an hour, you’ll learn how Anthos lets you develop, run and secure applications across your hybrid and multicloud environments.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of May 10-May 14 2021&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;APIs and Apigee power modern day due diligence&lt;/b&gt;–With APIs and Google Cloud's Apigee, business due diligence company DueDil revolutionized the way they harness and share their Big Information Graph (B.I.G.) with partners and customers. &lt;a href="https://cloud.google.com/blog/products/apigee/google-cloud-helps-duedil-to-deliver-data-insights-at-scale"&gt;Get the full story&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Cloud CISO Perspectives: May 2021&lt;/b&gt;–It’s been a busy month here at Google Cloud since our inaugural CISO perspectives blog post in April. Here, VP and CISO of Google Cloud Phil Venables recaps our cloud security and industry highlights, a sneak peak of what’s ahead from Google at RSA, and more. &lt;a href="https://cloud.google.com/blog/products/identity-security/cloud-ciso-perspectives-may-2021"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;4 new features to secure your Cloud Run services&lt;/b&gt;–We announced several new ways to secure Cloud Run environments to make developing and deploying containerized applications easier for developers. &lt;a href="https://cloud.google.com/blog/products/serverless/improving-the-security-of-your-cloud-run-environment"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Maximize your Cloud Run investments with new committed use discounts&lt;/b&gt;–We’re introducing self-service spend-based &lt;a href="https://cloud.google.com/run/cud"&gt;committed use discounts for Cloud Run&lt;/a&gt;, which let you commit for a year to spending a certain amount on Cloud Run and benefiting from a 17% discount on the amount you committed. &lt;a href="https://cloud.google.com/blog/products/serverless/introducing-committed-use-discounts-for-cloud-run"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Google Cloud Armor Managed Protection Plus is now generally available&lt;/b&gt;–Cloud Armor, our Distributed Denial of Service (DDoS) protection and Web-Application Firewall (WAF) service on Google Cloud, leverages the same infrastructure, network, and technology that has protected Google’s internet-facing properties from some of the &lt;a href="https://cloud.google.com/blog/products/identity-security/identifying-and-protecting-against-the-largest-ddos-attacks"&gt;largest attacks ever reported&lt;/a&gt;. These same tools protect customers’ infrastructure from DDoS attacks, which are increasing in both magnitude and complexity every year. Deployed at the very edge of our network, Cloud Armor absorbs malicious network- and protocol-based volumetric attacks, while mitigating the &lt;a href="https://owasp.org/www-project-top-ten/" target="_blank"&gt;OWASP Top 10 risks&lt;/a&gt; and maintaining the availability of protected services. &lt;a href="https://cloud.google.com/blog/products/identity-security/google-cloud-armor-managed-protection-plus-is-now-ga"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Announcing Document Translation for Translation API Advanced in preview&lt;/b&gt;–Translation is critical to many developers and localization providers, whether you’re releasing a document, a piece of software, training materials or a website in multiple languages. With Document Translation, now you can directly translate documents in 100+ languages and formats such as Docx, PPTx, XLSx, and PDF while preserving document formatting. &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/google-cloud-ai-translation-services-works-for-documents"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Introducing BeyondCorp Enterprise protected profiles&lt;/b&gt;–Protected profiles enable users to securely access corporate resources from an unmanaged device with the same threat and data protections available in BeyondCorp Enterprise–all from the Chrome Browser. &lt;a href="https://cloud.google.com/blog/products/identity-security/new-protected-profiles-enable-remote-work-with-zero-trust"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;How reCAPTCHA Enterprise protects unemployment and COVID-19 vaccination portals&lt;/b&gt;–With so many people visiting government websites to learn more about the COVID-19 vaccine, make vaccine appointments, or file for unemployment, these web pages have become prime targets for bot attacks and other abusive activities. But reCAPTCHA Enterprise has helped state governments protect COVID-19 vaccine registration portals and unemployment claims portals from abusive activities. &lt;a href="https://cloud.google.com/blog/products/identity-security/recaptcha-protects-public-services"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Day one with Anthos? Here are 6 ideas for how to get started&lt;/b&gt;–Once you have your new application platform in place, there are some things you can do to immediately get value and gain momentum. Here are six things you can do to get you started. &lt;a href="https://cloud.google.com/blog/topics/anthos/6-initiatives-to-start-on-your-day-one-anthos-journey"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;The era of the transformation cloud is here&lt;/b&gt;–Google Cloud’s president Rob Enslin shares how the era of the transformation cloud has seen organizations move beyond data centers to change not only where their business is done but, more importantly, how it is done. &lt;a href="https://cloud.google.com/blog/topics/inside-google-cloud/innovation-in-the-era-of-the-transformation-cloud"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/devops-sre/sre-at-google-our-complete-list-of-cre-life-lessons/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/DevOps.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;SRE at Google: Our complete list of CRE life lessons&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Find links to blog posts that share Google’s SRE best practices in one handy location.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;&lt;/h3&gt;&lt;h3&gt;Week of May 3-May 7 2021&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Transforming hard-disk drive maintenance with predictive ML&lt;/b&gt;–In collaboration with Seagate, we developed a machine learning system that can forecast the probability of a recurring failing disk—a disk that fails or has experienced three or more problems in 30 days. &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/seagate-and-google-predict-hard-disk-drive-failures-with-ml"&gt;Learn how we did it&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Agent Assist for Chat is now in public preview&lt;/b&gt;–&lt;a href="https://cloud.google.com/agent-assist"&gt;Agent Assist&lt;/a&gt; provides your human agents with continuous support during their calls, and now chats, by identifying the customers’ intent and providing them with real-time recommendations such as articles and FAQs as well as responses to customer messages to more effectively resolve the conversation. &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/contact-center-ai-agent-assist-for-chat-is-now-in-public-preview"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;New Google Cloud, AWS, and Azure product map&lt;/b&gt;–Our updated product map helps you understand similar offerings from Google Cloud, AWS, and Azure, and you can easily filter the list by product name or other common keywords. &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/handy-new-google-cloud-aws-and-azure-product-map"&gt;Read more&lt;/a&gt; or &lt;a href="https://cloud.google.com/free/docs/aws-azure-gcp-service-comparison?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=FY21-Q2-Product-Mapping-Blog&amp;amp;utm_content=documentation"&gt;view the map&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Join our Google Cloud Security Talks on May 12th&lt;/b&gt;–We’ll share expert insights into how we’re working to be your most &lt;a href="https://cloud.google.com/blog/products/identity-security/delivering-the-industrys-most-trusted-cloud"&gt;trusted cloud&lt;/a&gt;. Find the list of topics we’ll cover &lt;a href="https://cloud.google.com/blog/products/identity-security/google-cloud-security-talks-q2-2021"&gt;here&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Databricks is now GA on Google Cloud&lt;/b&gt;–Deploy or migrate Databricks Lakehouse to Google Cloud to combine the benefits of an open data cloud platform with greater analytics flexibility, unified infrastructure management, and optimized performance. &lt;a href="https://cloud.google.com/blog/products/data-analytics/databricks-on-google-cloud-now-ga"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;HPC VM image is now GA&lt;/b&gt;–The CentOS-based HPC VM image makes it quick and easy to create HPC-ready VMs on Google Cloud that are pre-tuned for optimal performance. Check out our &lt;a href="http://cloud.google.com/compute/docs/instances/create-hpc-vm"&gt;documentation and quickstart guide&lt;/a&gt; to start creating instances using the HPC VM image today.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Take the 2021 State of DevOps survey&lt;/b&gt;–Help us shape the future of DevOps and make your voice heard by completing the 2021 State of DevOps survey before June 11, 2021. &lt;a href="https://cloud.google.com/blog/products/devops-sre/take-2021-state-devops-survey-shape-future-devops"&gt;Read more&lt;/a&gt; or &lt;a href="https://google.qualtrics.com/jfe/form/SV_cIb0SmhJPfm8H7n" target="_blank"&gt;take the survey&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;OpenTelemetry Trace 1.0 is now available&lt;/b&gt;–OpenTelemetry has reached a key milestone: the OpenTelemetry Tracing Specification has reached version &lt;a href="https://github.com/open-telemetry/opentelemetry-specification" target="_blank"&gt;1.0&lt;/a&gt;. API and SDK release candidates are available for &lt;a href="https://github.com/open-telemetry/opentelemetry-java" target="_blank"&gt;Java&lt;/a&gt;, &lt;a href="https://github.com/open-telemetry/opentelemetry-erlang" target="_blank"&gt;Erlang&lt;/a&gt;, &lt;a href="https://github.com/open-telemetry/opentelemetry-python" target="_blank"&gt;Python&lt;/a&gt;, &lt;a href="https://github.com/open-telemetry/opentelemetry-go" target="_blank"&gt;Go&lt;/a&gt;, &lt;a href="https://github.com/open-telemetry/opentelemetry-js" target="_blank"&gt;Node.js&lt;/a&gt;, and &lt;a href="https://github.com/open-telemetry/opentelemetry-dotnet" target="_blank"&gt;.Net&lt;/a&gt;. Additional languages will follow over the next few weeks. &lt;a href="https://cloud.google.com/blog/products/operations/opentelemetry-specification-enables-standardized-tracing"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;New blueprint helps secure confidential data in AI Platform Notebooks&lt;/b&gt;–We’re adding to our portfolio of blueprints with the publication of our &lt;a href="https://cloud.google.com/architecture/blueprints/protecting-confidential-data-in-ai-platform-notebooks"&gt;Protecting confidential data in AI Platform Notebooks blueprint guide&lt;/a&gt; and &lt;a href="https://github.com/GoogleCloudPlatform/notebooks-blueprint-security" target="_blank"&gt;deployable blueprint&lt;/a&gt;, which can help you apply data governance and security policies that protect your AI Platform Notebooks containing confidential data. &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/ai-platform-notebooks-security-blueprint"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;The Liquibase Cloud Spanner extension is now GA&lt;/b&gt;–&lt;a href="https://www.liquibase.org/" target="_blank"&gt;Liquibase&lt;/a&gt;, an open-source library that works with a wide variety of databases, can be used for tracking, managing, and automating database schema changes. By providing the ability to integrate databases into your CI/CD process, Liquibase helps you more fully adopt &lt;a href="https://cloud.google.com/devops"&gt;DevOps&lt;/a&gt; practices. The Liquibase Cloud Spanner extension allows developers to use Liquibase's open-source database library to manage and automate schema changes in Cloud Spanner. &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/include-cloud-spanner-databases-your-cicd-process-liquibase-extension"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Cloud computing 101: Frequently asked questions&lt;/b&gt;–There are a number of terms and concepts in cloud computing, and not everyone is familiar with all of them. To help, we’ve put together a list of common questions, and the meanings of a few of those acronyms. &lt;a href="https://cloud.google.com/blog/topics/inside-google-cloud/cloud-computing-101-frequently-asked-questions"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/api-management/api-design-101-links-our-most-popular-posts/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/Google_Blog_api-management-01_1.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;API design 101: Links to our most popular posts&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Find our most requested blog posts on API design in one location to read now or bookmark for later.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Apr 26-Apr 30 2021&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Announcing the GKE Gateway controller, in Preview&lt;/b&gt;–GKE Gateway controller, Google Cloud’s implementation of the Gateway API, manages internal and external HTTP/S load balancing for a GKE cluster or a fleet of GKE clusters and provides multi-tenant sharing of load balancer infrastructure with centralized admin policy and control. &lt;a href="https://cloud.google.com/blog/products/containers-kubernetes/new-gke-gateway-controller-implements-kubernetes-gateway-api"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;See Network Performance for Google Cloud in Performance Dashboard&lt;/b&gt;–The &lt;a href="https://console.cloud.google.com/net-intelligence/performance/global-dashboard/packet-loss"&gt;Google Cloud performance view&lt;/a&gt;, part of the Network Intelligence Center, provides packet loss and latency metrics for traffic on Google Cloud. It allows users to do informed planning of their deployment architecture, as well as determine in real time the answer to the most common troubleshooting question: "Is it Google or is it me?" The Google Cloud performance view is now open for all Google Cloud customers as a public preview. &lt;a href="https://console.cloud.google.com/net-intelligence/performance/global-dashboard/packet-loss"&gt;Check it out&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Optimizing data in Google Sheets allows users to create no-code apps&lt;/b&gt;–Format columns and tables in Google Sheets to best position your data to transform into a fully customized, successful app–no coding necessary. Read our four best Google Sheets &lt;a href="https://cloud.google.com/blog/products/no-code-development/turn-google-sheet-data-into-a-no-code-web-app-with-appsheet"&gt;tips.&lt;/a&gt;&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Automation bots with AppSheet Automation&lt;/b&gt;–AppSheet recently released &lt;a href="https://cloud.google.com/blog/products/no-code-development/automation-no-code-appsheet-and-google-cloud"&gt;AppSheet Automation&lt;/a&gt;, infusing Google AI capabilities to AppSheet's trusted no-code app development platform. Learn step by step how to build your first automation bot on AppSheet &lt;a href="https://cloud.google.com/blog/products/no-code-development/automation-bots-with-appsheet-and-no-code"&gt;here.&lt;/a&gt; &lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Google Cloud announces a new region in Israel&lt;/b&gt;–Our new region in Israel will make it easier for customers to serve their own users faster, more reliably and securely. &lt;a href="https://cloud.google.com/blog/products/infrastructure/new-google-cloud-region-in-israel"&gt;Read more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;New multi-instance NVIDIA GPUs on GKE&lt;/b&gt;–We’re launching support for multi-instance GPUs in GKE (currently in Preview), which will help you drive better value from your GPU investments. &lt;a href="https://cloud.google.com/blog/products/containers-kubernetes/gke-now-supports-multi-instance-gpus"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Partnering with NSF to advance networking innovation&lt;/b&gt;–We announced our partnership with the U.S. National Science Foundation (NSF), joining other industry partners and federal agencies, as part of a combined $40 million investment in academic research for &lt;a href="https://www.nsf.gov/news/special_reports/announcements/042721.jsp" target="_blank"&gt;Resilient and Intelligent Next-Generation (NextG) Systems&lt;/a&gt;, or RINGS. &lt;a href="https://cloud.google.com/blog/products/networking/google-partnering-with-nsf-to-innovate-networking"&gt;Read more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Creating a policy contract with Configuration as Data&lt;/b&gt;–Configuration as Data is an emerging cloud infrastructure management paradigm that allows developers to &lt;a href="https://cloud.google.com/blog/products/containers-kubernetes/understanding-configuration-as-data-in-kubernetes"&gt;declare the desired state&lt;/a&gt; of their applications and infrastructure, without specifying the precise actions or steps for how to achieve it. However, declaring a configuration is only half the battle: you also want policy that defines how a configuration is to be used. &lt;a href="https://cloud.google.com/blog/products/containers-kubernetes/how-configuration-as-data-impacts-policy"&gt;This post shows you how&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Google Cloud products deliver real-time data solutions&lt;/b&gt;–Seven-Eleven Japan built Seven Central, its new platform for digital transformation, on Google Cloud. Powered by BigQuery, Cloud Spanner, and Apigee API management, Seven Central presents easy to understand data, ultimately allowing for quickly informed decisions. Read their story &lt;a href="https://cloud.google.com/blog/products/data-analytics/how-7-eleven-japan-built-its-new-data-platform"&gt;here&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/training-certifications/google-cloud-free-training-opportunities-q1-2021/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/GCP_Training.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;In case you missed it: All our free Google Cloud training opportunities from Q1&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Since January, we’ve introduced a number of no-cost training opportunities to help you grow your cloud skills. We've brought them togethe...&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Apr 19-Apr 23 2021&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Extreme PD is now GA&lt;/b&gt;–On April 20th, Google Cloud’s &lt;a href="https://cloud.google.com/persistent-disk"&gt;Persistent Disk&lt;/a&gt; launched general availability of Extreme PD, a high performance block storage volume with provisioned IOPS and up to 2.2 GB/s of throughput. &lt;a href="https://cloud.google.com/compute/docs/disks/extreme-persistent-disk"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Research: How data analytics and intelligence tools to play a key role post-COVID-19&lt;/b&gt;–A recent &lt;a href="https://inthecloud.withgoogle.com/it-leaders-research-21/overview-dl-cd.html?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=FY21-Q1-NORTHAM-demandgen-website-wd-gcp_gtm_reimagine_master&amp;amp;utm_content=homepage" target="_blank"&gt;Google-commissioned study by IDG&lt;/a&gt; highlighted the role of data analytics and intelligent solutions when it comes to helping businesses separate from their competition. The survey of 2,000 IT leaders across the globe reinforced the notion that the ability to derive insights from data will go a long way towards determining which companies win in this new era. &lt;a href="https://cloud.google.com/blog/products/data-analytics/post-covid-data-analytics-and-intelligence-tools"&gt;Learn more&lt;/a&gt; or &lt;a href="https://inthecloud.withgoogle.com/it-leaders-research-21/overview-dl-cd.html?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=FY21-Q1-NORTHAM-demandgen-website-wd-gcp_gtm_reimagine_master&amp;amp;utm_content=homepage" target="_blank"&gt;download the study&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Introducing PHP on Cloud Functions&lt;/b&gt;–We’re bringing support for PHP, a popular general-purpose programming language, to Cloud Functions. With the Functions Framework for PHP, you can write idiomatic PHP functions to build business-critical applications and integration layers. And with Cloud Functions for PHP, now available in Preview, you can deploy functions in a fully managed PHP 7.4 environment, complete with access to resources in a private VPC network. &lt;a href="https://cloud.google.com/blog/products/application-development/php-comes-to-cloud-functions"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Delivering our 2020 CCAG pooled audit&lt;/b&gt;–As our customers increased their use of cloud services to &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/what-zero-trust-identity-security"&gt;meet the demands of teleworking&lt;/a&gt; and&lt;a href="https://cloud.google.com/blog/topics/public-sector/getting-vaccines-local-communities-safely-and-effectively"&gt; aid in COVID-19 recovery&lt;/a&gt;, we’ve worked hard to meet our commitment to being the industry’s most &lt;a href="https://cloud.google.com/blog/products/identity-security/delivering-the-industrys-most-trusted-cloud"&gt;trusted cloud&lt;/a&gt;, despite the global pandemic. We’re proud to announce that Google Cloud completed an annual pooled audit with the CCAG in a completely remote setting, and were the only cloud service provider to do so in 2020. &lt;a href="https://cloud.google.com/blog/products/compliance/google-completes-annual-pooled-audit-with-ccag-completely-remote"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Anthos 1.7 now available&lt;/b&gt;–We recently released Anthos 1.7, our run-anywhere Kubernetes platform that’s connected to Google Cloud, delivering an array of capabilities that make multicloud more accessible and sustainable. &lt;a href="https://cloud.google.com/blog/topics/anthos/whats-new-in-anthos"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;New Redis Enterprise for Anthos and GKE&lt;/b&gt;–We’re making Redis Enterprise for Anthos and Google Kubernetes Engine (GKE) available in the Google Cloud Marketplace in private preview. &lt;a href="https://cloud.google.com/blog/products/containers-kubernetes/redis-enterprise-available-for-gke-and-anthos"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Updates to Google Meet&lt;/b&gt;–We introduced a refreshed user interface (UI), enhanced reliability features powered by the latest Google AI, and tools that make meetings more engaging—even fun—for everyone involved. &lt;a href="https://cloud.google.com/blog/products/google-meet/new-features-for-google-meet"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;DocAI solutions now generally available&lt;/b&gt;–&lt;a href="https://cloud.google.com/document-ai"&gt;Document (Doc) AI platform&lt;/a&gt;, &lt;a href="https://cloud.google.com/solutions/lending-doc-ai"&gt;Lending DocAI&lt;/a&gt; and &lt;a href="https://cloud.google.com/solutions/procurement-doc-ai"&gt;Procurement DocAI&lt;/a&gt;, built on decades of AI innovation at Google, bring powerful and useful solutions across lending, insurance, government and other industries. &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/get-more-value-from-your-documents-with-docai-and-industry-solutions"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Four consecutive years of 100% renewable energy&lt;/b&gt;–In 2020, Google again matched 100 percent of its global electricity use with purchases of renewable energy. All told, we’ve signed agreements to buy power from more than 50 renewable energy projects, with a combined capacity of 5.5 gigawatts–about the same as a million solar rooftops. &lt;a href="https://cloud.google.com/blog/topics/sustainability/google-achieves-four-consecutive-years-of-100-percent-renewable-energy"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Announcing the Google Cloud region picker&lt;/b&gt;–The &lt;a href="https://gcp-region-picker.web.app/" target="_blank"&gt;Google Cloud region picker&lt;/a&gt; lets you assess key inputs like price, latency to your end users, and carbon footprint to help you choose which Google Cloud region to run on. &lt;a href="https://cloud.google.com/blog/topics/sustainability/google-cloud-region-picker-helps-you-make-the-green-choice"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Google Cloud launches new security solution WAAP&lt;/b&gt;–WebApp and API Protection (WAAP) combines Google Cloud Armor, Apigee, and reCAPTCHA Enterprise to deliver improved threat protection, consolidated visibility, and greater operational efficiencies across clouds and on-premises environments. Learn more about WAAP &lt;a href="https://cloud.google.com/blog/products/api-management/better-app-security-with-google-clouds-waap"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;New in no-code&lt;/b&gt;–As discussed in our recent article, no-code hackathons are trending among innovative organizations. Since then, we've outlined how you can host one yourself specifically designed for your unique business innovation outcomes. Learn how &lt;a href="https://cloud.google.com/blog/products/no-code-development/no-code-hackathon-tips-and-best-practices"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Google Cloud Referral Program now available&lt;/b&gt;—Now you can share the power of Google Cloud and earn product credit for every new paying customer you refer. Once you join the program, you’ll get a unique referral link that you can share with friends, clients, or others. Whenever someone signs up with your link, they’ll get a $350 product credit—that’s $50 more than the standard trial credit. When they become a paying customer, we’ll reward you with a $100 product credit in your Google Cloud account. Available in the United States, Canada, Brazil, and Japan. &lt;a href="https://cloud.google.com/referral-program#apply-now"&gt;Apply for the Google Cloud Referral Program&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/gcp/5-google-cloud-product-cheat-sheets-2021/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/MigrationCheatsheet_UCAMRlT.1419060723021215.max-500x500.png')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;5 cheat sheets to help you get started on your Google Cloud journey&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Whether you need to determine the best way to move to the cloud, or decide on the best storage option, we've built a number of cheat shee...&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Apr 12-Apr 16 2021&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Announcing the Data Cloud Summit, May 26, 2021&lt;/b&gt;–At this half-day event, you’ll learn how leading companies like PayPal, Workday, Equifax, Zebra Technologies, Commonwealth Care Alliance and many others are driving competitive differentiation using Google Cloud technologies to build their data clouds and transform data into value that drives innovation. &lt;a href="https://cloud.google.com/blog/topics/events/news-updates-on-the-google-cloud-summit-digital-event-series-2021"&gt;Learn more&lt;/a&gt; and &lt;a href="https://cloudonair.withgoogle.com/events/summit-data-cloud" target="_blank"&gt;register at no cost&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Announcing the Financial Services Summit, May 27, 2021&lt;/b&gt;–In this 2 hour event, you’ll learn how Google Cloud is helping financial institutions including PayPal, Global Payments, HSBC, Credit Suisse, and more unlock new possibilities and accelerate business through innovation and better customer experiences. &lt;a href="https://cloud.google.com/blog/topics/events/news-updates-on-the-google-cloud-summit-digital-event-series-2021"&gt;Learn more&lt;/a&gt; and register for free: &lt;a href="https://cloudonair.withgoogle.com/events/summit-finserv" target="_blank"&gt;Global&lt;/a&gt; &amp;amp; &lt;a href="https://cloudonair.withgoogle.com/events/summit-emea-finserv" target="_blank"&gt;EMEA&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;How Google Cloud is enabling vaccine equity&lt;/b&gt;–In our latest update, we share more on how we’re working with US state governments to help produce equitable vaccination strategies at scale. &lt;a href="https://cloud.google.com/blog/topics/public-sector/how-google-cloud-enabling-vaccine-equity"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;The new Google Cloud region in Warsaw is open&lt;/b&gt;–The Google Cloud region in Warsaw is now ready for business, opening doors for organizations in Central and Eastern Europe. &lt;a href="https://cloud.google.com/blog/products/infrastructure/google-cloud-region-in-warsaw-poland-is-now-open"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;AppSheet Automation is now GA&lt;/b&gt;–Google Cloud’s AppSheet launches general availability of AppSheet Automation, a unified development experience for citizen and professional developers alike to build custom applications with automated processes, all without coding. Learn how companies and employees are reclaiming their time and talent with AppSheet Automation &lt;a href="https://cloud.google.com/blog/products/no-code-development/automation-no-code-appsheet-and-google-cloud"&gt;here.&lt;/a&gt;&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Introducing SAP Integration with Cloud Data Fusion&lt;/b&gt;–Google Cloud native data integration platform Cloud Data Fusion now offers the capability to seamlessly get data out of SAP Business Suite, SAP ERP and S/4HANA. &lt;a href="https://cloud.google.com/blog/products/data-analytics/introducing-sap-integration-cloud-data-fusion"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/devops-sre/sre-fundamentals-sli-vs-slo-vs-sla/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/DevOps_BlogHeader_B_Rnd3.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;SRE fundamentals 2021: SLIs vs SLAs vs SLOs&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;What’s the difference between an SLI, an SLO and an SLA? Google Site Reliability Engineers (SRE) explain.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Apr 5-Apr 9 2021&lt;br/&gt;&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;New Certificate Authority Service (CAS) whitepaper&lt;/b&gt;–“How to deploy a secure and reliable public key infrastructure with Google Cloud Certificate Authority Service” (written by Mark Cooper of PKI Solutions and Anoosh Saboori of Google Cloud) covers security and architectural recommendations for the use of the Google Cloud CAS by organizations, and describes critical concepts for securing and deploying a PKI based on CAS. &lt;a href="https://cloud.google.com/blog/products/identity-security/how-to-deploy-google-cas-whitepaper"&gt;Learn more&lt;/a&gt; or read the &lt;a href="https://services.google.com/fh/files/misc/deploying_public_key_infrastructure_with_cas.pdf" target="_blank"&gt;whitepaper&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Active Assist’s new feature, &lt;i&gt;predictive autoscaling&lt;/i&gt;, helps improve response times for your applications&lt;/b&gt;–When you enable predictive autoscaling, Compute Engine forecasts future load based on your Managed Instance Group’s (MIG) history and scales it out in advance of predicted load, so that new instances are ready to serve when the load arrives. Without predictive autoscaling, an autoscaler can only scale a group reactively, based on observed changes in load in real time. With predictive autoscaling enabled, the autoscaler works with real-time data as well as with historical data to cover both the current and forecasted load. That makes predictive autoscaling ideal for those apps with long initialization times and whose workloads vary predictably with daily or weekly cycles. For more information, see &lt;a href="https://cloud.google.com/compute/docs/autoscaler/predictive-autoscaling#how_predictive_autoscaling_works"&gt;How predictive autoscaling works&lt;/a&gt; or &lt;a href="https://cloud.google.com/compute/docs/autoscaler/predictive-autoscaling#checking_if_predictive_autoscaling_is_suitable_for_your_workload"&gt;check if predictive autoscaling is suitable for your workload&lt;/a&gt;, and to learn more about other intelligent features, check out &lt;a href="https://cloud.google.com/solutions/active-assist"&gt;Active Assist&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Introducing Dataprep BigQuery pushdown&lt;/b&gt;–BigQuery pushdown gives you the flexibility to run jobs using either BigQuery or Dataflow. If you select BigQuery, then Dataprep can automatically determine if data pipelines can be partially or fully translated in a BigQuery SQL statement. Any portions of the pipeline that cannot be run in BigQuery are executed in Dataflow. Utilizing the power of BigQuery results in highly efficient data transformations, especially for manipulations such as filters, joins, unions, and aggregations. This leads to better performance, optimized costs, and increased security with IAM and OAuth support. &lt;a href="https://cloud.google.com/blog/products/data-analytics/introducing-dataprep-bigquery-pushdown"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Announcing the Google Cloud Retail &amp;amp; Consumer Goods Summit&lt;/b&gt;–The Google Cloud Retail &amp;amp; Consumer Goods Summit brings together technology and business insights, the key ingredients for any transformation. Whether you're responsible for IT, data analytics, supply chains, or marketing, please join! Building connections and sharing perspectives cross-functionally is important to reimagining yourself, your organization, or the world. &lt;a href="https://cloud.google.com/blog/topics/events/google-cloud-retail-consumer-goods-summit-future-retail"&gt;Learn more&lt;/a&gt; or &lt;a href="https://cloudonair.withgoogle.com/events/retailsummit?utm_content=globalblog&amp;amp;utm_source=Google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=FY21-Q2-emea-eventindustry-onlineevent-er-RetailHorizonSummit" target="_blank"&gt;register for free&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;New IDC whitepaper assesses multicloud as a risk mitigation strategy&lt;/b&gt;–To better understand the benefits and challenges associated with a multicloud approach, we supported IDC’s new whitepaper that investigates how multicloud can help regulated organizations mitigate the risks of using a single cloud vendor. The whitepaper looks at different approaches to multi-vendor and hybrid clouds taken by European organizations and how these strategies can help organizations address concentration risk and vendor-lock in, improve their compliance posture, and demonstrate an exit strategy. &lt;a href="https://cloud.google.com/blog/topics/hybrid-cloud/idc-whitepaper-assesses-multicloud-as-risk-mitigation-strategy"&gt;Learn more&lt;/a&gt; or &lt;a href="https://services.google.com/fh/files/misc/idc_multicloud_whitepaper.pdf" target="_blank"&gt;download the paper&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Introducing request priorities for Cloud Spanner APIs&lt;/b&gt;–You can now specify request priorities for some Cloud Spanner APIs. By assigning a HIGH, MEDIUM, or LOW priority to a specific request, you can now convey the relative importance of workloads, to better align resource usage with performance objectives. &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/introducing-request-priorities-cloud-spanner-apis"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;How we’re working with governments on climate goals&lt;/b&gt;–Google Sustainability Officer Kate Brandt shares more on how we’re partnering with governments around the world to provide our technology and insights to drive progress in sustainability efforts. &lt;a href="https://cloud.google.com/blog/topics/public-sector/working-with-governments-on-climate-goals"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/inside-google-cloud/cloud-computing-101-frequently-asked-questions/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/BlogHeader_Set2_B.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Cloud computing 101: Frequently asked questions&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;What are containers? What’s a data lake? What does that acronym stand for? Get answers to the questions you're too afraid to ask.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Mar 29-Apr 2 2021&lt;br/&gt;&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Why Google Cloud is the ideal platform for Block.one and other DLT companies&lt;/b&gt;–Late last year, Google Cloud &lt;a href="https://block.one/news/welcome-google-cloud-to-eos-community/" target="_blank"&gt;joined&lt;/a&gt; the EOS community, a leading open-source platform for blockchain innovation and performance, and is taking steps to support the EOS Public Blockchain by becoming a &lt;a href="https://developers.eos.io/welcome/latest/protocol/consensus_protocol" target="_blank"&gt;block producer&lt;/a&gt; (BP). At the time, we outlined how our planned participation underscores the importance of blockchain to the future of business, government, and society. We're sharing more on why Google Cloud is uniquely positioned to be an excellent partner for Block.one and other distributed ledger technology (DLT) companies. &lt;a href="https://cloud.google.com/blog/topics/inside-google-cloud/why-google-cloud-is-the-ideal-platform-for-blockone-and-other-dlt-companies"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;New whitepaper: Scaling certificate management with Certificate Authority Service&lt;/b&gt;–As Google Cloud’s Certificate Authority Service (CAS) approaches general availability, we want to help customers understand the service better. Customers have asked us how CAS fits into our larger security story and how CAS works for various use cases. Our new white paper answers these questions and more. &lt;a href="https://cloud.google.com/blog/products/identity-security/how-to-scale-certificate-management-with-google-cas-whitepaper"&gt;Learn more&lt;/a&gt; and &lt;a href="https://services.google.com/fh/files/misc/scaling_certificate_management_cas.pdf" target="_blank"&gt;download the paper&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Build a consistent approach for API consumers&lt;/b&gt;–Learn the differences between REST and GraphQL, as well as how to apply REST-based practices to GraphQL. No matter the approach, discover how to manage and treat both options as API products &lt;a href="https://cloud.google.com/blog/products/api-management/interacting-with-apis-rest-and-graphql"&gt;here.&lt;/a&gt;&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Apigee X makes it simple to apply Cloud CDN to APIs&lt;/b&gt;–With Apigee X and Cloud CDN, organizations can expand their API programs' global reach. Learn how to deploy APIs across 24 regions and 73 zones &lt;a href="https://cloud.google.com/blog/products/api-management/api-performance-and-availability-with-apigee-x-and-cloud-cdn"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Enabling data migration with Transfer Appliances in APAC&lt;/b&gt;—We’re announcing the general availability of Transfer Appliances TA40/TA300 in Singapore. Customers are looking for fast, secure and easy to use options to migrate their workloads to Google Cloud and we are addressing their needs with Transfer Appliances &lt;a href="https://cloud.google.com/transfer-appliance/docs/4.0/overview#location-availability"&gt;globally in the US, EU and APAC&lt;/a&gt;. Learn more about &lt;a href="https://cloud.google.com/transfer-appliance/docs/4.0"&gt;Transfer Appliances TA40 and TA300&lt;/a&gt;.&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Windows Authentication is now supported on Cloud SQL for SQL Server in public preview&lt;/b&gt;—We’ve launched seamless integration with Google Cloud’s Managed Service for Microsoft Active Directory (AD). This capability is a critical requirement to simplify identity management and streamline the migration of existing SQL Server workloads that rely on AD for access control. &lt;a href="https://cloud.google.com/blog/products/databases/windows-authentication-now-supported-by-googles-cloud-sql-database"&gt;Learn more&lt;/a&gt; or &lt;a href="https://cloud.google.com/sql/docs/sqlserver/ad"&gt;get started&lt;/a&gt;.&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Using Cloud AI to whip up new treats with Mars Maltesers&lt;/b&gt;—Maltesers, a popular British candy made by Mars, teamed up with our own &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/baking-recipes-made-ai"&gt;AI baker&lt;/a&gt; and ML engineer extraordinaire, &lt;a href="https://twitter.com/SRobTweets" target="_blank"&gt;Sara Robinson&lt;/a&gt;, to create a brand new dessert recipe with Google Cloud AI. &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/baking-with-maltesers-marmite-and-cloud-ai"&gt;Find out what happened&lt;/a&gt; (recipe included).&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Simplifying data lake management with Dataproc Metastore, now GA&lt;/b&gt;—&lt;a href="https://cloud.google.com/dataproc-metastore/docs"&gt;Dataproc Metastore&lt;/a&gt;, a fully managed, serverless technical metadata repository based on the Apache Hive metastore, is now generally available. Enterprises building and migrating open source data lakes to Google Cloud now have a central and persistent metastore for their open source data analytics frameworks. &lt;a href="https://cloud.google.com/blog/products/data-analytics/data-lake-management-just-got-easier-dataproc-metastore-ga"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Introducing the Echo subsea cable&lt;/b&gt;—We announced our investment in Echo, the first-ever cable to directly connect the U.S. to Singapore with direct fiber pairs over an express route. Echo will run from Eureka, California to Singapore, with a stop-over in Guam, and plans to also land in Indonesia. Additional landings are possible in the future. &lt;a href="https://cloud.google.com/blog/products/infrastructure/introducing-the-echo-subsea-cable"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/inside-google-cloud/21-google-cloud-tools-each-explained-under-2-minutes/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/Public-Sector-Momentum.max-500x500.png')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;21 Google Cloud tools, each explained in under 2 minutes&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Need a quick overview of Google Cloud core technologies? Quickly learn these 21 Google Cloud products—each explained in under two minutes.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Mar 22-Mar 26 2021&lt;br/&gt;&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;10 new videos bring Google Cloud to life&lt;/b&gt;—The &lt;a href="https://www.youtube.com/channel/UCJS9pqu9BzkAMNTmzNMNhvg" target="_blank"&gt;Google Cloud Tech YouTube channel&lt;/a&gt;’s latest video series explains cloud tools for technical practitioners in about 5 minutes each. &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/picture-10-whiteboard-sketch-videos-bring-google-cloud-life"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;BigQuery named a Leader in the 2021 Forrester Wave: Cloud Data Warehouse, Q1 2021 report&lt;/b&gt;—Forrester gave &lt;a href="https://cloud.google.com/bigquery"&gt;BigQuery&lt;/a&gt; a score of 5 out of 5 across 19 different criteria. Learn more in our &lt;a href="https://cloud.google.com/blog/products/data-analytics/google-cloud-a-leader-in-2021-forrester-wave-cloud-data-warehouse"&gt;blog post&lt;/a&gt;, or &lt;a href="https://cloud.google.com/forrester-wave-cloud-data-warehouse-2021"&gt;download the report&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Charting the future of custom compute at Google&lt;/b&gt;—To meet users’ performance needs at low power, we’re doubling down on custom chips that use System on a Chip (SoC) designs. &lt;a href="https://cloud.google.com/blog/topics/systems/the-past-present-and-future-of-custom-compute-at-google"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Introducing Network Connectivity Center&lt;/b&gt;—We announced &lt;a href="https://cloud.google.com/network-connectivity-center"&gt;Network Connectivity Center&lt;/a&gt;, which provides a single management experience to easily create, connect, and manage heterogeneous on-prem and cloud networks leveraging Google’s global infrastructure. Network Connectivity Center serves as a vantage point to seamlessly connect VPNs, partner and dedicated interconnects, as well as third-party routers and Software-Defined WANs, helping you optimize connectivity, reduce operational burden and lower costs—wherever your applications or users may be. &lt;a href="https://cloud.google.com/blog/products/networking/announcing-network-connectivity-center"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Making it easier to get Compute Engine resources for batch processing&lt;/b&gt;—We announced a new method of obtaining &lt;a href="https://cloud.google.com/compute"&gt;Compute Engine&lt;/a&gt; instances for batch processing that accounts for availability of resources in zones of a region. Now available in preview for &lt;a href="https://cloud.google.com/compute/docs/instance-groups/regional-migs"&gt;regional managed instance groups&lt;/a&gt;, you can do this simply by specifying the &lt;code&gt;ANY&lt;/code&gt; value in the API. &lt;a href="https://cloud.google.com/blog/topics/hpc/a-new-way-to-obtain-compute-engine-vms-for-batch-processing"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Next-gen virtual automotive showrooms are here, thanks to Google Cloud, Unreal Engine, and NVIDIA&lt;/b&gt;—We teamed up with &lt;a href="https://www.unrealengine.com/en-US/industry/automotive-transportation" target="_blank"&gt;Unreal Engine&lt;/a&gt;, the open and advanced real-time 3D creation game engine, and NVIDIA, inventor of the GPU, to launch new virtual showroom experiences for automakers. Taking advantage of the &lt;a href="https://www.nvidia.com/en-us/design-visualization/solutions/rendering/" target="_blank"&gt;NVIDIA RTX platform&lt;/a&gt; on Google Cloud, these showrooms provide interactive 3D experiences, photorealistic materials and environments, and up to 4K cloud streaming on mobile and connected devices. Today, in collaboration with &lt;a href="https://www.mhp.com/en/company" target="_blank"&gt;MHP&lt;/a&gt;, the Porsche IT consulting firm, and &lt;a href="https://www.monkey-way.com/" target="_blank"&gt;MONKEYWAY&lt;/a&gt;, a real-time 3D streaming solution provider, you can see our first virtual showroom, the Pagani Immersive Experience Platform. &lt;a href="https://cloud.google.com/blog/topics/retail/virtual-automotive-showrooms-with-google-cloud-unreal-engine-and-nvidia"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Troubleshoot network connectivity with Dynamic Verification (public preview)&lt;/b&gt;—You can now check packet loss rate and one-way network latency between two VMs on GCP. This capability is an addition to existing Network Intelligence Center Connectivity Tests which verify reachability by analyzing network configuration in your VPCs. &lt;a href="https://cloud.google.com/network-intelligence-center/docs/connectivity-tests"&gt;See more in our documentation.&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Helping U.S. states get the COVID-19 vaccine to more people&lt;/b&gt;—In February, we announced our &lt;a href="https://cloud.google.com/blog/topics/public-sector/getting-vaccines-local-communities-safely-and-effectively"&gt;Intelligent Vaccine Impact solution (IVIs)&lt;/a&gt; to help communities rise to the challenge of getting vaccines to more people quickly and effectively. Many states have deployed IVIs, and have found it able to meet demand and easily integrate with their existing technology infrastructures. Google Cloud is proud to partner with a number of states across the U.S., including Arizona, the Commonwealth of Massachusetts, North Carolina, Oregon, and the Commonwealth of Virginia to support vaccination efforts at scale. &lt;a href="https://cloud.google.com/blog/topics/inside-google-cloud/how-google-cloud-is-supporting-covid-19-vaccination-efforts"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/developers-practitioners/picture-10-whiteboard-sketch-videos-bring-google-cloud-life/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/whiteboard.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Picture this: 10 whiteboard sketch videos that bring Google Cloud to life&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;If you’re looking for a visual way to learn Google Cloud products, we’ve got you covered. The Google Cloud Tech YouTube channel has a ser...&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Mar 15-Mar 19 2021&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;A2 VMs now GA: The largest GPU cloud instances with NVIDIA A100 GPUs&lt;/b&gt;—We’re announcing the general availability of A2 VMs based on the NVIDIA Ampere A100 Tensor Core GPUs in Compute Engine. This means customers around the world can now run their NVIDIA CUDA-enabled machine learning (ML) and high performance computing (HPC) scale-out and scale-up workloads more efficiently and at a lower cost. &lt;a href="https://cloud.google.com/blog/products/compute/a2-vms-with-nvidia-a100-gpus-are-ga"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Earn the new Google Kubernetes Engine skill badge for free&lt;/b&gt;—We’ve added a new &lt;a href="https://cloud.google.com/training/badges"&gt;skill badge&lt;/a&gt; this month, Optimize Costs for Google Kubernetes Engine (GKE), which you can earn for free when you sign up for the Kubernetes track of the &lt;a href="https://inthecloud.withgoogle.com/google-cloud-skills/register.html?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=FY21-Q1-global-trainingandenablement-website-other-skills_challenge&amp;amp;utm_content=gke_cost_optimization" target="_blank"&gt;skills challenge&lt;/a&gt;. The skills challenge provides 30 days free access to Google Cloud labs and gives you the opportunity to earn skill badges to showcase different cloud competencies to employers. &lt;a href="https://cloud.google.com/blog/topics/training-certifications/how-to-get-the-new-optimize-costs-for-google-kubernetes-engine-skills-badge"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Now available: carbon free energy percentages for our Google Cloud regions&lt;/b&gt;—Google first achieved carbon neutrality in 2007, and since 2017 we’ve purchased enough solar and wind energy to match 100% of our global electricity consumption. Now we’re building on that progress to target &lt;a href="https://blog.google/outreach-initiatives/sustainability/our-third-decade-climate-action-realizing-carbon-free-future/" target="_blank"&gt;a new sustainability goal&lt;/a&gt;: running our business on carbon-free energy 24/7, everywhere, by 2030. Beginning this week, we’re sharing data about how we are performing against that objective so our customers can select Google Cloud regions based on the carbon-free energy supplying them. &lt;a href="https://cloud.google.com/blog/topics/sustainability/sharing-carbon-free-energy-percentage-for-google-cloud-regions"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Increasing bandwidth to C2 and N2 VMs&lt;/b&gt;—We announced the public preview of 100, 75, and 50 Gbps high-bandwidth network configurations for &lt;a href="https://cloud.google.com/compute/docs/machine-types#n2_standard_machine_types"&gt;General Purpose N2&lt;/a&gt; and &lt;a href="https://cloud.google.com/compute/docs/machine-types#c2_machine_types"&gt;Compute Optimized C2&lt;/a&gt; Compute Engine VM families as part of &lt;a href="https://cloud.google.com/blog/products/networking/google-cloud-networking-in-depth-how-andromeda-2-2-enables-high-throughput-vms"&gt;continuous efforts to optimize our Andromeda host networking stack&lt;/a&gt;. This means we can now offer higher-bandwidth options on existing VM families when using the &lt;a href="https://cloud.google.com/compute/docs/networking/using-gvnic"&gt;Google Virtual NIC (gVNIC)&lt;/a&gt;. These VMs were previously limited to 32 Gbps. &lt;a href="https://cloud.google.com/blog/products/networking/increasing-bandwidth-to-c2-and-n2-vms"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;New research on how COVID-19 changed the nature of IT&lt;/b&gt;—To learn more about the impact of COVID-19 and the resulting implications to IT, Google commissioned a study by IDG to better understand how organizations are shifting their priorities in the wake of the pandemic. &lt;a href="https://cloud.google.com/blog/topics/research/new-research-covid-19s-surprising-acceleration-of-it-digital-transformation"&gt;Learn more&lt;/a&gt; and &lt;a href="https://inthecloud.withgoogle.com/it-leaders-research-21/overview-dl-cd.html?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=FY21-Q1-NORTHAM-demandgen-website-wd-gcp_gtm_reimagine_master&amp;amp;utm_content=homepage" target="_blank"&gt;download the report&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;New in API security&lt;/b&gt;—Google Cloud Apigee API management platform's latest release, &lt;a href="https://cloud.google.com/apigee"&gt;Apigee X&lt;/a&gt;, works with &lt;a href="https://cloud.google.com/armor"&gt;Cloud Armor&lt;/a&gt; to protect your APIs with advanced security technology including DDoS protection, geo-fencing, OAuth, and API keys. Learn more about our integrated security enhancements &lt;a href="https://cloud.google.com/blog/products/api-management/api-security-with-apigee-and-google-cloud-armor"&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Troubleshoot errors more quickly with Cloud Logging&lt;/b&gt;—The Logs Explorer now automatically breaks down your log results by severity, making it easy to spot spikes in errors at specific times. Learn more about our new histogram functionality &lt;a href="https://cloud.google.com/logging/docs/view/logs-viewer-interface#histogram-panel"&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Logs Explorer" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/logsexplorer.max-1000x1000.jpg"/&gt;&lt;figcaption class="article-image__caption "&gt;&lt;div class="rich-text"&gt;&lt;i&gt;The Logs Explorer histogram&lt;/i&gt;&lt;/div&gt;&lt;/figcaption&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Mar 8-Mar 12 2021&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Introducing #AskGoogleCloud on Twitter and YouTube—Our first segment on March 12th features Developer Advocates Stephanie Wong, Martin Omander and James Ward to answer questions on the best workloads for serverless, the differences between “serverless” and “cloud native,” how to accurately estimate costs for using Cloud Run, and much more. &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/introducing-askgooglecloud-community-driven-youtube-live-series"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;Learn about the value of no-code hackathons—Google Cloud’s no-code application development platform, AppSheet, helps to facilitate hackathons for “non-technical” employees with no coding necessary to compete. Learn about Globe Telecom’s no-code hackathon as well as their winning AppSheet app &lt;a href="https://cloud.google.com/blog/products/no-code-development/no-code-hackathons-with-google-clouds-appsheet"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;Introducing Cloud Code Secret Manager Integration—&lt;a href="https://cloud.google.com/secret-manager"&gt;Secret Manager&lt;/a&gt; provides a central place and single source of truth to manage, access, and audit secrets across Google Cloud. Integrating &lt;a href="https://cloud.google.com/code"&gt;Cloud Code&lt;/a&gt; with Secret Manager brings the powerful capabilities of both these tools together so you can create and manage your secrets right from within your preferred IDE, whether that be VS Code, IntelliJ, or Cloud Shell Editor. &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/introducing-cloud-code-secret-manager-integration"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;Flexible instance configurations in Cloud SQL—Cloud SQL for MySQL now supports flexible instance configurations which offer you the extra freedom to configure your instance with the specific number of vCPUs and GB of RAM that fits your workload. To set up a new instance with a flexible instance configuration, see our documentation &lt;a href="https://cloud.google.com/sql/docs/mysql/instance-settings#machine-type-2ndgen"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;The Cloud Healthcare Consent Management API is now generally available—The Healthcare Consent Management API is now GA, giving customers the ability to greatly scale the management of consents to meet increasing need, particularly amidst the &lt;a href="https://journals.sagepub.com/page/bds/collections/health_data_ecosystem" target="_blank"&gt;emerging task&lt;/a&gt; of managing health data for new care and research scenarios. &lt;a href="https://cloud.google.com/blog/topics/healthcare-life-sciences/google-cloud-healthcare-consent-management-api-generally-available"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/storage-data-transfer/a-peek-behind-colossus-googles-file-system/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/GCP_Storage_and_Data_Transfer.max-2800x2800.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Colossus under the hood: a peek into Google’s scalable storage system&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;An overview of Colossus, the file system that underpins Google Cloud’s storage offerings.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Mar 1-Mar 5 2021&lt;br/&gt;&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;Cloud Run is now available in all Google Cloud regions. &lt;a href="https://cloud.google.com/run/docs/locations"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;Introducing Apache Spark Structured Streaming connector for Pub/Sub Lite—We’re announcing the release of an open source &lt;a href="https://github.com/googleapis/java-pubsublite-spark" target="_blank"&gt;connector&lt;/a&gt; to read streams of messages from &lt;a href="https://cloud.google.com/pubsub/lite/docs"&gt;Pub/Sub Lite&lt;/a&gt; into Apache Spark.The connector &lt;a href="https://github.com/googleapis/java-pubsublite-spark#downloading-and-using-the-connector" target="_blank"&gt;works&lt;/a&gt; in all Apache Spark 2.4.X distributions, including Dataproc, Databricks, or manual Spark installations. &lt;a href="https://cloud.google.com/blog/products/data-analytics/pubsub-lite-for-your-streaming-applications"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;Google Cloud Next ‘21 is October 12-14, 2021—Join us and learn how the most successful companies have transformed their businesses with Google Cloud. Sign-up at &lt;a href="https://cloud.withgoogle.com/next/sf?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=FY21-Q1-Google-Cloud-Blog" target="_blank"&gt;g.co/cloudnext&lt;/a&gt; for updates. &lt;a href="https://cloud.google.com/blog/topics/google-cloud-next/announcing-google-cloud-next21"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;Hierarchical firewall policies now GA—Hierarchical firewalls provide a means to enforce firewall rules at the &lt;a href="https://cloud.google.com/resource-manager/docs/creating-managing-organization"&gt;organization&lt;/a&gt; and &lt;a href="https://cloud.google.com/resource-manager/docs/creating-managing-folders"&gt;folder&lt;/a&gt; levels in the &lt;a href="https://cloud.google.com/resource-manager/docs/cloud-platform-resource-hierarchy"&gt;GCP Resource Hierarchy&lt;/a&gt;. This allows security administrators at different levels in the hierarchy to define and deploy consistent firewall rules across a number of projects so they're applied to all VMs in currently existing and yet-to-be-created projects. &lt;a href="https://cloud.google.com/blog/products/identity-security/new-google-cloud-hierarchical-firewall-policies"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;Announcing the Google Cloud Born-Digital Summit—Over this half-day event, we’ll highlight proven best-practice approaches to data, architecture, diversity &amp;amp; inclusion, and growth with Google Cloud solutions. &lt;a href="https://cloud.google.com/blog/topics/events/announcing-the-google-cloud-born-digital-summit"&gt;Learn more&lt;/a&gt; and &lt;a href="https://cloudonair.withgoogle.com/events/born-digital-summit?utm_source=Google&amp;amp;utm_medium=Social&amp;amp;utm_content=BlogPostAMERICAS&amp;amp;utm_campaign=" target="_blank"&gt;register for free&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;Google Cloud products in 4 words or less (2021 edition)—Our popular “4 words or less Google Cloud developer’s cheat sheet” is back and updated for 2021. &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/back-popular-demand-google-cloud-products-4-words-or-less-2021-edition"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;Gartner names Google a leader in its 2021 Magic Quadrant for Cloud AI Developer Services report—We believe this recognition is based on Gartner’s evaluation of Google Cloud’s language, vision, conversational, and structured data services and solutions for developers. &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/google-cloud-a-leader-in-gartner-mq-for-cloud-ai-developer-services"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;Announcing the Risk Protection Program—The Risk Protection Program offers customers peace of mind through the technology to secure their data, the tools to monitor the security of that data, and an industry-first cyber policy offered by leading insurers. &lt;a href="https://cloud.google.com/blog/products/identity-security/google-cloud-risk-protection-program-now-in-preview"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Building the future of work—We’re introducing new innovations in Google Workspace to help people collaborate and find more time and focus, wherever and however they work. &lt;a href="https://cloud.google.com/blog/products/workspace/the-future-of-work-with-google-workspace"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Assured Controls and expanded Data Regions—We’ve added new information governance features in Google Workspace to help customers control their data based on their business goals. &lt;a href="https://cloud.google.com/blog/products/workspace/assured-controls-and-expanded-data-regions-for-google-workspace"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/workspace/gmail-meet-and-slides-tips-from-googles-productivity-expert-in-2021/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/workspace_B6u1Eaw.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;11 quick tips for making the most of Gmail, Meet, Calendar, and more in Google Workspace&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Whether you’re looking to stay on top of your inbox or make the most of virtual meetings, most of us can benefit from quick productivity ...&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Feb 22-Feb 26 2021&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;21 Google Cloud tools explained in 2 minutes—Need a quick overview of Google Cloud core technologies? Quickly learn these 21 Google Cloud products—each explained in under two minutes. &lt;a href="https://cloud.google.com/blog/topics/inside-google-cloud/21-google-cloud-tools-each-explained-under-2-minutes"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;BigQuery materialized views now GA—Materialized views (MV’s) are precomputed views that periodically cache results of a query to provide customers increased performance and efficiency. &lt;a href="https://cloud.google.com/blog/products/data-analytics/bigquery-materialized-views-now-ga"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;New in BigQuery BI Engine—We’re extending BigQuery BI Engine to work with any BI or custom dashboarding applications that require sub-second query response times. In this preview, BI Engine will work seamlessly with Looker and other popular BI tools such as Tableau and Power BI without requiring any change to the BI tools. &lt;a href="https://cloud.google.com/blog/products/data-analytics/bigquery-bi-engine-extends-integration-for-any-bi-solution"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Dataproc now supports Shielded VMs—All Dataproc clusters created using Debian 10 or Ubuntu 18.04 operating systems now use &lt;a href="https://cloud.google.com/security/shielded-cloud/shielded-vm"&gt;Shielded VMs&lt;/a&gt; by default and customers can provide their own &lt;a href="https://cloud.google.com/dataproc/docs/reference/rest/v1/ClusterConfig#ShieldedInstanceConfig"&gt;configurations&lt;/a&gt; for secure boot, vTPM, and Integrity Monitoring. This feature is just one of the many ways customers that have migrated their Hadoop and Spark clusters to GCP experience continued improvements to their security postures without any additional cost.&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;New Cloud Security Podcast by Google—Our new podcast brings you stories and insights on security in the cloud, delivering security from the cloud, and, of course, on what we’re doing at Google Cloud to help keep customer data safe and workloads secure. &lt;a href="https://cloud.google.com/blog/products/identity-security/introducing-google-clouds-new-cloud-security-podcast"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;New in Conversational AI and Apigee technology—Australian retailer Woolworths provides seamless customer experiences with their virtual agent, Olive. Apigee API Management and Dialogflow technology allows customers to talk to Olive through voice and chat. &lt;a href="https://cloud.google.com/blog/products/api-management/conversational-ai-with-dialogflow-and-apigee-api-management"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Introducing GKE Autopilot—GKE already offers an industry-leading level of automation that makes setting up and operating a Kubernetes cluster easier and more cost effective than do-it-yourself and other managed offerings. Autopilot represents a significant leap forward. In addition to the fully managed control plane that GKE has always provided, using the Autopilot mode of operation automatically applies industry best practices and can eliminate all node management operations, maximizing your cluster efficiency and helping to provide a stronger security posture. &lt;a href="https://cloud.google.com/blog/products/containers-kubernetes/introducing-gke-autopilot"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Partnering with Intel to accelerate cloud-native 5G—As we continue to grow cloud-native services for the telecommunications industry, we’re excited to announce a collaboration with Intel to develop reference architectures and integrated solutions for communications service providers to accelerate their deployment of 5G and edge network solutions. &lt;a href="https://cloud.google.com/blog/topics/partners/speeding-up-cloud-native-5g-networks-with-intel-and-google-cloud"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Veeam Backup for Google Cloud now available—&lt;a href="https://www.veeam.com/blog/google-cloud-backup-and-recovery.html" target="_blank"&gt;Veeam Backup for Google Cloud&lt;/a&gt; automates Google-native snapshots to securely protect VMs across projects and regions with ultra-low RPOs and RTOs, and store backups in Google Object Storage to enhance data protection while ensuring lower costs for long-term retention.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Migrate for Anthos 1.6 GA—With&lt;a href="https://cloud.google.com/migrate/anthos/docs"&gt; Migrate for Anthos&lt;/a&gt;, customers and partners can automatically migrate and modernize traditional application workloads running in VMs into containers running on Anthos or GKE. Included in this new release: &lt;/p&gt;&lt;/li&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;In-place modernization for Anthos on AWS (Public Preview) to help customers accelerate on-boarding to Anthos AWS while leveraging their existing investment in AWS data sources, projects, VPCs, and IAM controls.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Additional Docker registries and artifacts repositories support (GA) including AWS ECR, basic-auth docker registries, and AWS S3 storage to provide further flexibility for customers using Anthos Anywhere (on-prem, AWS, etc). &lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;HTTPS Proxy support (GA) to enable M4A functionality (access to external image repos and other services) where a proxy is used to control external access.&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/devops-sre/5-google-sre-resources-to-get-started/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/DevOps_BlogHeader_A_Rnd3.max-2800x2800.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;5 resources to help you get started with SRE&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Here are our top five Google Cloud resources for getting started on your SRE journey.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Feb 15-Feb 19 2021&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;Introducing Cloud Domains in preview—Cloud Domains simplify domain registration and management within Google Cloud, improve the custom domain experience for developers, increase security, and support stronger integrations around DNS and SSL. &lt;a href="https://cloud.google.com/blog/products/networking/introducing-cloud-domains"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Announcing Databricks on Google Cloud—Our partnership with Databricks enables customers to accelerate Databricks implementations by simplifying their data access, by jointly giving them powerful ways to analyze their data, and by leveraging our combined AI and ML capabilities to impact business outcomes. &lt;a href="https://cloud.google.com/blog/products/data-analytics/databricks-on-google-cloud"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Service Directory is GA—As the number and diversity of services grows, it becomes increasingly challenging to maintain an inventory of all of the services across an organization. Last year, we launched Service Directory to help simplify the problem of service management. Today, it’s generally available. &lt;a href="https://cloud.google.com/blog/products/networking/service-directory-is-ga-and-adds-automatic-service-registration"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/chrome-enterprise/16-posts-on-chrome-browser-best-practices-for-admins/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/chrome_browser.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Optimize your browser deployment: Links to our most popular Chrome Insider posts&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Find all the posts in our Chrome Insider blog series so you can read them all in one place or bookmark them for later.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Feb 8-Feb 12 2021&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;Introducing Bare Metal Solution for SAP workloads—We’ve expanded our Bare Metal Solution—dedicated, single-tenant systems designed specifically to run workloads that are too large or otherwise unsuitable for standard, virtualized environments—to include SAP-certified hardware options, giving SAP customers great options for modernizing their biggest and most challenging workloads. &lt;a href="https://cloud.google.com/blog/products/sap-google-cloud/introducing-bare-metal-solution-for-sap-workloads"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;9TB SSDs bring ultimate IOPS/$ to Compute Engine VMs—You can now attach 6TB and 9TB Local SSD to second-generation general-purpose &lt;a href="https://cloud.google.com/compute/docs/machine-types"&gt;N2 Compute Engine VMs&lt;/a&gt;, for great IOPS per dollar. &lt;a href="https://cloud.google.com/blog/products/storage-data-transfer/n2-vms-run-low-latency-io-intensive-workloads-with-9tb-ssd"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Supporting the Python ecosystem—As part of our longstanding support for the Python ecosystem, we are happy to increase our support for the &lt;a href="https://www.python.org/psf/" target="_blank"&gt;Python Software Foundation&lt;/a&gt;, the non-profit behind the Python programming language, ecosystem and community. &lt;a href="https://cloud.google.com/blog/products/open-source/supporting-the-python-ecosystem"&gt;Learn more&lt;/a&gt;. &lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Migrate to regional backend services for Network Load Balancing—We now support backend services with Network Load Balancing—a significant enhancement over the prior approach, target pools, providing a common unified data model for all our load-balancing family members and accelerating the delivery of exciting features on Network Load Balancing. &lt;a href="https://cloud.google.com/blog/products/networking/migrate-to-regional-backend-services-for-network-load-balancing"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/developers-practitioners/giant-list-google-cloud-resources/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/Google_Cloud_LastMonthToday.max-500x500.png')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;A giant list of Google Cloud resources&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;The growth of Google Cloud has been staggering. I decided to invest some time in building you a comprehensive list of resources.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Feb 1-Feb 4 2021&lt;br/&gt;&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Apigee launches Apigee X—Apigee celebrates its 10 year anniversary with Apigee X, a new release of the Apigee API management platform. Apigee X harnesses the best of Google technologies to accelerate and globalize your API-powered digital initiatives. Learn more about Apigee X and digital excellence &lt;a href="https://cloud.google.com/blog/products/api-management/apigee-x-google-clouds-more-powerful-api-management-platform"&gt;here.&lt;/a&gt;&lt;/li&gt;&lt;li&gt;Celebrating the success of Black founders with Google Cloud during Black History Month—February is Black History Month, a time for us to come together to celebrate and remember the important people and history of the African heritage. Over the next four weeks, we will highlight four Black-led startups and how they use Google Cloud to grow their businesses. Our &lt;a href="https://cloud.google.com/blog/topics/startups/black-history-month-celebrating-success-black-founders-google-cloud"&gt;first feature &lt;/a&gt;highlights TQIntelligence and its founder, Yared.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/service-mesh-era/service-mesh-era-all-posts-our-istio-best-practices-blog-series/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/Kub_Istio_05JVI6I.max-500x500.png')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;The Service Mesh era: All the posts in our best practices blog series&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Find all the posts in our Service Mesh Era blog series in one convenient location—to read now or bookmark for later.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Week of Jan 25-Jan 29 2021&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;BeyondCorp Enterprise now generally available—BeyondCorp Enterprise is a zero trust solution, built on Google’s global network, which provides customers with simple and secure access to applications and cloud resources and offers integrated threat and data protection. To learn more, read the &lt;a href="https://cloud.google.com/blog/products/identity-security/introducing-beyondcorp-enterprise"&gt;blog post&lt;/a&gt;, visit our &lt;a href="https://cloud.google.com/beyondcorp-enterprise"&gt;product homepage,&lt;/a&gt; and register for our upcoming &lt;a href="https://www.brighttalk.com/webcast/18282/465874?utm_source=Google+Cloud+Security&amp;amp;utm_medium=brighttalk&amp;amp;utm_campaign=465874" target="_blank"&gt;webinar&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/databases/6-database-trends-to-watch/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/6_database_trend.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;6 database trends to watch&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Using managed cloud database services like Cloud SQL, Spanner, and more, can bring performance, scale, and more. See what’s next for mode...&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;&lt;/h3&gt;&lt;h3&gt;&lt;/h3&gt;&lt;h3&gt;&lt;/h3&gt;&lt;h3&gt;Week of Jan 18-Jan 22 2021&lt;br/&gt;&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Cloud Operations Sandbox now available&lt;/b&gt;—Cloud Operations Sandbox is an open-source tool that helps you learn SRE practices from Google and apply them on cloud services using Google Cloud’s operations suite (formerly Stackdriver), with everything you need to get started in one click. You can read our &lt;a href="https://cloud.google.com/blog/products/operations/on-the-road-to-sre-with-cloud-operations-sandbox"&gt;blog post&lt;/a&gt;, or get started by visiting &lt;a href="http://cloud-ops-sandbox.dev/" target="_blank"&gt;cloud-ops-sandbox.dev&lt;/a&gt;, exploring the &lt;a href="https://github.com/GoogleCloudPlatform/cloud-ops-sandbox" target="_blank"&gt;project repo&lt;/a&gt;, and following along in the &lt;a href="https://github.com/GoogleCloudPlatform/cloud-ops-sandbox/blob/master/docs/README.md" target="_blank"&gt;user guide&lt;/a&gt;. &lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;New data security strategy whitepaper&lt;/b&gt;—Our new whitepaper shares our best practices for how to deploy a modern and effective data security program in the cloud. Read the &lt;a href="https://cloud.google.com/blog/products/identity-security/start-a-data-security-program-in-a-cloud-native-way-on-google-cloud"&gt;blog post&lt;/a&gt; or download the &lt;a href="https://services.google.com/fh/files/misc/designing_and_deploying_data_security_strategy.pdf" target="_blank"&gt;paper&lt;/a&gt;.   &lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;WebSockets, HTTP/2 and gRPC bidirectional streams come to Cloud Run&lt;/b&gt;—With these capabilities, you can deploy new kinds of applications to Cloud Run that were not previously supported, while taking advantage of serverless infrastructure. These features are now available in public preview for all Cloud Run locations. Read the &lt;a href="https://cloud.google.com/blog/products/serverless/cloud-run-gets-websockets-http-2-and-grpc-bidirectional-streams"&gt;blog post&lt;/a&gt; or check out the &lt;a href="https://deploy.cloud.run/?git_repo=https://github.com/socketio/socket.io.git&amp;amp;dir=examples/whiteboard&amp;amp;revision=f8a66fd11acffb72fcb90750affd5dce42bef977" target="_blank"&gt;WebSockets demo app&lt;/a&gt; or the &lt;a href="https://github.com/GoogleCloudPlatform/golang-samples/tree/master/run/h2c" target="_blank"&gt;sample h2c server&lt;/a&gt; app.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;New tutorial: Build a no-code workout app in 5 steps&lt;/b&gt;—Looking to crush your new year’s resolutions? Using AppSheet, Google Cloud’s no-code app development platform, you can build a custom fitness app that can do things like record your sets, reps and weights, log your workouts, and show you how you’re progressing. &lt;a href="https://cloud.google.com/blog/products/no-code-development/build-your-own-workout-app-5-stepswithout-coding"&gt;Learn how&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;h3&gt;&lt;hr/&gt;&lt;/h3&gt;&lt;h3&gt;Week of Jan 11-Jan 15 2021&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;State of API Economy 2021 Report now available&lt;/b&gt;—Google Cloud details the changing role of APIs in 2020 amidst the COVID-19 pandemic, informed by a comprehensive study of Apigee API usage behavior across industry, geography, enterprise size, and more. Discover these 2020 trends along with a projection of what to expect from APIs in 2021. Read our blog post &lt;a href="https://cloud.google.com/blog/products/api-management/top-5-trends-for-api-powered-digital-transformation-in-2021"&gt;here&lt;/a&gt; or download and read the report &lt;a href="https://pages.apigee.com/api-economy-report-register/?utm_source=social&amp;amp;utm_medium=organic&amp;amp;utm_campaign=api-economy" target="_blank"&gt;here.&lt;/a&gt;&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;New in the state of no-code&lt;/b&gt;—Google Cloud's AppSheet looks back at the key no-code application development themes of 2020. AppSheet contends the rising number of citizen developer app creators will ultimately change the state of no-code in 2021. Read more &lt;a href="https://cloud.google.com/blog/products/no-code-development/no-code-momentum-in-2020"&gt;here.&lt;/a&gt;&lt;br/&gt;&lt;/li&gt;&lt;/ul&gt;&lt;h3&gt;&lt;hr/&gt;&lt;/h3&gt;&lt;h3&gt;Week of Jan 4-Jan 8 2021&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Last year's most popular API posts&lt;/b&gt;—In an arduous year, thoughtful API design and strategy is critical to empowering developers and companies to use technology for global good. Google Cloud looks back at the must-read API posts in 2020. Read it &lt;a href="https://cloud.google.com/blog/products/api-management/google-cloud-2020-api-and-api-management-highlights"&gt;here.&lt;/a&gt;&lt;/li&gt;&lt;/ul&gt;&lt;b&gt;&lt;h3&gt;&lt;hr/&gt;&lt;/h3&gt;&lt;h3&gt;Week of Dec 21-Dec 25 2020&lt;/h3&gt;&lt;/b&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;A look back at the year across Google Cloud&lt;/b&gt;—Looking for some holiday reading? We’ve published recaps of our year across &lt;a href="https://cloud.google.com/blog/products/databases/managed-cloud-database-services-what-was-new-in-2020"&gt;databases&lt;/a&gt;, &lt;a href="https://cloud.google.com/blog/products/serverless/google-cloud-serverless-and-app-dev-services-in-2020"&gt;serverless&lt;/a&gt;, &lt;a href="https://cloud.google.com/blog/products/data-analytics/data-analytics-at-google-cloud-what-was-new-in-2020"&gt;data analytics&lt;/a&gt;, and &lt;a href="https://cloud.google.com/blog/products/no-code-development/2020-no-code-app-development-retrospective"&gt;no-code development&lt;/a&gt;. Or take a look at our &lt;a href="https://cloud.google.com/blog/topics/inside-google-cloud/top-9-posts-from-google-cloud-in-2020"&gt;9 most popular posts of 2020&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;/p&gt;&lt;h3&gt;&lt;hr/&gt;&lt;/h3&gt;&lt;h3&gt;Week of Dec 14-Dec 18 2020&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Memorystore for Redis enables TLS encryption support (Preview)&lt;/b&gt;—With this release, you can now use Memorystore for applications requiring sensitive data to be encrypted between the client and the Memorystore instance. Read more &lt;a href="https://cloud.google.com/memorystore/docs/redis/tls-encryption-overview"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Monitoring Query Language (MQL) for Cloud Monitoring is now generally available&lt;/b&gt;—Monitoring Query language provides developers and operators on IT and development teams powerful metric querying, analysis, charting, and alerting capabilities. This functionality is needed for Monitoring use cases that include troubleshooting outages, root cause analysis, custom SLI / SLO creation, reporting and analytics, complex alert logic, and more. &lt;a href="https://cloud.google.com/monitoring/mql"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;b&gt;&lt;h3&gt;&lt;hr/&gt;&lt;/h3&gt;&lt;h3&gt;Week of Dec 7-Dec 11 2020&lt;/h3&gt;&lt;/b&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Memorystore for Redis now supports Redis AUTH&lt;/b&gt;—With this release you can now use OSS Redis AUTH feature with Memorystore for Redis instances. Read more &lt;a href="https://cloud.google.com/memorystore/docs/redis/auth-overview"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;New in serverless computing&lt;/b&gt;—Google Cloud API Gateway and its service-first approach to developing serverless APIs helps organizations accelerate innovation by eliminating scalability and security bottlenecks for their APIs. Discover more benefits &lt;a href="https://cloud.google.com/blog/products/serverless/build-restful-apis-on-serverless-platforms-like-cloud-run"&gt;here.&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Environmental Dynamics, Inc. makes a big move to no-code&lt;/b&gt;—The environmental consulting company EDI built and deployed 35+ business apps with no coding skills necessary with Google Cloud’s AppSheet. This no-code effort not only empowered field workers, but also saved employees over 2,550 hours a year. Get the full story &lt;a href="https://cloud.google.com/blog/products/workspace/google-workspace-and-appsheet-help-consultant-edi-with-productivity"&gt;here.&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Introducing Google Workspace for Government&lt;/b&gt;—Google Workspace for Government is an offering that brings the best of Google Cloud’s collaboration and communication tools to the government with pricing that meets the needs of the public sector. Whether it’s powering social care visits, employment support, or virtual courts, Google Workspace helps governments meet the unique challenges they face as they work to provide better services in an increasingly virtual world. &lt;a href="https://cloud.google.com/blog/topics/public-sector/google-cloud-unveils-new-offerings-its-global-public-sector-summit"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;/p&gt;&lt;h3&gt;&lt;hr/&gt;&lt;/h3&gt;&lt;h3&gt;Week of Nov 30-Dec 4 2020&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Google enters agreement to acquire Actifio&lt;/b&gt;—Actifio, a leader in backup and disaster recovery (DR), offers customers the opportunity to protect virtual copies of data in their native format, manage these copies throughout their entire lifecycle, and use these copies for scenarios like development and test. This planned acquisition further demonstrates Google Cloud’s commitment to helping enterprises protect workloads on-premises and in the cloud. &lt;a href="https://cloud.google.com/blog/products/storage-data-transfer/google-enters-agreement-to-acquire-actifio"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Traffic Director can now send traffic to services and gateways hosted outside of Google Cloud&lt;/b&gt;—Traffic Director support for Hybrid Connectivity &lt;a href="https://cloud.google.com/load-balancing/docs/negs"&gt;Network Endpoint Groups&lt;/a&gt; (NEGs), now generally available, enables services in your VPC network to interoperate more seamlessly with services in other environments. It also enables you to build advanced solutions based on Google Cloud's portfolio of networking products, such as Cloud Armor protection for your private on-prem services. &lt;a href="https://cloud.google.com/blog/products/networking/traffic-director-extends-its-service-mesh-to-non-gcp-endpoints"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Google Cloud launches the Healthcare Interoperability Readiness Program&lt;/b&gt;—This program, powered by APIs and Google Cloud’s Apigee, helps patients, doctors, researchers, and healthcare technologists alike by making patient data and healthcare data more accessible and secure. Learn more &lt;a href="https://cloud.google.com/blog/topics/healthcare-life-sciences/google-cloud-healthcare-interoperability-readiness-program"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Container Threat Detection in Security Command Center&lt;/b&gt;—We announced the general availability of Container Threat Detection, a built-in service in Security Command Center. This release includes multiple detection capabilities to help you monitor and secure your container deployments in Google Cloud. Read more &lt;a href="https://cloud.google.com/blog/products/identity-security/container-threat-detection-is-ga"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Anthos on bare metal now GA&lt;/b&gt;—Anthos on bare metal opens up new possibilities for how you run your workloads, and where. You can run Anthos on your existing virtualized infrastructure, or eliminate the dependency on a hypervisor layer to modernize applications while reducing costs. &lt;a href="https://cloud.google.com/blog/topics/hybrid-cloud/anthos-on-bare-metal-is-now-ga"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;/p&gt;&lt;h3&gt;&lt;hr/&gt;&lt;/h3&gt;&lt;h3&gt;Week of Nov 23-27 2020&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Tuning control support in Cloud SQL for MySQL&lt;/b&gt;—We’ve made all 80 flags that were previously in preview now generally available (GA), empowering you with the controls you need to optimize your databases. See the full list &lt;a href="https://cloud.google.com/sql/docs/mysql/flags#list-flags-mysql"&gt;here&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;New in BigQuery ML&lt;/b&gt;—We announced the general availability of boosted trees using XGBoost, deep neural networks (DNNs) using TensorFlow, and model export for online prediction. &lt;a href="https://cloud.google.com/blog/products/data-analytics/new-ml-models-built-into-cloud-data-analytics"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;New AI/ML in retail report&lt;/b&gt;—We recently commissioned a survey of global retail executives to better understand which AI/ML use cases across the retail value chain drive the highest value and returns in retail, and what retailers need to keep in mind when going after these opportunities. &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/top-10-ai-use-cases-in-retail"&gt;Learn more&lt;/a&gt;  or &lt;a href="https://inthecloud.withgoogle.com/retail-whitepaper/dl-cd.html" target="_blank"&gt;read the report&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;/p&gt;&lt;h3&gt;&lt;hr/&gt;&lt;/h3&gt;&lt;h3&gt;Week of Nov 16-20 2020&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;New whitepaper on how AI helps the patent industry&lt;/b&gt;—Our new paper outlines a methodology to train a BERT (bidirectional encoder representation from transformers) model on over 100 million patent publications from the U.S. and other countries using open-source tooling. &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/how-ai-improves-patent-analysis"&gt;Learn more&lt;/a&gt; or &lt;a href="https://services.google.com/fh/files/blogs/bert_for_patents_white_paper.pdf" target="_blank"&gt;read the whitepaper&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Google Cloud support for .NET 5.0&lt;/b&gt;—Learn more about our support of .NET 5.0, as well as &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/net-50-google-cloud"&gt;how to deploy it to Cloud Run&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;.NET Core 3.1 now on Cloud Functions&lt;/b&gt;—With this integration you can write cloud functions using your favorite .NET Core 3.1 runtime with our Functions Framework for .NET for an idiomatic developer experience. &lt;a href="https://cloud.google.com/blog/products/application-development/introducing-net-google-cloud-functions"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Filestore Backups in preview&lt;/b&gt;—We announced the availability of the Filestore Backups preview in all regions, making it easier to migrate your business continuity, disaster recovery and backup strategy for your file systems in Google Cloud. &lt;a href="https://cloud.google.com/blog/products/storage-data-transfer/introducing-filestore-backups"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Introducing Voucher, a service to help secure the container supply chain&lt;/b&gt;—Developed by the Software Supply Chain Security team at Shopify to work with Google Cloud tools, Voucher evaluates container images created by CI/CD pipelines and signs those images if they meet certain predefined security criteria. Binary Authorization then validates these signatures at deploy time, ensuring that only explicitly authorized code that meets your organizational policy and compliance requirements can be deployed to production. &lt;a href="https://cloud.google.com/blog/products/devops-sre/introducing-voucher-service-help-secure-container-supply-chain"&gt;Learn more&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;10 most watched from Google Cloud Next ‘20: OnAir&lt;/b&gt;—Take a stroll through the 10 sessions that were most popular from Next OnAir, covering everything from data analytics to cloud migration to no-code development. &lt;a href="https://cloud.google.com/blog/topics/google-cloud-next/10-most-popular-sessions-from-google-cloud-next20-onair"&gt;Read the blog&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Artifact Registry is now GA&lt;/b&gt;—With support for container images, Maven, npm packages, and additional formats coming soon, Artifact Registry helps your organization benefit from scale, security, and standardization across your software supply chain. &lt;a href="https://cloud.google.com/blog/products/devops-sre/artifact-registry-is-ga"&gt;Read the blog&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;h3&gt;&lt;hr/&gt;&lt;/h3&gt;&lt;h3&gt;Week of Nov 9-13 2020&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Introducing the Anthos Developer Sandbox&lt;/b&gt;—The Anthos Developer Sandbox gives you an easy way to learn to develop on Anthos at no cost, available to anyone with a Google account. &lt;a href="https://cloud.google.com/blog/topics/anthos/introducing-the-anthos-developer-sandbox"&gt;Read the blog&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Database Migration Service now available in preview&lt;/b&gt;—Database Migration Service (DMS) makes migrations to Cloud SQL simple and reliable. DMS supports migrations of self-hosted MySQL databases—either on-premises or in the cloud, as well as managed databases from other clouds—to Cloud SQL for MySQL. Support for PostgreSQL is currently available for limited customers in preview, with SQL Server coming soon. &lt;a href="https://cloud.google.com/blog/products/databases/database-migration-service-now-available-for-cloud-sql-and-more"&gt;Learn more&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Troubleshoot deployments or production issues more quickly with new logs tailing&lt;/b&gt;—We’ve added support for a new API to tail logs with low latency. Using gcloud, it allows you the convenience of tail -f with the powerful query language and centralized logging solution of Cloud Logging. &lt;a href="https://cloud.google.com/logging/docs/reference/tools/gcloud-logging#live-tailing"&gt;Learn more&lt;/a&gt; about this preview feature.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Regionalized log storage now available in 5 new regions in preview&lt;/b&gt;—You can now select where your logs are stored from one of five regions in addition to global—asia-east1, europe-west1, us-central1, us-east1, and us-west1. When you create a logs bucket, you can set the region in which you want to store your logs data. Get started with this &lt;a href="https://cloud.google.com/logging/docs/regionalized-logs"&gt;guide&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;/p&gt;&lt;h3&gt;&lt;hr/&gt;&lt;/h3&gt;&lt;h3&gt;Week of Nov 2-6 2020&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Cloud SQL adds support for PostgreSQL 13&lt;/b&gt;—Shortly after its community GA, Cloud SQL has added support for PostgreSQL 13. You get access to the latest features of PostgreSQL while Cloud SQL handles the heavy operational lifting, so your team can focus on accelerating application delivery. Read more &lt;a href="https://cloud.google.com/blog/products/databases/cloud-sql-database-service-adds-postgresql-13"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Apigee creates value for businesses running on SAP&lt;/b&gt;—Google Cloud’s API Management platform Apigee is optimized for data insights and data monetization, helping businesses running on SAP innovate faster without fear of SAP-specific challenges to modernization. Read more &lt;a href="https://cloud.google.com/blog/products/api-management/how-google-cloud-lets-you-do-more-with-sap"&gt;here.&lt;/a&gt; &lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Document AI platform is live&lt;/b&gt;—The new Document AI (DocAI) platform, a unified console for document processing, is now available in preview. You can quickly access all parsers, tools and solutions (e.g. &lt;a href="https://cloud.google.com/solutions/lending-doc-ai"&gt;Lending DocAI&lt;/a&gt;, &lt;a href="https://cloud.google.com/solutions/procurement-doc-ai"&gt;Procurement DocAI&lt;/a&gt;) with a unified API, enabling an end-to-end document solution from evaluation to deployment. Read the full story &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/google-cloud-announces-document-ai-platform"&gt;here&lt;/a&gt; or check it out in your &lt;a href="https://console.cloud.google.com/ai/document-ai?_ga=2.29960255.-1847873956.1563554776"&gt;Google Cloud&lt;/a&gt;console.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Accelerating data migration with Transfer Appliances TA40 and TA300&lt;/b&gt;—We’re announcing the general availability of new Transfer Appliances. Customers are looking for fast, secure and easy to use options to migrate their workloads to Google Cloud and we are addressing their needs with &lt;a href="https://youtu.be/4U3tn_0mX_c?t=1003" target="_blank"&gt;next generation Transfer Appliances&lt;/a&gt;. Learn more about &lt;a href="https://cloud.google.com/transfer-appliance/docs/4.0"&gt;Transfer Appliances TA40 and TA300&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;/p&gt;&lt;h3&gt;&lt;hr/&gt;&lt;/h3&gt;&lt;h3&gt;Week of Oct 26-30 2020&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;B.H., Inc. accelerates digital transformation&lt;/b&gt;—The Utah based contracting and construction company BHI eliminated IT backlog when non technical employees were empowered to build equipment inspection, productivity, and other custom apps by choosing Google Workspace and the no-code app development platform, AppSheet. Read the full story &lt;a href="https://cloud.google.com/blog/products/no-code-development/digital-transformation-with-google-workspace-and-appsheet"&gt;here&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Globe Telecom embraces no-code development&lt;/b&gt;—Google Cloud’s AppSheet empowers Globe Telecom employees to do more innovating with less code. The global communications company kickstarted their no-code journey by combining the power of AppSheet with a unique adoption strategy. As a result, AppSheet helped Globe Telecom employees build 59 business apps in just 8 weeks. &lt;a href="https://www.youtube.com/watch?v=L0saoOdF_Nw&amp;amp;feature=youtu.be" target="_blank"&gt;Get the full story&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Cloud Logging now allows you to control access to logs via Log Views&lt;/b&gt;—Building on the control offered via&lt;a href="https://cloud.google.com/logging/docs/buckets"&gt; Log Buckets&lt;/a&gt; (&lt;a href="https://cloud.google.com/blog/products/management-tools/cloud-logging-adds-log-buckets-feature"&gt;blog post&lt;/a&gt;), you can now configure who has access to logs based on the source project, resource type, or log name, all using standard IAM controls. Logs views, currently in Preview, can help you build a system using the principle of least privilege, limiting sensitive logs to only users who need this information.&lt;a href="https://cloud.google.com/logging/docs/logs-views"&gt; Learn more about Log Views&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Document AI is HIPAA compliant&lt;/b&gt;—&lt;a href="https://cloud.google.com/document-ai"&gt;Document AI&lt;/a&gt; now enables HIPAA compliance. Now Healthcare and Life Science customers such as health care providers, health plans, and life science organizations can unlock insights by quickly extracting structured data from medical documents while safeguarding individuals’ protected health information (PHI). &lt;a href="https://cloud.google.com/security/compliance/hipaa-compliance/"&gt;Learn more&lt;/a&gt; about Google Cloud’s nearly 100 products that support HIPAA-compliance.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;/p&gt;&lt;h3&gt;&lt;hr/&gt;&lt;/h3&gt;&lt;h3&gt;Week of Oct 19-23 2020&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Improved security and governance in Cloud SQL for PostgreSQL&lt;/b&gt;—Cloud SQL for PostgreSQL now integrates with &lt;a href="https://cloud.google.com/iam/docs/overview"&gt;Cloud IAM&lt;/a&gt; (preview) to provide simplified and consistent authentication and authorization. Cloud SQL has also enabled &lt;a href="https://cloud.google.com/sql/docs/postgres/pg-audit"&gt;PostgreSQL Audit Extension&lt;/a&gt; (preview) for more granular audit logging. &lt;a href="https://cloud.google.com/blog/products/databases/securely-managed-cloud-sql-databases"&gt;Read the blog&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Announcing the AI in Financial Crime Compliance webinar&lt;/b&gt;—Our executive digital forum will feature industry executives, academics, and former regulators who will discuss how AI is transforming financial crime compliance on November 17. &lt;a href="https://cloudonair.withgoogle.com/events/financial_crime_compliance?utm_source=google&amp;amp;utm_medium=blogpost&amp;amp;utm_campaign=&amp;amp;utm_content=blogpost" target="_blank"&gt;Register now.&lt;/a&gt;&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Transforming retail with AI/ML&lt;/b&gt;—New research provides insights on high value AI/ML use cases for food, drug, mass merchant and speciality retail that can drive significant value and build resilience for your business. Learn what the top use cases are for your sub-segment and read real world success stories. Download the &lt;a href="https://inthecloud.withgoogle.com/retail-whitepaper/dl-cd.html" target="_blank"&gt;ebook here&lt;/a&gt; and view this companion &lt;a href="https://cloudonair.withgoogle.com/events/transforming-retail-with-ai-20" target="_blank"&gt;webinar&lt;/a&gt; which also features insights from Zulily. &lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;New release of Migrate for Anthos&lt;/b&gt;—We’re introducing two important new capabilities in the 1.5 release of &lt;a href="https://cloud.google.com/migrate/anthos"&gt;Migrate for Anthos&lt;/a&gt;, Google Cloud's solution to easily migrate and modernize applications currently running on VMs so that they instead run on containers in Google Kubernetes Engine or Anthos. The first is GA support for modernizing IIS apps running on Windows Server VMs. The second is a new utility that helps you identify which VMs in your existing environment are the best targets for modernization to containers. &lt;a href="https://pantheon.corp.google.com/kubernetes/migrate" target="_blank"&gt;Start migrating&lt;/a&gt; or check out the assessment tool documentation (&lt;a href="https://cloud.google.com/migrate/anthos/docs/linux-assessment-tool"&gt;Linux&lt;/a&gt; | &lt;a href="https://cloud.google.com/migrate/anthos/docs/windows-assessment-tool"&gt;Windows&lt;/a&gt;). &lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;New Compute Engine autoscaler controls&lt;/b&gt;—New &lt;a href="https://cloud.google.com/compute/docs/autoscaler#scale-in_controls"&gt;scale-in controls&lt;/a&gt; in Compute Engine let you limit the VM deletion rate by preventing the autoscaler from reducing a MIG's size by more VM instances than your workload can tolerate to lose. &lt;a href="https://cloud.google.com/blog/products/compute/compute-engine-managed-instance-groups-get-scale-in-controls"&gt;Read the blog&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Lending DocAI in preview&lt;/b&gt;—&lt;a href="https://cloud.google.com/solutions/lending-doc-ai"&gt;Lending DocAI&lt;/a&gt; is a specialized solution in our Document AI portfolio for the mortgage industry that processes borrowers’ income and asset documents to speed-up loan applications. &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/lending-docai-fast-tracks-the-home-loan-process"&gt;Read the blog&lt;/a&gt;, or &lt;a href="https://www.youtube.com/watch?v=akp0zeI6_6c&amp;amp;feature=youtu.be&amp;amp;t=885" target="_blank"&gt;check out the product demo&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;/p&gt;&lt;h3&gt;&lt;p&gt;&lt;/p&gt;&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;h3&gt;&lt;hr/&gt;&lt;/h3&gt;&lt;h3&gt;Week of Oct 12-16 2020&lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;New maintenance controls for Cloud SQL&lt;/b&gt;—Cloud SQL now offers maintenance deny period controls, which allow you to prevent automatic maintenance from occurring during a 90-day time period. &lt;a href="https://cloud.google.com/blog/products/databases/new-maintenance-deny-periods-from-cloud-sql"&gt;Read the blog&lt;/a&gt;. &lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;Trends in volumetric DDoS attacks&lt;/b&gt;—This week we published a deep dive into DDoS threats, detailing the trends we’re seeing and giving you a closer look at how we prepare for multi-terabit attacks so your sites stay up and running. &lt;a href="https://cloud.google.com/blog/products/identity-security/identifying-and-protecting-against-the-largest-ddos-attacks"&gt;Read the blog&lt;/a&gt;.&lt;br/&gt;&lt;/li&gt;&lt;li&gt;&lt;b&gt;New in BigQuery&lt;/b&gt;—We shared a number of updates this week, including &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/smile-new-user-friendly-sql-capabilities-bigquery"&gt;new SQL capabilities&lt;/a&gt;, more granular control over your partitions with &lt;a href="https://cloud.google.com/bigquery/docs/creating-column-partitions"&gt;time unit partitioning&lt;/a&gt;, the general availability of &lt;a href="https://cloud.google.com/bigquery/docs/table-access-controls-intro?hl=en"&gt;Table ACLs&lt;/a&gt;, and BigQuery System Tables Reports, a solution that aims to help you monitor BigQuery &lt;a href="https://cloud.google.com/bigquery/pricing#flat_rate_pricing"&gt;flat-rate&lt;/a&gt; slot and reservation utilization by leveraging BigQuery’s underlying INFORMATION_SCHEMA views. &lt;a href="https://cloud.google.com/blog/products/data-analytics/speed-up-queries-organize-data-with-new-data-warehouse-features"&gt;Read the blog&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Cloud Code makes YAML easy for hundreds of popular Kubernetes CRDs&lt;/b&gt;—We announced authoring support for more than 400 popular Kubernetes CRDs out of the box, any existing CRDs in your Kubernetes cluster, and any CRDs you add from your local machine or a URL. &lt;a href="https://cloud.google.com/blog/products/containers-kubernetes/cloud-code-features-expanded-support-for-kubernetes-crds"&gt;Read the blog&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Google Cloud’s data privacy commitments for the AI era&lt;/b&gt;—We’ve outlined how our AI/ML Privacy Commitment reflects our belief that customers should have both the &lt;a href="https://cloud.google.com/security/"&gt;highest level of security&lt;/a&gt; and the highest level of control over data stored in the cloud. &lt;a href="https://cloud.google.com/blog/products/ai-machine-learning/google-cloud-unveils-ai-and-ml-privacy-commitment"&gt;Read the blog&lt;/a&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;New, lower pricing for Cloud CDN&lt;/b&gt;—We’ve reduced the price of cache fill (content fetched from your origin) charges across the board, by up to 80%, along with our recent introduction of &lt;a href="https://cloud.google.com/cdn/docs/release-notes#September_14_2020"&gt;a new set of flexible caching capabilities&lt;/a&gt;, to make it even easier to use Cloud CDN to optimize the performance of your applications. &lt;a href="https://cloud.google.com/blog/products/networking/introducing-lower-simpler-pricing-for-cloud-cdn"&gt;Read the blog&lt;/a&gt;.&lt;br/&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Expanding the BeyondCorp Alliance&lt;/b&gt;—Last year, we announced our BeyondCorp Alliance with partners that share our Zero Trust vision. Today, we’re announcing new partners to this alliance. &lt;a href="https://cloud.google.com/blog/products/identity-security/google-cloud-announces-new-partners-in-its-beyondcorp-alliance"&gt;Read the blog&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;New data analytics training opportunities&lt;/b&gt;—Throughout October and November, we’re offering a number of no-cost ways to learn data analytics, with trainings for beginners to advanced users. &lt;a href="https://cloud.google.com/blog/topics/training-certifications/google-cloud-data-analytics-training-for-all-skills-levels"&gt;Learn more&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;New BigQuery blog series&lt;/b&gt;—BigQuery Explained provides overviews on storage, data ingestion, queries, joins, and more. &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/bigquery-explained-blog-series"&gt;Read the series&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;h3&gt;&lt;hr/&gt;&lt;/h3&gt;&lt;h3&gt;Week of Oct 5-9 2020&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Introducing the Google Cloud Healthcare Consent Management API&lt;/b&gt;—This API gives healthcare application developers and clinical researchers a simple way to manage individuals’ consent of their health data, particularly important given the new and emerging virtual care and research scenarios related to COVID-19. &lt;a href="https://cloud.google.com/blog/topics/healthcare-life-sciences/googles-healthcare-consent-management-api-protects-user-data"&gt;Read the blog&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Announcing Google Cloud buildpacks&lt;/b&gt;—Based on the CNCF buildpacks v3 specification, these buildpacks produce container images that follow best practices and are suitable for running on all of our container platforms: Cloud Run (fully managed), Anthos, and Google Kubernetes Engine (GKE). &lt;a href="https://cloud.google.com/blog/products/containers-kubernetes/google-cloud-now-supports-buildpacks"&gt;Read the blog&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Providing open access to the Genome Aggregation Database (gnomAD)&lt;/b&gt;—Our collaboration with Broad Institute of MIT and Harvard provides free access to one of the world's most comprehensive public genomic datasets. &lt;a href="https://cloud.google.com/blog/topics/healthcare-life-sciences/google-cloud-providing-free-access-to-genome-aggregation-database"&gt;Read the blog&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Introducing HTTP/gRPC server streaming for Cloud Run&lt;/b&gt;—Server-side HTTP streaming for your serverless applications running on Cloud Run (fully managed) is now available. This means your Cloud Run services can serve larger responses or stream partial responses to clients during the span of a single request, enabling quicker server response times for your applications. &lt;a href="https://cloud.google.com/blog/products/serverless/cloud-run-now-supports-http-grpc-server-streaming"&gt;Read the blog&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;New security and privacy features in Google Workspace&lt;/b&gt;—Alongside the announcement of Google Workspace we also shared more information on new security features that help facilitate safe communication and give admins increased visibility and control for their organizations. &lt;a href="https://cloud.google.com/blog/products/workspace/announcing-new-security-and-privacy-updates-in-google-workspace"&gt;Read the blog&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Introducing Google Workspace&lt;/b&gt;—Google Workspace includes all of the productivity apps you know and use at home, at work, or in the classroom—Gmail, Calendar, Drive, Docs, Sheets, Slides, Meet, Chat and more—now more thoughtfully connected. &lt;a href="https://cloud.google.com/blog/products/workspace/introducing-google-workspace"&gt;Read the blog&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;New in Cloud Functions: languages, availability, portability, and more&lt;/b&gt;—We extended Cloud Functions—our scalable pay-as-you-go Functions-as-a-Service (FaaS) platform that runs your code with zero server management—so you can now use it to build end-to-end solutions for several key use cases. &lt;a href="https://cloud.google.com/blog/products/serverless/a-roundup-of-cloud-functions-enhancements"&gt;Read the blog&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Announcing the Google Cloud Public Sector Summit, Dec 8-9&lt;/b&gt;—Our upcoming two-day virtual event will offer thought-provoking panels, keynotes, customer stories and more on the future of digital service in the public sector. &lt;a href="https://cloudonair.withgoogle.com/events/public-sector-summit?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=-&amp;amp;utm_content=google-living-blog" target="_blank"&gt;Register at no cost&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Wed, 11 Aug 2021 22:00:00 -0000</pubDate><guid>https://cloud.google.com/blog/topics/inside-google-cloud/whats-new-google-cloud/</guid><category>Google Cloud</category><category>Google Workspace</category><category>Inside Google Cloud</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/images/whats_new_gcp.max-600x600.jpg" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>What’s new with Google Cloud</title><description>Find our newest updates, announcements, resources, events, learning opportunities, and more in one handy location.</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/images/whats_new_gcp.max-600x600.jpg</image><site_name>Google</site_name><url>https://cloud.google.com/blog/topics/inside-google-cloud/whats-new-google-cloud/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>The Google Cloud content marketing team </name><title></title><department></department><company></company></author></item><item><title>Fuel your custom models with Vertex AI</title><link>https://cloud.google.com/blog/topics/developers-practitioners/fuel-your-custom-models-vertex-ai/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;In May we announced Vertex AI, our new unified AI platform which provides options for everything from using pre-trained models to building your models with a variety of frameworks. In this post we'll do a deep dive on training and deploying a custom model on Vertex AI. There are many different tools provided in Vertex AI, as you can see in the diagram below. In this scenario we’ll be using the products highlighted in green:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="in green" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_80.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;AutoML is a great choice if you don’t want to write your model code yourself, but many organizations have scenarios that require building custom models with open-source ML frameworks like TensorFlow, XGBoost, or PyTorch. In this example, we’ll build a custom TensorFlow model (built upon &lt;a href="https://www.tensorflow.org/tutorials/keras/regression" target="_blank"&gt;this tutorial&lt;/a&gt;) that predicts the fuel efficiency of a vehicle, using the &lt;a href="https://www.kaggle.com/uciml/autompg-dataset" target="_blank"&gt;Auto MPG dataset&lt;/a&gt; from Kaggle.&lt;/p&gt;&lt;p&gt;If you’d prefer to dive right in, check out the &lt;a href="https://codelabs.developers.google.com/codelabs/vertex-ai-custom-models#0" target="_blank"&gt;codelab&lt;/a&gt; or watch the two minute video below for a quick overview of our demo scenario: &lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-video"&gt;&lt;div class="article-module article-video "&gt;&lt;figure&gt;&lt;a class="h-c-video h-c-video--marquee" data-glue-modal-disabled-on-mobile="true" data-glue-modal-trigger="uni-modal-bHoAXR26hWo-" href="https://youtube.com/watch?v=bHoAXR26hWo"&gt;&lt;img alt="Fuel your custom models on Vertex AI" src="//img.youtube.com/vi/bHoAXR26hWo/maxresdefault.jpg"/&gt;&lt;svg class="h-c-video__play h-c-icon h-c-icon--color-white" role="img"&gt;&lt;use xlink:href="#mi-youtube-icon"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/a&gt;&lt;/figure&gt;&lt;/div&gt;&lt;div class="h-c-modal--video" data-glue-modal="uni-modal-bHoAXR26hWo-" data-glue-modal-close-label="Close Dialog"&gt;&lt;a class="glue-yt-video" data-glue-yt-video-autoplay="true" data-glue-yt-video-height="99%" data-glue-yt-video-vid="bHoAXR26hWo" data-glue-yt-video-width="100%" href="https://youtube.com/watch?v=bHoAXR26hWo" ng-cloak=""&gt;&lt;/a&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Environment setup &lt;/h3&gt;&lt;p&gt;There are many options for setting up an environment to run these training and prediction steps. In the lab linked above, we use the IDE in &lt;a href="https://cloud.google.com/shell"&gt;Cloud Shell&lt;/a&gt; to build our model training application, and we pass our training code to Vertex AI as a Docker container. You can use whichever IDE you’re most comfortable working with, and if you’d prefer not to containerize your training code, you &lt;a href="https://cloud.google.com/vertex-ai/docs/training/create-python-pre-built-container"&gt;can create a Python package&lt;/a&gt; that runs on one of Vertex AI’s supported pre-built containers.&lt;/p&gt;&lt;p&gt;If you would like to use Pandas or another data science library to do exploratory data analysis, you can use the &lt;a href="https://cloud.google.com/vertex-ai/docs/general/notebooks"&gt;hosted Jupyter notebooks in Vertex AI&lt;/a&gt; as your IDE. For example, here we wanted to inspect the correlation between fuel efficiency and one of our data attributes, cylinders. We used Pandas to plot this relationship directly in our notebook.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Table chart" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_81.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;To get started, you’ll want to make sure you have a Google Cloud project with the relevant services enabled. You can enable all the products we’ll be using in one command using the gcloud SDK:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Then create a Cloud Storage bucket to store our saved model assets. With that, you’re ready to start developing your model training code.&lt;/p&gt;&lt;h3&gt;Containerizing training code&lt;/h3&gt;Here we’ll develop our training code as a Docker container, and deploy that container to &lt;a href="https://cloud.google.com/container-registry?utm_campaign=CDR_sar_aiml_ucaiplabs_011321&amp;amp;utm_source=external&amp;amp;utm_medium=web"&gt;Google Container Registry&lt;/a&gt; (GCR). To do that, create a directory with a Dockerfile at the root, along with a trainer subdirectory containing a train.py file. This is where you’ll write the bulk of your training code:&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="training code" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_82.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;To train this model, we’ll build a deep neural network using the Keras Sequential Model API:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;br/&gt;&lt;p&gt;We won’t include the full model training code here, but you can find it in &lt;a href="https://codelabs.developers.google.com/codelabs/vertex-ai-custom-models#3" target="_blank"&gt;this step&lt;/a&gt; of the codelab. Once your training code is complete, you can build and test your container locally. The IMAGE_URI in the snippet below corresponds to the location where you’ll deploy your container image in GCR. Replace $GOOGLE_CLOUD_PROJECT below with the name of your Cloud project:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;All that’s left to do is push your container to GCR by running docker push $IMAGE_URI. In the GCR section of your console, you should see your newly deployed container:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="deployed container" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_83.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Running the training job &lt;/h3&gt;&lt;p&gt;Now you're ready to train your model. You can select the container you created above in the models section of the platform. You can also specify key details like the training method, compute preferences (GPUs, RAM, etc.) and hyperparameter tuning if required.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="if required" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_84.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Now you can hand over training your model and let Vertex do the heavy lifting for you.&lt;/p&gt;&lt;h3&gt;Deploy to endpoint&lt;/h3&gt;&lt;p&gt;Next, let's get your new model incorporated into your app or service. Once your model is done training you will see an option to create a new endpoint. You can test out your endpoint in the console during your development process. Using the client libraries, you can easily create a reference to your endpoint and get a prediction with a single line of code:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Start building today&lt;/h3&gt;&lt;p&gt;Ready to start using Vertex AI? We have you covered for all your use cases spanning from simply using pre-trained models to every step of the lifecycle of a custom model. &lt;/p&gt;&lt;ul&gt;&lt;li&gt;Use Jupyter notebooks for a development experience that combines text, code and data&lt;/li&gt;&lt;li&gt;Fewer lines of code required for custom modeling&lt;/li&gt;&lt;li&gt;Use MLOps to manage your data with confidence and scale&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;Get started today by trying out &lt;a href="https://codelabs.developers.google.com/codelabs/vertex-ai-custom-models" target="_blank"&gt;this codelab&lt;/a&gt; yourself or watching &lt;a href="https://www.youtube.com/watch?v=aB2OxnyfP0c" target="_blank"&gt;this one hour workshop&lt;/a&gt;. &lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/ai-machine-learning/google-cloud-launches-vertex-ai-unified-platform-for-mlops/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/Cloud_AI.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Google Cloud unveils Vertex AI, one platform, every ML tool you need&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Google Cloud launches Vertex AI, a managed platform for experimentation, versioning and deploying ML models into production.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Wed, 11 Aug 2021 16:30:00 -0000</pubDate><guid>https://cloud.google.com/blog/topics/developers-practitioners/fuel-your-custom-models-vertex-ai/</guid><category>Google Cloud</category><category>AI &amp; Machine Learning</category><category>Developers &amp; Practitioners</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/images/VertexAndChrome-BlogHeader-02.max-600x600.png" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>Fuel your custom models with Vertex AI</title><description>Learn how to use Vertex AI - Google Cloud's newly announced managed ML platform - to build end-to-end ML workflows. You'll learn how to go from raw data to deployed model, and will leave this workshop ready to develop and productionize your own ML projects with Vertex AI.</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/images/VertexAndChrome-BlogHeader-02.max-600x600.png</image><site_name>Google</site_name><url>https://cloud.google.com/blog/topics/developers-practitioners/fuel-your-custom-models-vertex-ai/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Sara Robinson</name><title>Senior Developer Relations Engineer, Google Cloud</title><department></department><company></company></author><author xmlns:author="http://www.w3.org/2005/Atom"><name>Anu Srivastava</name><title>Senior Developer Relations Engineer, Google Cloud</title><department></department><company></company></author></item><item><title>Building a unified analytics data platform on Google Cloud</title><link>https://cloud.google.com/blog/products/data-analytics/building-unified-analytics-data-platform-google-cloud/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Every company runs on data, but not every organization knows how to create value out of the data it generates. The first step to becoming a data driven company is to create the right ecosystem for data processing in a holistic way. Traditionally, organizations’ data ecosystems consisted of point solutions that provide data services. But that point solution approach is, for many companies, no longer sufficient. &lt;/p&gt;&lt;p&gt;One of the most common questions we get from customers is, “Do I need a data lake, or should I consider a data warehouse? Do you recommend I consider both?” Traditionally, these two architectures have been viewed as separate systems, applicable to specific data types and user skill sets. Increasingly,  we see a blurring of lines between data warehouses and data lakes, which provides customers with an opportunity to create a more comprehensive platform that gives them the best of both worlds.&lt;br/&gt;&lt;/p&gt;&lt;p&gt;What if we don't need to compromise, and we instead create an end-to-end solution covering the entire data management and processing stages, from data collection to data analysis and machine learning? The result is a data platform that can store vast amounts of data in varying formats and do so without compromising on latency. At the same time, this platform can satisfy the needs of all users throughout the data lifecycle. &lt;/p&gt;&lt;h3&gt;Emerging Trends &lt;/h3&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;There is no one-size-fits-all approach to building an end-to-end data solution. Emerging concepts include data lakehouses, data meshes, and data vaults that seek to meet specific technical and organizational needs. Some are not new and have been around in different shapes and formats, however, all of them work naturally within a Google Cloud environment. Let’s look into both ends of the spectrum of enabling data and enabling teams. &lt;/p&gt;&lt;p&gt;Data mesh facilitates a decentralized approach to data ownership, allowing individual lines of business to publish and subscribe to data in a standardized manner, instead of forcing data access and stewardship through a single, centralized team. On the other hand, a data lakehouse brings raw and processed data closer together, allowing for a more streamlined and centralized repository of data needed throughout the organization. Processing can be done in transit via ELT, reducing the need to copy datasets across systems. This allows for easier data exploration and easier governance. The Data lakehouse works to store the data in a single-source-of-truth, making minimal copies of the data. This architecture offers low-cost storage in an open format accessible by a variety of processing engines like Spark, while also providing powerful management and optimization features. Consistent security and governance is key to any lakehouse. Finally, a data vault is designed to separate data-driven and model-driven activities. Data integrated into the raw vault enables parallel loading to facilitate scaling of large implementations.&lt;br/&gt;&lt;/p&gt;&lt;p&gt;In Google Cloud, there is no need to keep them separate. In fact, with interoperability among our portfolio of data analytics products, you can easily provide access to data residing in different places, effectively bringing your data lake and data warehouse together on a single platform.&lt;br/&gt;&lt;/p&gt;&lt;p&gt;Let's look at some of the technological innovations that make this reality. &lt;a href="https://cloud.google.com/bigquery"&gt;BigQuery&lt;/a&gt;’s storage API allows treating a data warehouse like a data lake, letting you access the data residing in BigQuery. For example, you can use Spark to access data residing in the data warehouse without it affecting the performance of any other jobs accessing it. This is all made possible by the underlying architecture, which separates compute and storage. Likewise, &lt;a href="https://cloud.google.com/dataplex"&gt;Dataplex&lt;/a&gt;, our intelligent data fabric service, provides data governance and security capabilities across various lakehouse storage tiers built on &lt;a href="https://cloud.google.com/storage"&gt;GCS&lt;/a&gt; and BigQuery. &lt;br/&gt;&lt;/p&gt;&lt;p&gt;We will continue to offer specialized products and solutions around data lake and data warehouse functionality but over time we expect to see a significant enough convergence of the two systems that the terminology will change. At Google Cloud, we consider this combination an “analytics data platform”.&lt;br/&gt;&lt;/p&gt;&lt;h3&gt;Tactical or Strategical&lt;/h3&gt;&lt;p&gt;Google Cloud’s data analytics platform is differentiated by being open, intelligent, flexible, and tightly integrated. There are many technologies in the market which provide tactical solutions that may feel comfortable and familiar. However, this can be a rather short-term approach that simply lifts and shifts a siloed solution into the cloud. In contrast, an analytics data platform built on Google Cloud offers modern data warehousing and data lake capabilities with close integration to our &lt;a href="https://cloud.google.com/vertex-ai"&gt;AI Platform&lt;/a&gt;. It also provides built-in streaming, ML, and geospatial capabilities and an in-memory solution for BI use cases. Depending on your organizational data needs, Google Cloud has the set of products, tools, and services to create the right data platform for you. &lt;/p&gt;&lt;p&gt;To become a truly data-driven organization, the first step is to design and implement an analytics data platform that meets your technical and business needs. Whether you want to empower teams to own, publish, and share their data across the organization, or you want to create a streamlined store of raw and processed data for easier discovery, there is a solution that best meets the needs of your company.&lt;br/&gt;&lt;/p&gt;&lt;p&gt;&lt;br/&gt;To learn more about the elements of a unified analytics data platform built on Google Cloud, and the differences in platform architectures and organizational structures, read our &lt;a href="https://services.google.com/fh/files/misc/googlecloud_unified_analytics_data_platform_paper_2021.pdf" target="_blank"&gt;Unified Analytics Platform paper&lt;/a&gt;. &lt;br/&gt;&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Wed, 11 Aug 2021 16:00:00 -0000</pubDate><guid>https://cloud.google.com/blog/products/data-analytics/building-unified-analytics-data-platform-google-cloud/</guid><category>Data Analytics</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/images/image1_f9x0T5Y.max-600x600.png" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>Building a unified analytics data platform on Google Cloud</title><description>Data-driven transformation starts with a modern data platform, built on Google Cloud</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/images/image1_f9x0T5Y.max-600x600.png</image><site_name>Google</site_name><url>https://cloud.google.com/blog/products/data-analytics/building-unified-analytics-data-platform-google-cloud/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Susan Pierce</name><title>Solutions Manager, AI and Smart Analytics, Google Cloud</title><department></department><company></company></author><author xmlns:author="http://www.w3.org/2005/Atom"><name>Firat Tekiner</name><title>EMEA Solution Lead, Data Analytics</title><department></department><company></company></author></item><item><title>Businesses realize the full value of visual data using Plainsight Vision AI on Google Cloud</title><link>https://cloud.google.com/blog/topics/startups/plainsight-ai-solutions-powered-by-google-cloud/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Data-savvy businesses increasingly rely on images and videos for critical functions, and yet are challenged by the sheer mass of information—more than 3.2 billion images and 720,000 hours of video are &lt;a href="https://www.sciencedaily.com/releases/2020/10/201021112337.htm" target="_blank"&gt;created daily&lt;/a&gt;. This explosion in visual data has paved the way for the growth of computer vision, a form of artificial intelligence (AI) that enables computers to “see” the world similarly to the way people do, but with unblinking consistency, and greater accuracy. &lt;/p&gt;&lt;p&gt;The transformational impact and value of computer vision solutions are significant and has been a guiding objective for companies and AI developers. And yet, even as the applications for computer vision increase dramatically, architecting and implementing vision AI solutions remain highly complex. Visual data, such as images and video, are made up of thousands of pixels of information that represent millions of different patterns and meanings, which can make interpreting even a single image overwhelming from a computational perspective.&lt;/p&gt;&lt;p&gt;Many organizations struggle with deployments and fail to operationalize vision AI solutions due to development delays, machine learning and data science hiring challenges, inaccurate output, a lack of integration with existing infrastructure, difficulty of use, and high cost. Plainsight, with the power of Google Cloud resources, is addressing all these challenges and helping businesses by enabling the deployment of vision AI within enterprise private networks that can be managed easily and scaled economically.&lt;/p&gt;&lt;p&gt;&lt;a href="https://plainsight.ai/plainsight-on-google-cloud-marketplace/?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=plainsightongooglecloud&amp;amp;utm_content=textlink" target="_blank"&gt;Plainsight has announced availability&lt;/a&gt; of its vision AI platform on Google Cloud Marketplace. Businesses can now easily deploy end-to-end vision AI to private clouds to realize the full value of their video and other visual data for accurate, actionable insights across diverse use cases.&lt;/p&gt;&lt;h3&gt;Delivering on the Promise of AI: Seeing What’s Hiding In Plain Sight&lt;/h3&gt;&lt;p&gt;For organizations to integrate AI and machine learning into their businesses successfully, the technology must be powerful enough to solve real challenges, yet fast, easy, and accessible enough to ensure the innovation potential is realized. Plainsight on Google Cloud delivers the power of enterprise vision AI that’s quick and easy to use with Google Cloud resources that enable global scale, increased security, bolstered privacy, unified billing, and cost savings. &lt;/p&gt;&lt;p&gt;To streamline vision AI workflows, Plainsight facilitates the entire pipeline, from visual data ingestion and annotation, through continuous model training, deployment, and monitoring for easier innovation and faster time-to-production. Our platform accelerates vision AI development in a manner that is complete, accurate, and accessible to non-technical business leaders. We believe that AI should be available and accessible to anyone and everyone—so that teams across entire organizations can reap the benefits. &lt;/p&gt;&lt;p&gt;By integrating Plainsight into their private networks, companies worldwide can now leverage one intuitive platform for centralized control of streamlined vision AI model creation and training with optimized visual data handling for diverse enterprise solutions. These use cases include: &lt;i&gt;social distancing monitoring, medical imaging, drug compound screening, defect detection in manufacturing processes, identifying gas leaks, or even livestock counting and crop health monitoring for agriculture&lt;/i&gt;, to name a few.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-video"&gt;&lt;div class="article-module article-video "&gt;&lt;figure&gt;&lt;a class="h-c-video h-c-video--marquee" data-glue-modal-disabled-on-mobile="true" data-glue-modal-trigger="uni-modal-A7U_0UkjvEg-" href="https://youtube.com/watch?v=A7U_0UkjvEg"&gt;&lt;img alt="An end to end walkthrough featuring Plainsight's vision AI platform for healthcare organizations." src="//img.youtube.com/vi/A7U_0UkjvEg/maxresdefault.jpg"/&gt;&lt;svg class="h-c-video__play h-c-icon h-c-icon--color-white" role="img"&gt;&lt;use xlink:href="#mi-youtube-icon"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/a&gt;&lt;/figure&gt;&lt;/div&gt;&lt;div class="h-c-modal--video" data-glue-modal="uni-modal-A7U_0UkjvEg-" data-glue-modal-close-label="Close Dialog"&gt;&lt;a class="glue-yt-video" data-glue-yt-video-autoplay="true" data-glue-yt-video-height="99%" data-glue-yt-video-vid="A7U_0UkjvEg" data-glue-yt-video-width="100%" href="https://youtube.com/watch?v=A7U_0UkjvEg" ng-cloak=""&gt;&lt;/a&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;We enable customers so they can create successful solutions that enable them to clearly see their business from all angles and to take advantage of the knowledge visual data can reveal by simply and quickly operationalizing practical vision AI applications.&lt;/p&gt;&lt;h3&gt;AI-Powered Dataset Creation, Automated Model Training &amp;amp; Easy Deployment Without A Single Line Of Code &lt;/h3&gt;&lt;p&gt;For vision AI applications, success is inextricably dependent on the quality and quantity of the datasets required to train the relevant models. To aid enterprises in this vital stage, the Plainsight platform provides built-in data annotation for the fast and easy creation of datasets. This includes AI-powered features that accelerate the speed and quality of labeling such as &lt;a href="https://www.youtube.com/watch?v=yHtwwjmrEss&amp;amp;ab_channel=Sixgill%2CLLC" target="_blank"&gt;SmartPoly&lt;/a&gt;, for the automated polygon masking of objects, TrackForward, to predict and automatically label objects from frame to frame in video annotations, and AutoLabel for automated object recognition and labeling based on pre-trained machine learning models, to highlight a few.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="vision AI application.gif" src="https://storage.googleapis.com/gweb-cloudblog-publish/original_images/vision_AI_application.gif"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;In addition, to ensure the success of AI integration, we significantly reduce time-intensive processes with Plainsight vision AI’s automated machine learning with continuous model training and easy deployment capabilities. In just a few clicks, users can leverage optimizations for the most reliable model training without endless experimentation cycles. And, models are easily deployed at scale all within one, easy-to-manage model operationalization process for the business.&lt;/p&gt;&lt;h3&gt;Growing With Google Cloud&lt;/h3&gt;&lt;p&gt;Plainsight is a vision AI innovation leader, developing solutions that address unmet needs for challenger brands and Fortune 500s across vertical markets. As a team recognized for succeeding where others have failed, our expanding partnership with Google Cloud provides a powerful combination that helps customers see and activate the value of their visual data with a suite of services in a secure and private manner.&lt;/p&gt;&lt;p&gt;Our vision AI Platform simplifies building and operationalizing AI to solve business problems enterprises are facing every day—and the demand is increasing. To accelerate our journey to faster, more accessible AI for enterprises, we knew we needed strong support to grow Plainsight and scale our backend tools to match our vision. &lt;/p&gt;&lt;p&gt;Google Cloud delivered everything, and more, in one program. &lt;a href="https://cloud.google.com/startup"&gt;The Startup Program by Google Cloud&lt;/a&gt; provided the technology and services for scale and the support we needed to maximize the value the Program provided us. The Startup Program has been a springboard for architecting Plainsight vision AI in the cloud, accelerating our goals and optimizing innovation, efficiency, and growth. The team also helped us optimize &lt;a href="https://ads.google.com/home/" target="_blank"&gt;Google Ads&lt;/a&gt; campaigns, fueling adoption of Plainsight. &lt;/p&gt;&lt;p&gt;After launching the SaaS version of Plainsight Data Annotation in November 2020, we grew our user base by nearly 110x in just three short months. Google Ads has also dramatically increased website traffic, growing new users by nearly 5.75X and page views by over 5X. The Google team helped us identify where Google Cloud offerings could be leveraged instead of developing in-house solutions and offered best practices that enabled us to deliver faster on our initiatives. &lt;/p&gt;&lt;p&gt;Kubernetes was already the underlying component of our platform and leveraging Google Kubernetes Engine (GKE) as a managed service removed a layer of complexity. By combining GKE and Anthos, we were able to standardize our deployments, aligning to how our customers leverage Anthos for enterprise applications in their own organizations. In addition, as a fast-moving, customer-centric company we use Google Workspace to help us centralize and manage our day-to-day work internally. By leveraging multiple products across Google’s ecosystem, we take advantage of a holistic partnership that has helped our business tremendously as we scale. &lt;/p&gt;&lt;h3&gt;Leveraging Google’s Partners for Strategic Consultation&lt;/h3&gt;&lt;p&gt;To facilitate this expansion of our partnership with Google and to maximize our use of Google Cloud services, we are working with &lt;a href="https://www.doit-intl.com/doit-named-2020-google-cloud-global-reseller-partner-of-the-year/" target="_blank"&gt;DoiT International&lt;/a&gt;, a Google Managed Services Provider and 2020 Global Reseller Partner of the Year. DoiT provides us with ongoing technical consultation for cloud-native architecture, Google Cloud Marketplace integration, production-grade Kubernetes support, Google Cloud cost optimization, and technical support. The DoiT team has been invaluable in compiling best practices, tips, and strategies from their vast experience with various cloud customers to ease our Marketplace integration and is providing input for infrastructure strategy to support our continued rapid growth.&lt;/p&gt;&lt;h3&gt;Plainsight Delivers Enterprise Vision AI Through The Google Cloud Platform Marketplace&lt;/h3&gt;&lt;p&gt;&lt;a href="https://plainsight.ai/plainsight-on-google-cloud-marketplace/?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=plainsightongooglecloud&amp;amp;utm_content=textlink" target="_blank"&gt;Plainsight vision AI is now available&lt;/a&gt; to Google Cloud Customers on Google Cloud Marketplace enabling organizations across industries to deploy private Plainsight instances within their own environments. Marketplace customers will benefit from Google Cloud privacy, security, scalability and unified billing through their Google Cloud account. &lt;/p&gt;&lt;p&gt;Combining the powerful benefits provided by Google Cloud resources with &lt;a href="https://plainsight.ai/plainsight-on-google-cloud-marketplace/?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=plainsightongooglecloud&amp;amp;utm_content=textlink" target="_blank"&gt;Plainsight’s vision AI Platform&lt;/a&gt; into private networks, enterprises worldwide can now leverage one intuitive platform for centralized control of streamlined vision AI model creation and training with optimized visual data handling for diverse enterprise solutions. &lt;/p&gt;&lt;p&gt;Through our Google partnership, we’re able to leverage a powerful foundation that allows us to rapidly innovate, scale and accelerate delivery on our vision AI platform capabilities. By executing on our vision to make AI easier, faster and more accessible for all users across entire enterprises, we’re helping businesses see more and by seeing more, they’ll have the power to solve more. &lt;/p&gt;&lt;p&gt;&lt;i&gt;If you want to learn more about how Google Cloud can help your startup, visit our page &lt;a href="https://inthecloud.withgoogle.com/startup/dl-cd.html?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=FY21-Q1-global-demandgen-website-cs-startup_program_mc&amp;amp;utm_content=sixgill_post_ref" target="_blank"&gt;here&lt;/a&gt; where you can &lt;a href="https://inthecloud.withgoogle.com/startup/dl-cd.html?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=FY21-Q1-global-demandgen-website-cs-startup_program_mc&amp;amp;utm_content=sixgill_post_ref" target="_blank"&gt;apply&lt;/a&gt; for our Startup Program, and &lt;a href="https://docs.google.com/forms/d/e/1FAIpQLSfowlgaSsVDQojZ1JDDhRMfZ5TAFY6do4UPZXqkuToX63K2dQ/viewform" target="_blank"&gt;sign up for our monthly startup newsletter&lt;/a&gt; to get a peek at our community activities, digital events, special offers, and more.&lt;/i&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout_external"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href=""&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Wed, 11 Aug 2021 16:00:00 -0000</pubDate><guid>https://cloud.google.com/blog/topics/startups/plainsight-ai-solutions-powered-by-google-cloud/</guid><category>AI &amp; Machine Learning</category><category>Startups</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/images/Google_Cloud_AI_Vision.max-600x600.jpg" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>Businesses realize the full value of visual data using Plainsight Vision AI on Google Cloud</title><description>AI startup Plainsight built its Vision AI offering on Google Cloud, enabling companies to extract accurate, actionable insights from videos and images.</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/images/Google_Cloud_AI_Vision.max-600x600.jpg</image><site_name>Google</site_name><url>https://cloud.google.com/blog/topics/startups/plainsight-ai-solutions-powered-by-google-cloud/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Carlos Anchia</name><title>CEO of Plainsight</title><department></department><company></company></author><author xmlns:author="http://www.w3.org/2005/Atom"><name>Chris Curtis</name><title>Startup Marketing Manager</title><department></department><company></company></author></item><item><title>IT leaders are choosing a sustainable future</title><link>https://cloud.google.com/blog/topics/sustainability/it-leaders-are-choosing-sustainable-future/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;The global-wide interruptions of the coronavirus pandemic provided the opportunity for businesses to take a closer look at how we work, learn, live, and consume. With work stoppages and quarantine orders in place, carbon emissions and pollution levels saw significant reductions, highlighting how business and environmental sustainability are linked. As the world returns to work, businesses need to adjust to the new reality and make decisions that will ensure the continuing sustainability of our planet. &lt;/p&gt;&lt;h3&gt;Understanding the need for sustainability&lt;/h3&gt;&lt;p&gt;A recent &lt;a href="https://inthecloud.withgoogle.com/it-leaders-research-21/overview-dl-cd.html" target="_blank"&gt;Google-commissioned study by IDG&lt;/a&gt;, based on a global survey of more than 2,000 IT decision-makers, illustrated the importance of sustainability for organizations’ IT departments while also revealing that significant work is needed before businesses meet their sustainability goals. The good news is that ninety percent say that sustainability is a priority of their IT department and as more organizations consider the benefits of digital transformation and cloud migration, 75% report that sustainability is a “must-have” or “major consideration” when evaluating cloud providers.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout_external"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href=""&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Businesses face both internal and external pressures to reform their practices and processes to meet new targets around sustainability, and IT leaders have an important role to play in that effort. Specifically, 54% said sustainability initiatives were important to their companies because of corporate social responsibility, 46% said it was because of environmental regulations and mandates, and 40% said it was due to the climate change impacts.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="why is sustainability important to your company.jpg" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/why_is_sustainability_important_to_your_co.max-1000x1000.jpg"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;The research also examined the areas where IT leaders are looking to make positive changes for sustainability in their organizations. Not surprisingly, 62% said in their IT infrastructure and data centers, 53% their supply chain, and 46% in their facilities and building operations.&lt;/p&gt;&lt;h3&gt;Facing the obstacles to progress&lt;/h3&gt;&lt;p&gt;However, for many organizations, behaviors lag intent. Only 67% actually have environmental sustainability targets in place. The gap between behavior and intent widens when looking at digitally forward vs. digitally conservative organizations. &lt;i&gt;For the purposes of the study, digitally forward companies were defined as either digital natives or those with a digital strategy and implementation in place. Digitally conservative organizations either have no strategy in place, or are in the process of considering or planning digital transformations, but haven’t reached the stage of implementation.&lt;/i&gt; &lt;/p&gt;&lt;p&gt;The study revealed that digital natives embracing cloud solutions were more likely than more conservative organizations to demonstrate a commitment to sustainability. Twenty percent of digital natives are implementing IT initiatives to reduce emissions to become more environmentally sustainable, while only 10% of digitally conservative organizations have done so. &lt;/p&gt;&lt;p&gt;When considering the effects of the COVID-10 pandemic, 20% of respondents from digital conservative organizations delayed or cancelled initiatives around reducing emissions to become more environmentally sustainable compared to only 12% for digitally forward respondents. &lt;/p&gt;&lt;h3&gt;Sustainability is in our DNA&lt;/h3&gt;&lt;p&gt;Sustainability has been a core value for Google since our inception. We were the first major company to become carbon neutral in 2007 and we were the first major company to match our energy use with 100 percent renewable energy in 2017 — and have continued to do so every year since. We are proud to operate the cleanest global cloud in the industry, and we’re the world’s largest corporate purchaser of renewable energy. It is this commitment that makes Google the leading choice for organizations with sustainability initiatives. &lt;/p&gt;&lt;p&gt;To learn more about the IDG findings and how IT leaders are implementing sustainable solutions, &lt;a href="https://inthecloud.withgoogle.com/it-leaders-research-21/overview-dl-cd.html?utm_source=google&amp;amp;utm_medium=blog&amp;amp;utm_campaign=FY21-Q1-NORTHAM-demandgen-website-wd-gcp_gtm_reimagine_master&amp;amp;utm_content=homepage" target="_blank"&gt;download the full report&lt;/a&gt;.&lt;/p&gt;&lt;hr/&gt;&lt;p&gt;&lt;i&gt;Interested in how Google Cloud's commitment to providing &lt;a href="https://sustainability.google/" target="_blank"&gt;sustainable solutions&lt;/a&gt; can help IT leaders and their organizations meet their own sustainability targets?&lt;/i&gt;&lt;/p&gt;&lt;p&gt;Google is focused on enhancing our products to help billions of people take action to reduce their environmental footprint.&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;Google creates tools and invests in technology that will foster a carbon-free future for everyone and reduce carbon footprints.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Google Cloud is the only major cloud provider to purchase enough renewable energy to cover our entire operations, enabling us to operate the cleanest cloud in the industry.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;When running on Google Cloud, an organization’s usage is net carbon neutral, and the electricity used to power workloads is matched 100% with renewable energy.&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;Looking for more? &lt;a href="https://cloud.google.com/blog/topics/sustainability/google-cloud-region-picker-helps-you-make-the-green-choice"&gt;Check out our blog&lt;/a&gt; that introduces the &lt;a href="https://gcp-region-picker.web.app/" target="_blank"&gt;Google Cloud Region Picker&lt;/a&gt;, which helps organizations choose a Google Cloud region based on carbon footprint, price, and latency.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/sustainability/google-cloud-region-picker-helps-you-make-the-green-choice/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/gcp_sustainability_region_picker.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Faster, cheaper, greener? Pick the Google Cloud region that’s right for you&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;New Google Cloud region picker makes it easy for you to take advantage of our greenest data centers.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Wed, 11 Aug 2021 16:00:00 -0000</pubDate><guid>https://cloud.google.com/blog/topics/sustainability/it-leaders-are-choosing-sustainable-future/</guid><category>Google Cloud</category><category>Research</category><category>Sustainability</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/images/gcp_sustainability.max-600x600.jpg" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>IT leaders are choosing a sustainable future</title><description>A recent Google-commissioned study by IDG highlights the importance of sustainable solutions when selecting a cloud provider.</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/images/gcp_sustainability.max-600x600.jpg</image><site_name>Google</site_name><url>https://cloud.google.com/blog/topics/sustainability/it-leaders-are-choosing-sustainable-future/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Natalie Lambert</name><title>Director of Messaging and Content, Google Cloud</title><department></department><company></company></author></item><item><title>Streaming VR and AR content from Google Cloud with NVIDIA CloudXR</title><link>https://cloud.google.com/blog/products/gaming/streaming-vr-and-ar-content-google-cloud-nvidia-cloudxr/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;The Opportunity for Streamed AR/VR Content &lt;/h3&gt;&lt;p&gt;What if you could get a high quality AR/VR experience without a dedicated physical computer—or even without a physical tether?  &lt;/p&gt;&lt;p&gt;In the past, interacting with VR required a dedicated, high-end workstation and, depending on the headset, wall-mounted sensors and a dedicated physical space. Complex tasks in VR can push the limits of sensor reach, cable-length, and spatial boundaries, entangling the artist and restricting their movement. This solution was not scalable beyond a handful of advanced use cases.&lt;/p&gt;&lt;p&gt;Recently, tetherless VR headsets from manufacturers such as HTC and Oculus have emerged that free the user from these odious restrictions, enabling a new freedom to experience VR and AR from just about anywhere. The enhanced portability and reduced cost has led to substantially increased adoption among consumers and opened up many new exploratory use cases in enterprise.&lt;/p&gt;&lt;p&gt;However, while these headsets are more accessible and portable, a tradeoff in compute power was required in order to achieve these goals. The limited on-device compute power of this new generation of Head Mounted Displays (HMDs) is acceptable for many consumer applications such as casual gaming. However, advanced enterprise workloads with heavy graphics, compute, or memory requirements can be difficult or even impossible to run on these lightweight devices.&lt;/p&gt;&lt;p&gt;By combining &lt;a href="https://www.nvidia.com/en-us/design-visualization/solutions/cloud-xr/" target="_blank"&gt;NVIDIA CloudXR&lt;/a&gt; with their powerful GPUs running &lt;a href="https://www.nvidia.com/en-us/design-visualization/virtual-workstation/" target="_blank"&gt;NVIDIA RTX Virtual Workstations (vWS&lt;/a&gt;) in Google Cloud data centers, you can experience high-fidelity VR and AR applications from just about anywhere with a good internet connection. The heavy computations are performed in the cloud on a GPU-attached VM and the content streams to any CloudXR compatible headset. &lt;/p&gt;&lt;p&gt;The combination of Google Cloud's private fiber optic network — the same network we built for global delivery of YouTube content — and CloudXR's QoS technologies provides the user with the highest possible quality of service. In fact, the streaming experience is comparable to that of a headset tethered to a powerful physical workstation, but without the friction of hardware and cables. &lt;/p&gt;&lt;p&gt;This combination of power and portability sets the stage to unlock the potential of high-quality gaming and enterprise AR/VR experiences anywhere on Earth.&lt;/p&gt;&lt;h3&gt;Case Study: Creating a Masterpiece&lt;/h3&gt;&lt;p&gt;Digital character creation is a core skill for many 3D artists today. One of the preferred methods of 3D character creation is digital sculpting, which lets artists create both hard-surface and organic shapes with high levels of accuracy and control.&lt;/p&gt;&lt;p&gt;Sculpting is one of the many tasks in the character designer's skillset. Artists must also master the texturing, rigging, and posing of characters as part of the digital character creation pipeline. &lt;/p&gt;&lt;p&gt;Mastering all these tasks can be challenging, and often, the technology gets in the way; working in multiple applications requires switching contexts and workflows, breaking the artist's creative flow. The traditional user experience can also be unintuitive, forcing the creator to translate what they want to do with their hands and head into mouse movement and keyboard presses.&lt;/p&gt;&lt;p&gt;Masterpiece Studio Pro revolutionizes this character creation workflow by giving artists the first fully immersive 3D creation pipeline. Artists work in VR, giving them a far more intuitive and seamless way to work which combines the best of the digital and physical worlds.&lt;/p&gt;&lt;p&gt;In Masterpiece Studio Pro, the artist can work on the character or object at any scale, using familiar &lt;a href="https://www.youtube.com/watch?v=kx6-IjnuJXk" target="_blank"&gt;tools and hand gestures to sculpt a model&lt;/a&gt;, much as they would a real clay figure. Performing other tasks such as skeleton creation is simple, allowing the artist to work directly with the limbs of a character to place and adjust the joints.&lt;/p&gt;&lt;h3&gt;Bringing It All Together&lt;/h3&gt;&lt;p&gt;NVIDIA CloudXR streaming from the cloud provides tremendous opportunities for new creative use cases within gaming and across the enterprise. This solution joins &lt;a href="https://developersonair.withgoogle.com/events/game-dev-summit-2021/watch?talk=bringing-the-game-studio-to-the-cloud" target="_blank"&gt;Virtual Studio for Gaming&lt;/a&gt; as the latest in our series to help developers build better games.&lt;/p&gt;&lt;p&gt;The Masterpiece Studio use case is a powerful demonstration of new modalities for content creation and collaboration. To try CloudXR with NVIDIA RTX vWS on Google Cloud for yourself, see &lt;a href="https://cloud.google.com/community/tutorials/streaming-vr-content-from-a-virtual-workstation-using-nvidia-cloudxr"&gt;this tutorial&lt;/a&gt;. Masterpiece Studio Pro also offers extensive &lt;a href="https://masterpiecestudio.com/learn" target="_blank"&gt;learning materials&lt;/a&gt; and a &lt;a href="https://masterpiecestudio.com/studio" target="_blank"&gt;free trial&lt;/a&gt;.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/retail/virtual-automotive-showrooms-with-google-cloud-unreal-engine-and-nvidia/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/gcp_x_pagani.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Next-gen virtual automotive showrooms are here, thanks to Google Cloud, Unreal Engine, and NVIDIA&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Next-gen virtual showrooms provide interactive 3D experiences, photorealistic materials and environments, and up to 4K cloud streaming on...&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Wed, 11 Aug 2021 16:00:00 -0000</pubDate><guid>https://cloud.google.com/blog/products/gaming/streaming-vr-and-ar-content-google-cloud-nvidia-cloudxr/</guid><category>Google Cloud</category><category>Media &amp; Entertainment</category><category>Gaming</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/images/gaming_Dt0gyUV.max-600x600.jpg" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>Streaming VR and AR content from Google Cloud with NVIDIA CloudXR</title><description>How NVIDIA CloudXR in Google Cloud data centers enables high-fidelity VR and AR applications.</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/images/gaming_Dt0gyUV.max-600x600.jpg</image><site_name>Google</site_name><url>https://cloud.google.com/blog/products/gaming/streaming-vr-and-ar-content-google-cloud-nvidia-cloudxr/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Rob Martin</name><title>Chief Architect - Google Cloud for Games</title><department></department><company></company></author></item><item><title>Running Anthos inside Google</title><link>https://cloud.google.com/blog/topics/developers-practitioners/running-anthos-inside-google/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;With everyone and their dog shifting to containers, and away from virtual machines (VMs), we realized that running vendor-provided software on VMs at Google was slowing us down. &lt;/p&gt;&lt;h3&gt;So we moved. &lt;/h3&gt;&lt;p&gt;Enter &lt;a href="https://cloud.google.com/anthos"&gt;Anthos&lt;/a&gt;, Google Cloud's managed application platform, and its associated developer tools. Today we'll take you through our process of moving Confluence and Acrolinx from VMs running in our private data center environment over to a fully managed, containerized deployment for Google. Both Confluence and Acrolinx were deployed before on the Google Compute Engine platform and have been used within Google for content management. &lt;/p&gt;&lt;p&gt;In the past, Google used internal systems for allocating application resources, automating replication and providing high availability for enterprise applications, but these systems relied on customized infrastructure and they were often incompatible with enterprise software. &lt;/p&gt;&lt;p&gt;The many frustrations that came with running enterprise applications on VMs included:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;Service turnup times in the order of days&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Hard-to-manage infrastructure and workloads programmatically&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Challenges with VM monoliths management (as compared to microservices)&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Reliable rollback of application install/upgrade failures&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Challenges with enforcing security policies at scale&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;… and many others&lt;/p&gt;&lt;p&gt;To mitigate these frustrations, we made the shift to an industry-standard, universally available managed platform: Kubernetes.&lt;/p&gt;&lt;h3&gt;Kubernetes and Anthos&lt;/h3&gt;&lt;p&gt;Deploying &lt;a href="https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/" target="_blank"&gt;Kubernetes&lt;/a&gt; gave us the ability to configure, manage, and extend workloads running on containers rather than VMs. The good news was that it could handle the scale of our deployments with ease.&lt;/p&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/anthos/docs/concepts/overview#:~:text=Anthos%20is%20a%20modern%20application,you%20can%20leverage%20its%20features."&gt;Anthos&lt;/a&gt; is Google Cloud's platform of tools and technologies designed to ease the management of containerized workloads, whether running on Google Cloud, other clouds, or on-premises. It brings configuration management, service management, telemetry, logging and cluster management tooling. In addition, it saves operational overhead for our application teams.&lt;/p&gt;&lt;p&gt;As our vendor-provided software became compatible with containerization, we could build on 15 years of experience running containerized workloads and enjoy the perks of using a fully managed cloud service for our applications.&lt;/p&gt;&lt;p&gt;Adopting Anthos gave us some big benefits right away:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;Automated resource provisioning&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Application lifecycle management&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Security Policies Management&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Config-as-code for workload state&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;This removed substantial manual toil from our team, freeing them up for more productive work. Using Anthos Config Connector we could express the compute, networking and storage needs through code, allowing Anthos to allocate them without manual interaction. We also relied on Anthos to administer creating Kubernetes clusters and manage a single admin cluster that would host the Config Connector. This gave us simpler orchestration when we needed to create new Kubernetes clusters to run our applications.&lt;/p&gt;&lt;h3&gt;How we modernized operations&lt;/h3&gt;&lt;p&gt;Our continuous integration and continuous deployment process benefitted from Anthos as well. By using &lt;a href="https://cloud.google.com/anthos/config-management"&gt;Anthos Config Management&lt;/a&gt; (Config Sync), a multi-repository configuration sync utility, we can automate the process of applying our desired configuration to the Kubernetes clusters that we would otherwise have applied manually before via kubectl. The multi-repo Config Sync provides a consistent experience when managing both the common security policies across clusters and the workload specific configs that are namespace-scoped.&lt;/p&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/anthos-config-management/docs/config-sync-overview"&gt;Config Sync&lt;/a&gt; is a Kubernetes Custom Resource Definition (CRD) resource which is installed on a user cluster by GKE Hub.&lt;/p&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/anthos/multicluster-management/fleets"&gt;GKE Hub&lt;/a&gt; provides networking assistance within Anthos, and lets you logically group together similar GKE clusters. Once the clusters are registered with a GKE Hub, the same security policies can be administered on all the registered clusters. Onboarding a new application then wouldn’t incur any additional overhead, because the same security policies would be applied automatically.&lt;/p&gt;&lt;p&gt;The resulting clusters and administration of these applications looks like this:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="diagram" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/diagram_anthos.max-1000x1000.png"/&gt;&lt;figcaption class="article-image__caption "&gt;&lt;div class="rich-text"&gt;&lt;p&gt;A high level view of Anthos-managed workloads running on GKE clusters&lt;/p&gt;&lt;/div&gt;&lt;/figcaption&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Our updated deployment process&lt;/h3&gt;&lt;p&gt;We've deployed a variety of third-party applications on Anthos. Today, we'll walk you through how we set up Confluence and Acrolinx.&lt;/p&gt;&lt;p&gt;To provision and deploy, we need to:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;Ensure that all the configs (both security policies and workload configs) are stored in a single source of truth (i.e Git repos). Any changes must be reviewed and approved by multiple parties to prevent unilateral changes.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Deploy and enforce our required security policies.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Express the desired state of workload configs in a Git repo.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Deploy a Continuous Integration and Continuous Deployment pipeline to ensure that changes to the configs are tested before committing them to a Git repo. Such configs will then be applied to the target clusters to ensure the desired state of both the applications.&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;Even though we're running multiple segmented workloads, we can apply common security policies to them all. We also delegate application deployment to the developers while maintaining security guardrails to prevent mistakes.&lt;/p&gt;&lt;h3&gt;How we set up Anthos clusters&lt;/h3&gt;&lt;p&gt;We know what we want to deploy, and how to protect them. Let's dig into how we can set up those clusters with Terraform and then how to make sure all our security policies are applied. Once that is complete we can let the developer or operator manage any future changes to the application, while the cluster admin retains control of any cluster policy changes.&lt;/p&gt;&lt;p&gt;We'll register the cluster with the right GKE Hub, then apply our desired configuration to that cluster, and finally deploy the applications to their namespaces.&lt;/p&gt;Let’s start with the prod GKE cluster. We can create it using &lt;a href="https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/container_cluster" target="_blank"&gt;these Terraform templates&lt;/a&gt;,then cluster with GKE Hub using:&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Next, we will enable the ACM/Config Sync feature for the GKE Hub, hub-prod, using the gcloud command-line:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Here, a ConfigManagement resource configures Config Sync on the prod GKE cluster with the relevant root Git repo (root-prod).&lt;/p&gt;&lt;p&gt;After creating the GKE clusters, we will set up cluster namespaces to deploy Confluence and Acrolinx:&lt;/p&gt;&lt;p&gt;Here's one way the root and namespace repos can be organized in a root-prod &lt;a href="https://cloud.google.com/anthos-config-management/docs/config-sync-overview#repositories"&gt;structured repo&lt;/a&gt;.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;All the cluster-scoped resources will be kept in the cluster directory while all the namespace scoped resources for the given applications will be kept in each of the namespaces sub-directories. This separation allows us to define the common cluster scoped security policies at a higher level while still defining application configs at each application namespace level. The cluster admins can own the security policies while delegating namespace ownership to the developers. &lt;/p&gt;&lt;p&gt;We now have a GKE cluster prod that is registered with a GKE Hub. Since the cluster is registered with the GKE Hub with Config Sync enabled, the security policies now can be applied on this cluster.&lt;/p&gt;&lt;h3&gt;Deploying changes to applications&lt;/h3&gt;&lt;p&gt;In order for Config Sync to apply config changes to resources of Confluence and Acrolinx applications, each of the Namespace resources and Namespace repos must first be configured.&lt;/p&gt;&lt;p&gt;Looking at an example of a root-prod Git repo as shown above and the respective Namespaces repos, RepoSync resources and how Confluence and Acrolinx application resources will be managed by Config Sync in the prod GKE cluster.&lt;/p&gt;&lt;p&gt;The following is an example of a Namespace and RepoSync resource in the confluence-prod directory.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Config Sync will read the Namespace config file and create confluence-prod Namespace in the same prod GKE cluster. &lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;The RepoSync resource sets up a process to connect to the Git repo to find configuration information that will be used by the Confluence application.&lt;/p&gt;&lt;p&gt;We are now ready to create Kubernetes resources for Confluence from its namespace Git repo.&lt;/p&gt;&lt;p&gt;Next, we can deploy a &lt;a href="https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/" target="_blank"&gt;StatefulSet&lt;/a&gt; resource that defines the container’s spec (CPU, RAM, etc.) for running Confluence app in the confluence-prod namespace repo: &lt;/p&gt;&lt;p&gt;After submission to the repo, Config Sync will read the StatefulSet and deploy the image based on the resources listed.&lt;/p&gt;&lt;h3&gt;Our security practice&lt;/h3&gt;&lt;p&gt;Every organization has a requirement to ensure that the workloads are made secure without any additional efforts from the developers and that there is a central governing process that enforces such security policies across all the workloads. Thisensures that everyone follows best practices when deploying workloads.  It also reduces much of the burden and cognitive load from the developers when ensuring that workloads follow such security principles and policies.&lt;/p&gt;&lt;p&gt;Historically, when running applications on VMs, it has been traditionally difficult to micro-segment applications, apply a different set of policies to the micro-segmented applications and/or based on workload identities. Some examples of such policies are: whether an application is built and deployed in a verifiable manner; preventing privilege escalation (e.g setuid binaries) and applying that config for a group of workloads etc.&lt;/p&gt;&lt;p&gt;With the advent of Kubernetes and standards such as &lt;a href="https://www.openpolicyagent.org/" target="_blank"&gt;OPA&lt;/a&gt; (Open Policy Agent), it is possible now to micro-segment workloads, define a set of policies that can enforce certain constraints and rules at the workload identity level for a group of similar workload resources. &lt;a href="https://cloud.google.com/anthos-config-management/docs/reference/constraint-template-library"&gt;This&lt;/a&gt; is one such library of OPA Constraints that can be used to enforce policy across Cluster workloads.&lt;/p&gt;&lt;a href="https://cloud.google.com/anthos-config-management/docs/concepts/policy-controller"&gt;Policy Controller&lt;/a&gt; enables the enforcement of fully programmable policies. You can use these policies to actively block non-compliant API requests, or simply to audit the configuration of your clusters and report violations. Policy Controller is based on the open source &lt;a href="https://github.com/open-policy-agent/gatekeeper" target="_blank"&gt;Open Policy Agent Gatekeeper project&lt;/a&gt; and comes with a &lt;a href="https://cloud.google.com/anthos-config-management/docs/reference/constraint-template-library"&gt;full library of pre-built policies&lt;/a&gt; for common security and compliance controls.&lt;/div&gt;&lt;/div&gt;&lt;div class="block-pull_quote"&gt;&lt;div class="uni-pull-quote h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;div class="uni-pull-quote__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;div class="uni-pull-quote__inner-wrapper h-c-copy h-c-copy"&gt;&lt;q class="uni-pull-quote__text"&gt;This will allow the developers to focus on just the application lifecycle management while the platform admins will ensure that such security policies are enforced on all the registered clusters and workloads.&lt;/q&gt;&lt;/div&gt;&lt;/div&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Conclusion&lt;/h3&gt;&lt;p&gt;In the end we got to a much better place by deploying our applications with Anthos, backed by Kubernetes.&lt;/p&gt;&lt;p&gt;Our security policies were enforced automatically, we scale up and down with demand, and new versions could be deployed smoothly. Our developers enjoyed faster workflows, whether spinning up a new environment or testing out an update for stability. Provisioning got easier too, with less overhead for the team, especially as deployments grew to service the whole of Google.&lt;/p&gt;&lt;p&gt;Overall we're quite happy with how we improved developer productivity with faster application turnup times, going from days to just hours for a new application. At the same time we’re better able to reliably enforce policies that ensure that applications are hosted in a secure and reliable environment. &lt;/p&gt;&lt;p&gt;We're glad we can share some of our journey with you; if you want to try it out yourself, &lt;a href="https://cloud.google.com/anthos/docs/setup/overview"&gt;get started with Anthos today&lt;/a&gt;.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/public-sector/accelerating-governments-zero-trust-journey/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Accelerating Government’s Zero Trust Journey&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;In May, the White House issued an Executive Order aiming to improve the nation’s cybersecurity defenses and requiring US Federal agencies...&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Tue, 10 Aug 2021 22:00:00 -0000</pubDate><guid>https://cloud.google.com/blog/topics/developers-practitioners/running-anthos-inside-google/</guid><category>Google Cloud</category><category>Anthos</category><category>Developers &amp; Practitioners</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/images/hero_anthos.max-600x600.png" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>Running Anthos inside Google</title><description>How we run our third-party software on Anthos at Google</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/images/hero_anthos.max-600x600.png</image><site_name>Google</site_name><url>https://cloud.google.com/blog/topics/developers-practitioners/running-anthos-inside-google/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Bikramjeet Assal</name><title>Systems Engineer</title><department></department><company></company></author><author xmlns:author="http://www.w3.org/2005/Atom"><name>Max Saltonstall</name><title>Developer Advocate, Google Cloud</title><department></department><company></company></author></item><item><title>Zero trust with reverse proxy</title><link>https://cloud.google.com/blog/topics/developers-practitioners/zero-trust-reverse-proxy/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;A reverse proxy stands in front of your data, services, or virtual machines, catching requests from anywhere in the world and carefully checking each one to see if it is allowed.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-pull_quote"&gt;&lt;div class="uni-pull-quote h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;div class="uni-pull-quote__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;div class="uni-pull-quote__inner-wrapper h-c-copy h-c-copy"&gt;&lt;q class="uni-pull-quote__text"&gt;In order to decide (yes or no) the proxy will look at who and what.&lt;/q&gt;&lt;/div&gt;&lt;/div&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Who are you (the individual making the request)? What is your role? Do you have access permission (authorization)?&lt;br/&gt;&lt;/p&gt;&lt;p&gt;What device are you using to make the request? How healthy is your device right now? Where are you located? &lt;/p&gt;&lt;p&gt;At what time are you making the request?&lt;br/&gt;&lt;/p&gt;&lt;p&gt;This issue of GCP Comics presents an example of accessing some rather confidential data from an airplane, and uses that airplane as a metaphor to explain what the proxy is doing.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;a href="https://storage.googleapis.com/gweb-cloudblog-publish/images/Google-Cloud-Comic-issue8-Exports_full-com.max-2200x2200.png" rel="external" target="_blank"&gt;&lt;img alt="Zero trust" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/Google-Cloud-Comic-issue8-Exports_full-com.max-1000x1000.png"/&gt;&lt;/a&gt;&lt;figcaption class="article-image__caption "&gt;&lt;div class="rich-text"&gt;Click to enlarge&lt;/div&gt;&lt;/figcaption&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Reverse proxies work as part of the load balancing step when requests are made to web apps or services, and they can be thought of as another element of the network infrastructure that helps route requests to the right place. No one can access your resources unless they meet certain rules and conditions.&lt;/p&gt;&lt;p&gt;If a request is invalid or doesn’t meet the necessary criteria set by your administrators, either because it is from an unauthorized person or an unsafe device, then the proxy will deny the request.&lt;/p&gt;&lt;p&gt;Why might the proxy say no to my request? When assessing the user making the request, denial of access could be due to reasons such as:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;I'm in Engineering, but I am trying to access Finance data.&lt;/li&gt;&lt;li&gt;I'm not even a part of the company.&lt;/li&gt;&lt;li&gt;My job changed, and I lost access.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;Looking at the device originating the request, the proxy could deny access due to a number of factors, such as:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Device operating system out of date&lt;/li&gt;&lt;li&gt;Malware detected&lt;/li&gt;&lt;li&gt;Device is not reporting in&lt;/li&gt;&lt;li&gt;Disk encryption missing&lt;/li&gt;&lt;li&gt;Device doesn’t have screen lock&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;Leveraging identity and device information to secure access to your organization’s resources improves your security posture.&lt;/p&gt;&lt;h3&gt;Resources&lt;/h3&gt;&lt;p&gt;To learn more about proxies and Zero Trust, check out the following resources:&lt;/p&gt;&lt;a href="https://cloud.google.com/iap/docs/concepts-overview"&gt;&lt;/a&gt;&lt;ul&gt;&lt;a href="https://cloud.google.com/iap/docs/concepts-overview"&gt;&lt;/a&gt;&lt;a href="https://cloud.google.com/iap/docs/concepts-overview"&gt;&lt;/a&gt;&lt;li&gt;&lt;a href="https://cloud.google.com/iap/docs/concepts-overview"&gt;&lt;/a&gt;&lt;a href="https://cloud.google.com/iap/docs/concepts-overview"&gt;&lt;/a&gt;&lt;a href="https://cloud.google.com/iap/docs/concepts-overview"&gt;Overview of Identity-Aware Proxy&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://blog.google/products/chrome/extending-enterprise-zero-trust-models-web/" target="_blank"&gt;Extending Zero Trust models to the web&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://cloud.google.com/endpoint-verification/docs/creating-device-access-level"&gt;Creating a device-based access level&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://cloud.google.com/iap/docs/enabling-on-prem-howto"&gt;How to set up a proxy for on-premises apps&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://cloud.google.com/beyondcorp-enterprise/docs/quickstart"&gt;BeyondCorp Enterprise Quickstart Guide&lt;/a&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;Want more GCP Comics? Visit &lt;a href="https://gcpcomics.com/" target="_blank"&gt;gcpcomics.com&lt;/a&gt; &amp;amp; follow us on medium &lt;a href="https://medium.com/@pvergadia/" target="_blank"&gt;pvergadia&lt;/a&gt; &amp;amp; &lt;a href="https://max-saltonstall.medium.com/" target="_blank"&gt;max-saltonstall&lt;/a&gt;, and on Twitter at &lt;a href="https://twitter.com/pvergadia" target="_blank"&gt;@pvergadia&lt;/a&gt; and &lt;a href="https://twitter.com/maxsaltonstall" target="_blank"&gt;@maxsaltonstall&lt;/a&gt;. Be sure not to miss the next issue!&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/developers-practitioners/what-zero-trust-identity-security/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;What is zero trust identity security?&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;A zero trust network is one in which no person, device, or network enjoys inherent trust. All trust, which allows access to information, ...&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Tue, 10 Aug 2021 17:00:00 -0000</pubDate><guid>https://cloud.google.com/blog/topics/developers-practitioners/zero-trust-reverse-proxy/</guid><category>Google Cloud</category><category>Developers &amp; Practitioners</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/images/zero_trust_header.max-600x600.png" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>Zero trust with reverse proxy</title><description>A reverse proxy stands in front of your data, services, or virtual machines, catching requests from anywhere in the world and carefully checking each one to see if it is allowed.</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/images/zero_trust_header.max-600x600.png</image><site_name>Google</site_name><url>https://cloud.google.com/blog/topics/developers-practitioners/zero-trust-reverse-proxy/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Priyanka Vergadia</name><title>Developer Advocate, Google</title><department></department><company></company></author><author xmlns:author="http://www.w3.org/2005/Atom"><name>Max Saltonstall</name><title>Developer Advocate, Google Cloud</title><department></department><company></company></author></item><item><title>Troubleshoot GKE apps faster with monitoring data in Cloud Logging</title><link>https://cloud.google.com/blog/products/operations/troubleshoot-gke-faster-with-monitoring-data-in-your-logs/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;When you’re troubleshooting an application on Google Kubernetes Engine (GKE), the more context that you have on the issue, the faster you can resolve it. For example, did the pod exceed it’s memory allocation? Was there a permissions error reserving the storage volume? Did a rogue regex in the app pin the CPU? All of these questions require developers and operators to build a lot of troubleshooting context. &lt;/p&gt;&lt;h3&gt;Cloud Monitoring data for GKE in Cloud Logging&lt;/h3&gt;&lt;p&gt;To make it easier to troubleshoot GKE apps, we’ve added contextual &lt;a href="https://cloud.google.com/monitoring"&gt;Cloud Monitoring&lt;/a&gt; data accessible right from &lt;a href="https://cloud.google.com/logging"&gt;Cloud Logging&lt;/a&gt;. With this new feature, you can easily see the relevant pod, node and cluster events, metrics, alerts, and SLOs right from the log line itself. Additionally, the data loaded for a specific log entry is scoped to the Kubernetes resource, which saves you valuable time while investigating an app error.&lt;/p&gt;&lt;p&gt;Today’s announcement builds on other recent integrations including the addition of a logs tab nested in the details page of &lt;a href="https://cloud.google.com/blog/products/operations/easy-access-to-your-gke-logs-from-the-cloud-console"&gt;each of your GKE resources&lt;/a&gt; and combining metrics and logs in the GKE Dashboard in Monitoring. Now, wherever you start your troubleshooting journey – in Monitoring, Logging or GKE – you have the observability data at your fingertips. &lt;/p&gt;&lt;p&gt;For example, if you’re troubleshooting a GKE app error in Cloud Logging and looking at the app logs, you can now view the metric charts for container restarts, uptime, memory, CPU and storage without leaving the log entry. Active alerts are highlighted on the alerts tab, which can provide helpful context for troubleshooting. This unique and integrated experience brings together critical log and metric data for the specific Kubernetes resource where your app is running.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="GCP container details.gif" src="https://storage.googleapis.com/gweb-cloudblog-publish/original_images/GCP_container_details.gif"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Viewing Monitoring data for GKE from a log line&lt;/h3&gt;From a &lt;i&gt;k8s_container, k8s_pod, k8s_node, or k8s_cluster&lt;/i&gt; log, select the blue chip with the resource.labels resource name and then select “View Monitoring details” to access an integrated metrics panel directly from the Logs Explorer. Selecting “View in GKE” opens the detailed view of the GKE resource in the Cloud Console on a new tab.&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="2 Viewing Monitoring data for GKE from a log line.jpg" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/2_Viewing_Monitoring_data_for_GKE_from_a_l.max-1000x1000.jpg"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;The metrics panel provides a lot of contextual data including &lt;i&gt;alerts, Kubernetes events&lt;/i&gt; and &lt;i&gt;metrics&lt;/i&gt; related to the GKE resource.&lt;/p&gt;&lt;h3&gt;Alerts &lt;/h3&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/monitoring/alerts"&gt;Alerts&lt;/a&gt; triggered by the GKE resource are displayed under the alerts tab. The color-coded alert status provides an easy way to see ongoing, acknowledged and closed incidents. Selecting “VIEW INCIDENT” opens the incident details in Cloud Monitoring. If you want to create a new alert, use the link to create a brand new alert policy.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="3 Alerts.jpg" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/3_Alerts.max-1000x1000.jpg"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Kubernetes events for clusters and pods&lt;/h3&gt;&lt;p&gt;The metrics panel provides select events for clusters and pods.  For each event, the name, associated resource and a link to view/copy the log message are displayed. Kubernetes events can provide important information to help determine the root cause of an issue. For example, if a FailedScheduling event is displayed, this can quickly guide troubleshooting to check the resources available to the Kubernetes resource.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="4 Kubernetes events for clusters and pods .jpg" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/4_Kubernetes_events_for_clusters_and_pods_.max-1000x1000.jpg"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Metrics for containers, pods and nodes&lt;/h3&gt;&lt;p&gt;The metrics tab contains metrics bundles for container (default), pod and node metrics collected from the GKE cluster and reported in Cloud Monitoring. Each metric bundle offers pre-built charts that can be selected to view the CPU, memory, storage and container restarts. For example, by looking at the CPU or memory, you can determine whether there were any spikes in the metrics for the Kubernetes resources.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="5 Metrics for containers, pods and nodes.jpg" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/5_Metrics_for_containers_pods_and_nodes.max-1000x1000.jpg"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;More to come&lt;/h3&gt;&lt;p&gt;We’re committed to making &lt;a href="https://cloud.google.com/products/operations"&gt;Google Cloud’s operations suite&lt;/a&gt; the best place to troubleshoot your GKE apps. We’ve integrated logs directly into GKE resource details pages and built a specialized integrated GKE Dashboard, all to make it easier to troubleshoot GKE apps. However, there is still more coming and we’re already working hard to add new features to the metrics panels to surface even more context for troubleshooting GKE apps.&lt;/p&gt;&lt;h3&gt;Get started today&lt;/h3&gt;&lt;p&gt;If you haven’t already, to get started with Cloud Logging and Cloud Monitoring on GKE, view&lt;a href="https://cloud.google.com/monitoring/kubernetes-engine/observing"&gt;documentation&lt;/a&gt;, watch a quick video on &lt;a href="https://youtu.be/--4WWwx4Log" target="_blank"&gt;troubleshooting services on GKE&lt;/a&gt; and join the discussion in our new &lt;a href="https://www.googlecloudcommunity.com/gc/Cloud-Operations/bd-p/cloud-operations" target="_blank"&gt;Cloud Operations page&lt;/a&gt; on the Google Cloud Community site.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/operations/gke-operations-magic-alert-resolution-5-steps/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/world_map1.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;GKE operations magic: From an alert to resolution in 5 steps&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Teams operating microservices increasingly rely on metrics, logs, and traces to identify and troubleshoot problems. The GKE Dashboard bri...&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Tue, 10 Aug 2021 16:00:00 -0000</pubDate><guid>https://cloud.google.com/blog/products/operations/troubleshoot-gke-faster-with-monitoring-data-in-your-logs/</guid><category>GKE</category><category>Compute Engine</category><category>Google Cloud</category><category>Cloud Operations</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/images/cloudmonitoring.max-600x600.jpg" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>Troubleshoot GKE apps faster with monitoring data in Cloud Logging</title><description>View contextual Monitoring data in your GKE log lines. Easily see the relevant pod, node and cluster events and metrics for your pod.</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/images/cloudmonitoring.max-600x600.jpg</image><site_name>Google</site_name><url>https://cloud.google.com/blog/products/operations/troubleshoot-gke-faster-with-monitoring-data-in-your-logs/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Charles Baer</name><title>Product Manager, Google Cloud</title><department></department><company></company></author></item><item><title>Let’s get it started! Triggering ML pipeline runs</title><link>https://cloud.google.com/blog/topics/developers-practitioners/lets-get-it-started-triggering-ml-pipeline-runs/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;ML pipelines are great at automating end-to-end ML workflows, but what if you want to go one step further and automate the execution of your pipeline? In this post I’ll show you how to do exactly that. You’ll learn how to trigger your Vertex Pipelines runs in response to data added to a BigQuery table.&lt;/p&gt;&lt;p&gt;I’ll focus on automating pipeline executions rather than building pipelines from scratch. If you want to learn how to build ML pipelines, check out &lt;a href="https://codelabs.developers.google.com/vertex-pipelines-intro#0" target="_blank"&gt;this codelab&lt;/a&gt; or &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/use-vertex-pipelines-build-automl-classification-end-end-workflow"&gt;this blog post&lt;/a&gt;.&lt;/p&gt;&lt;h3&gt;What are ML pipelines? A quick refresher&lt;/h3&gt;&lt;p&gt;If the term ML pipeline is throwing you for a loop, you’re not alone! Let’s first understand what that means and the tools we’ll be using to implement it. ML pipelines are part of the larger practice of MLOps, which is concerned with productionizing ML workflows in a reproducible, reliable way. &lt;/p&gt;&lt;p&gt;When you’re building out an ML system and have established steps for gathering and preprocessing data, and model training, deployment, and evaluation, you might start by building out these steps as ad-hoc, disparate processes. You may want to share the workflow you’ve developed with another team and ensure they get the same results as you when they go through the steps. This will be tricky if your ML steps aren’t connected, and that’s where pipelines can help. With pipelines, you define your ML workflow as a series of steps or components. Each step in a pipeline is embodied by a container, and the output of each step will be fed as input to the next step. &lt;/p&gt;&lt;p&gt;How do you build a pipeline? There are open source libraries that do a lot of this heavy lifting by providing tooling for expressing and connecting pipeline steps and converting them to containers. Here I’ll be using Vertex Pipelines, a serverless tool for building, monitoring, and running ML pipelines. The best part? It supports pipelines built with two popular open source frameworks: &lt;a href="https://www.kubeflow.org/docs/components/pipelines/overview/pipelines-overview/" target="_blank"&gt;Kubeflow Pipelines&lt;/a&gt; (which I’ll use here) and &lt;a href="https://www.tensorflow.org/tfx" target="_blank"&gt;Tensorflow Extended&lt;/a&gt; (TFX).&lt;/p&gt;&lt;h3&gt;Compiling Vertex Pipelines with the Kubeflow Pipelines SDK&lt;/h3&gt;&lt;p&gt;This post assumes you’ve already defined a pipeline that you’d like to automate. Let’s imagine you’ve done this using the Kubeflow Pipelines SDK. Once you’ve defined your pipeline, the next step is to compile it. This will generate a JSON file with your pipeline definition that you’ll use when running the pipeline.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;With your pipeline compiled, you’re ready to run it. If you’re curious what a pipeline definition looks like, check out &lt;a href="https://codelabs.developers.google.com/vertex-pipelines-intro#4" target="_blank"&gt;this tutorial&lt;/a&gt;.&lt;/p&gt;&lt;h3&gt;Triggering a pipeline run from data added to BigQuery&lt;/h3&gt;&lt;p&gt;In MLOps, it’s common to retrain your model when new data is available. Here, we’ll look specifically at how to trigger a pipeline run when more data is added to a table in BigQuery. This assumes your pipeline is using data from BigQuery to train a model, but you could use the same approach outlined below and replace BigQuery with a different data source. Here’s a diagram of what we’ll build:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Build" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_70.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;To implement this, we’ll use a Cloud Function to check for new data, and if there is we’ll execute our pipeline. The first step here is to determine how many new rows of data should trigger model retraining. In this example we’ll use 1000 as the threshold, but you can customize this value based on your use case. Inside the Cloud Function, we’ll compare the number of rows in our BigQuery table to the amount of data last used to train our model. If it exceeds our threshold, we’ll kick off a new pipeline run to retrain our model.&lt;/p&gt;&lt;p&gt;There are a few types of Cloud Functions to choose from. For this we’ll use an HTTP function so that we can trigger it with Cloud Scheduler. The function will take two parameters: the name of the BigQuery dataset where you’re storing model training data, along with the table containing that data. The function then creates a table called count in that dataset, and uses it to keep a snapshot of the number of rows used last time you ran your retraining pipeline:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;If the current number of rows in the table exceeds the latest value in the count table by your predetermined threshold, it’ll kick off a pipeline run and update count to the new number of rows with the Kubeflow Pipelines SDK method &lt;code&gt;create_run_from_job_spec&lt;/code&gt;:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;The resulting count table will show a running log of the size of your data table each time the function kicked off a pipeline run:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Pipeline run" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_71.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;You can see the full function code in &lt;a href="https://gist.github.com/sararob/1eea7ae2b08e85851855ec2eff8c2d8b" target="_blank"&gt;this gist&lt;/a&gt;, where the check_table_size function is the Cloud Functions entrypoint. Note that you’ll want to add error handling based on your use case to catch scenarios where the pipeline run fails.&lt;/p&gt;&lt;p&gt;When you deploy your function, include a requirements.txt file with both the kfp and google-cloud-bigquery libraries. You’ll also need your compiled pipeline JSON file. Once your function is deployed, it’s time to create a Cloud Scheduler job that will run this function on a recurring basis. &lt;/p&gt;&lt;p&gt;You can do this right from the &lt;a href="https://console.cloud.google.com/cloudscheduler"&gt;Cloud Scheduler section&lt;/a&gt; of your console. First, click Create job and give the job a name, frequency, and time zone. The frequency will largely depend on how often new data is added in your application. Setting this won’t necessarily run your pipeline with the frequency you specify, it’ll only be checking for new data in BigQuery[1]. &lt;/p&gt;&lt;p&gt;In this example we’ll run this function weekly, on Mondays at 9:00am EST:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Cloud Scheduler" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_72.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Next, set HTTP as the target type and add the URL of the Cloud Function you deployed. In the body, add the JSON with the two parameters this function takes: your BigQuery dataset and table name:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Table name" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_73.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Then create a service account that has the &lt;a href="https://cloud.google.com/functions/docs/securing/authenticating#authenticating_function_to_function_calls"&gt;Cloud Functions Invoker role&lt;/a&gt;. Under Auth header, select Add OIDC token and add the service account you just created:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="just created" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_74.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;With that, you can create the Scheduler job, sit back, and relax with the comforting thought that your retraining pipeline will run when enough new data becomes available. Want to see everything working? Go to the &lt;a href="https://console.cloud.google.com/cloudscheduler"&gt;Cloud Scheduler in your console&lt;/a&gt; to see the last time your job ran:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="job ran" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_75.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;You can also click the Run Now button on the right if you don’t want to wait for the next scheduled time. To test out the function directly, you can go to the &lt;a href="https://console.cloud.google.com/functions/list"&gt;Functions section of your console&lt;/a&gt; and test it right in the browser, passing in the two parameters the function expects:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Function Expects" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_76.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Finally, you can see the &lt;a href="https://console.cloud.google.com/vertex-ai/pipelines"&gt;pipeline running in Vertex AI&lt;/a&gt;. Here’s an example of what a completed pipeline run looks like:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Pipeline" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_77.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;What’s next?&lt;/h3&gt;&lt;p&gt;In this post I showed you how to trigger your pipeline when new data is added to a BigQuery table. To learn more about the different products I covered here, check out these resources:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;a href="https://codelabs.developers.google.com/vertex-pipelines-intro#0" target="_blank"&gt;Vertex Pipelines intro codelab&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://cloud.google.com/scheduler/docs/quickstart"&gt;Cloud Scheduler quickstart&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://cloud.google.com/functions/docs/quickstarts"&gt;Cloud Functions quickstart&lt;/a&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;Do you have comments on this post or ideas for more ML content you’d like to see? Let me know on Twitter at &lt;a href="https://twitter.com/srobtweets" target="_blank"&gt;@SRobTweets&lt;/a&gt;.&lt;/p&gt;&lt;p&gt;[1] Note that if you’d like to run your pipeline on a recurring schedule you can use the create_schedule_from_job_spec method as described in the &lt;a href="https://cloud.google.com/vertex-ai/docs/pipelines/run-pipeline#scheduling_a_recurring_pipeline_run_using_the"&gt;docs&lt;/a&gt;. This will create a Cloud Scheduler job that runs your pipeline at the specified frequency, rather than running it only in response to changes in your Cloud environment.&lt;/p&gt;&lt;p&gt;&lt;i&gt;Thank you to &lt;a href="https://twitter.com/mco_dev" target="_blank"&gt;Marc Cohen&lt;/a&gt; and &lt;a href="https://twitter.com/polonglin" target="_blank"&gt;Polong Lin&lt;/a&gt; for their feedback on this post.&lt;/i&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/developers-practitioners/use-vertex-pipelines-build-automl-classification-end-end-workflow/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Use Vertex Pipelines to build an AutoML classification end-to-end workflow&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;How you can use Vertex Pipelines to build an end-to-end ML workflow for training a custom model using AutoML&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Mon, 09 Aug 2021 17:30:00 -0000</pubDate><guid>https://cloud.google.com/blog/topics/developers-practitioners/lets-get-it-started-triggering-ml-pipeline-runs/</guid><category>Google Cloud</category><category>AI &amp; Machine Learning</category><category>Developers &amp; Practitioners</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/images/GCP_brain.max-600x600.jpg" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>Let’s get it started! Triggering ML pipeline runs</title><description>Learn two approaches to automate your Vertex Pipeline runs using Cloud Scheduler and Cloud Functions.</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/images/GCP_brain.max-600x600.jpg</image><site_name>Google</site_name><url>https://cloud.google.com/blog/topics/developers-practitioners/lets-get-it-started-triggering-ml-pipeline-runs/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Sara Robinson</name><title>Developer Advocate, Google Cloud Platform</title><department></department><company></company></author></item><item><title>Use log buckets for data governance, now supported in 23 regions</title><link>https://cloud.google.com/blog/products/operations/keep-your-logs-data-compliant-with-regional-log-buckets/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Logs are an essential part of troubleshooting applications and services. However, ensuring your developers, DevOps, ITOps, and SRE teams have access to the logs they need, while accounting for operational tasks such as scaling up, access control, updates, and keeping your data compliant, can be challenging. To help you offload these operational tasks associated with running your own logging stack, we offer &lt;a href="https://cloud.google.com/logging"&gt;Cloud Logging&lt;/a&gt;. If you don’t need to worry about data residency, Cloud Logging will pick a region to store and process your logs. &lt;/p&gt;&lt;p&gt;If you do have data governance and compliance requirements, we’re excited to share that Cloud Logging now offers even more flexibility and control by providing you a choice of which region to store and process your logging data. In addition to the information below, we recently &lt;a href="https://cloud.google.com/resources/data-governance-logs-best-practices-whitepaper"&gt;published a whitepaper&lt;/a&gt; that details compliance best practices for logs data.&lt;/p&gt;&lt;p&gt;Choose from 23 regions to help keep your logs data compliant&lt;/p&gt;&lt;p&gt;Log entries from apps and services running on Google Cloud will automatically be received by Cloud Logging within the region where the resource is running. From there, logs will be stored in log buckets. Log buckets have many attributes in common with Cloud Storage buckets, including the ability to:&lt;/p&gt;&lt;ol&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/logging/docs/storage#logs-retention"&gt;Set retention&lt;/a&gt; from 1 day to 10 years&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/logging/docs/buckets#locking-logs-buckets"&gt;Lock a log bucket&lt;/a&gt; to prevent anyone from deleting logs or reducing the retention period of the bucket&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/logging/docs/regionalized-logs"&gt;Choose a region&lt;/a&gt; for your log bucket. We recently introduced support for 23 regions to host your log buckets:&lt;/p&gt;&lt;/li&gt;&lt;/ol&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/about/locations/#europe"&gt;Europe&lt;/a&gt; - &lt;code&gt;europe-central2&lt;/code&gt;, &lt;code&gt;europe-north1&lt;/code&gt;, &lt;code&gt;europe-west1&lt;/code&gt;, &lt;code&gt;europe-west2&lt;/code&gt;, &lt;code&gt;europe-west3&lt;/code&gt;, &lt;code&gt;europe-west4&lt;/code&gt;, &lt;code&gt;europe-west6&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/about/locations/#americas"&gt;Americas&lt;/a&gt; - &lt;code&gt;us-central1&lt;/code&gt;, &lt;code&gt;us-east1&lt;/code&gt;, &lt;code&gt;us-east4&lt;/code&gt;, &lt;code&gt;us-west1&lt;/code&gt;, &lt;code&gt;us-west2&lt;/code&gt;, &lt;code&gt;us-west3&lt;/code&gt;, &lt;code&gt;northamerica-northeast1&lt;/code&gt;, &lt;code&gt;southamerica-east1&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/about/locations/#asia-pacific"&gt;Asia Pacific&lt;/a&gt; - &lt;code&gt;asia-east1&lt;/code&gt;, &lt;code&gt;asia-east2&lt;/code&gt;, &lt;code&gt;asia-northeast1&lt;/code&gt;, &lt;code&gt;asia-northeast2&lt;/code&gt;, &lt;code&gt;asia-northeast3&lt;/code&gt;, &lt;code&gt;asia-south1&lt;/code&gt;, &lt;code&gt;asia-southeast1&lt;/code&gt;, &lt;code&gt;australia-southeast1&lt;/code&gt;    &lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;h3&gt;How to create a log bucket&lt;/h3&gt;&lt;p&gt;You can get started with regionalized log storage in less than five minutes.&lt;/p&gt;&lt;ol&gt;&lt;li&gt;&lt;p&gt;Go to the Cloud Console and go to Logging&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Navigate to Logs Storage and click on “Create logs bucket”&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Name the log bucket and choose the desired region. &lt;b&gt;Note that the region cannot be changed later.&lt;/b&gt; &lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Set the retention period and then click Create Bucket.&lt;/p&gt;&lt;/li&gt;&lt;/ol&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="create log bucket.gif" src="https://storage.googleapis.com/gweb-cloudblog-publish/original_images/create_log_bucket.gif"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Once you have created the bucket, you need to point the incoming logs to that bucket. To complete this:&lt;/p&gt;&lt;ol&gt;&lt;li&gt;&lt;p&gt;Go to the Logs Router section of the Cloud Console and click on the dots to the right of the _Default sink. &lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Select “Edit Sink”&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Under Sink Destination, change the log bucket selected from “projects/.../_Default” to “projects/.../ (name of newly created bucket)”. &lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Scroll to the bottom and select “Update sink” to save the changes&lt;/p&gt;&lt;/li&gt;&lt;/ol&gt;&lt;p&gt;If you need more detailed information on this topic, please see our &lt;a href="https://services.google.com/fh/files/misc/whitepaper_data_governance_logs_how_to.pdf" target="_blank"&gt;step by step getting started guide&lt;/a&gt; for overcoming common logs data compliance challenges. &lt;/p&gt;&lt;h3&gt;More about data residency in Cloud Logging&lt;/h3&gt;&lt;p&gt;We have covered a lot of information about logs in this blog. For more on this topic and other best practices for compliance with logs data, please download this &lt;a href="https://cloud.google.com/resources/data-governance-logs-best-practices-whitepaper"&gt;whitepaper&lt;/a&gt;. We hope this helps you focus on managing your apps rather than your operations. If you would like to pose a question or join the conversation about Google Cloud operations with other professionals, please visit our &lt;a href="https://www.googlecloudcommunity.com/gc/Cloud-Operations/bd-p/cloud-operations" target="_blank"&gt;new community page&lt;/a&gt;. Happy Logging!&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout_external"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href=""&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Mon, 09 Aug 2021 16:00:00 -0000</pubDate><guid>https://cloud.google.com/blog/products/operations/keep-your-logs-data-compliant-with-regional-log-buckets/</guid><category>GKE</category><category>Compute</category><category>Google Cloud</category><category>Cloud Operations</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/images/compute.max-600x600.jpg" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>Use log buckets for data governance, now supported in 23 regions</title><description>If you have data governance and compliance requirements, Cloud Logging can help you meet them with regional log buckets, available now in 23 regions.</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/images/compute.max-600x600.jpg</image><site_name>Google</site_name><url>https://cloud.google.com/blog/products/operations/keep-your-logs-data-compliant-with-regional-log-buckets/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Mary Koes</name><title>Product Manager, Google Cloud</title><department></department><company></company></author><author xmlns:author="http://www.w3.org/2005/Atom"><name>Andrew Eames</name><title>Software Engineer</title><department></department><company></company></author></item><item><title>Data Validation for Oracle to Cloud Spanner Migration</title><link>https://cloud.google.com/blog/topics/developers-practitioners/data-validation-oracle-cloud-spanner-migration/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h1&gt;Introduction&lt;/h1&gt;&lt;p&gt;The Oracle database is a popular RDBMS in the database industry,  but for a highly scalable environments we see customers choosing to leverage Cloud Spanner with its multi-region, unlimited scalability and up to 99.999% availability. Migration of Oracle to Spanner can be addressed &lt;a href="https://cloud.google.com/architecture/migrating-oracle-to-cloud-spanner"&gt;using many approaches&lt;/a&gt; but one of the most important stages of the migration is Data Validation. One tool that helps simplify the data validation process is the &lt;a href="https://github.com/GoogleCloudPlatform/professional-services-data-validator" target="_blank"&gt;Data Validation Tool (DVT).&lt;/a&gt;&lt;/p&gt;&lt;br/&gt;&lt;p&gt;Data Validation Tool is an open sourced python CLI based tool that compares heterogeneous data source tables and provides multi-level validation functions. DVT can connect to different types of source and target databases and create validation reports based on the criteria defined. In this blog, we will see how to set up the Data Validation Tool to compare data between Oracle and Spanner tables and create validation reports in BigQuery.&lt;/p&gt;&lt;br/&gt;&lt;h2&gt;Installing Data Validation Tool&lt;/h2&gt;&lt;p&gt;To install and run a data validation tool (DVT), we need a machine with python 3.6+, pip, gcc installed.  DVT natively supports BigQuery and Spanner connections with Google Cloud SDK installed. DVT has the capability to write the output of data validation into BigQuery tables directly (console output is default).  &lt;/p&gt;&lt;br/&gt;&lt;p&gt;BigQuery output tables need to be created beforehand for this and we can use terraform to create them easily. This will need a terraform cli to be installed on your machine. In addition to that, make sure git packages are installed to clone the repository.&lt;/p&gt;&lt;br/&gt;&lt;p&gt;Clone the repository onto your machine.&lt;/p&gt;&lt;p&gt;&lt;br/&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Inside the repository, navigate to the terraform folder. Perform the below steps to create the BigQuery tables necessary for data validation reports.&lt;/p&gt;&lt;ol&gt;&lt;li&gt;&lt;p&gt;Delete the testenv.tf file inside the terraform folder&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;View variables.tf inside the terraform folder and replace default "pso-kokoro-resources" with default = "YOUR_PROJECT_ID"&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Run the following commands from terraform directory&lt;/p&gt;&lt;/li&gt;&lt;/ol&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Alternatively, we can use the bq command line tool as a part of Google Cloud SDK to create the tables.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Before installing the dvt tool, create a virtual environment in python to sandbox the tool.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Now we can install the tool in the selected environment using pip.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;To configure the Oracle connection, install cw_Oracle package using pip. &lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;This will also require an Oracle instant client library installed on the local machine.  Follow the steps mentioned &lt;a href="https://docs.oracle.com/en/database/oracle/oracle-database/21/lacli/install-instant-client-using-zip.html#GUID-D3DCB4FB-D3CA-4C25-BE48-3A1FB5A22E84" target="_blank"&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;h2&gt;Creating Connections:&lt;/h2&gt;&lt;p&gt;Before running the validation, connection to the Oracle system needs to be created. &lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt; Fill in the below details before executing the command&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;&lt;table&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;&lt;p&gt;&amp;lt;oracle-host-name&amp;gt;&lt;/p&gt;&lt;/td&gt;&lt;td&gt;&lt;p&gt;Oracle hostname&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;p&gt;&amp;lt;oracle-port-number&amp;gt;&lt;/p&gt;&lt;/td&gt;&lt;td&gt;&lt;p&gt;Oracle portname&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;p&gt;&amp;lt;user-name&amp;gt;&lt;/p&gt;&lt;/td&gt;&lt;td&gt;&lt;p&gt;Oracle login username&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;p&gt;&amp;lt;password&amp;gt;&lt;/p&gt;&lt;/td&gt;&lt;td&gt;&lt;p&gt;Oracle login password&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;p&gt;&amp;lt;database-name&amp;gt;&lt;/p&gt;&lt;/td&gt;&lt;td&gt;&lt;p&gt;Oracle database name&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Next step, create a connection to the Spanner instance.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt; Fill in the below details before executing the command:&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;p&gt;&lt;table&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;&lt;p&gt;&amp;lt;gcp-project-id&amp;gt;&lt;/p&gt;&lt;/td&gt;&lt;td&gt;&lt;p&gt;GCP Project ID where Spanner instance is located&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;p&gt;&amp;lt;spanner-instance-id&amp;gt;&lt;/p&gt;&lt;/td&gt;&lt;td&gt;&lt;p&gt;Spanner Instance ID&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;p&gt;&amp;lt;spanner-database-id&amp;gt;&lt;/p&gt;&lt;/td&gt;&lt;td&gt;&lt;p&gt;Spanner Database ID&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Use `gcloud auth application-default login` to use the default user to connect to GCP services. In case if you want to use a service account to create a connection, you can provide a service account json key file in the custom parameter `google_service_account_key_path`.&lt;/p&gt;&lt;h2&gt;Validation Types:&lt;/h2&gt;&lt;p&gt;DVT supports the following validation types:&lt;/p&gt;&lt;br/&gt;&lt;p&gt;Table level&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;Table row count&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Group by row count&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Column aggregation - SUM(), MIN(), MAX(), AVG()&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Filters and limits&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;Column level&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;Full column data type&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;br/&gt;&lt;p&gt;Based on the scenario, different validation types can be used. For this tutorial, we will be doing each column aggregation based validation using count, min and max functions. To visualize and customize the validation, provide the config-file option to write the validation configuration as a yaml file.&lt;/p&gt;&lt;br/&gt;&lt;p&gt;For example, we are considering validation of employee table which has 11 columns. We will be adding count based aggregation on all columns and min and max aggregation on the salary column.&lt;/p&gt;&lt;p&gt;&lt;br/&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Fill in the below details before executing the command&lt;/p&gt;&lt;br/&gt;&lt;p&gt;&lt;table&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;&lt;p&gt;&amp;lt;gcp-project-id&amp;gt;&lt;/p&gt;&lt;/td&gt;&lt;td&gt;&lt;p&gt;GCP Project ID where Spanner instance is located&lt;/p&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;/p&gt;&lt;p&gt;Config.yaml is created with the validation criteria.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h2&gt;Running Validation:&lt;/h2&gt;&lt;p&gt;After the configuration file is created, we can run the validation based on the confg.yaml.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;This command will run the validation tool and store the results in the BigQuery table.&lt;/p&gt;&lt;p&gt;&lt;/p&gt;&lt;h2&gt;Analyzing Validation Report&lt;/h2&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="image2.png" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/image2_bJKVuW2.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;The validation report shows aggregation values and differences for each column defined in the config file. Status column automatically fills the value based on the threshold provided (default threshold is 0.0). &lt;/p&gt;&lt;br/&gt;&lt;h3&gt;Further Information&lt;/h3&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/blog/products/databases/automate-data-validation-with-dvt"&gt;Introducing the Data Validation Tool for EDW migrations&lt;/a&gt;&lt;/p&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/architecture/migrating-oracle-to-cloud-spanner"&gt;Migrating from Oracle to Cloud Spanner&lt;/a&gt;&lt;/p&gt;&lt;br/&gt;&lt;p&gt;Data Validation Tool is under active development. Please give them a try and help out by giving feedback, filing issues and sending PRs for fixes and enhancements. These open source tools are part of the user community and are not officially supported by Google Cloud. &lt;/p&gt;&lt;p&gt;&lt;br/&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Mon, 09 Aug 2021 14:00:00 -0000</pubDate><guid>https://cloud.google.com/blog/topics/developers-practitioners/data-validation-oracle-cloud-spanner-migration/</guid><category>Spanner</category><category>Databases</category><category>Developers &amp; Practitioners</category><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>Data Validation for Oracle to Cloud Spanner Migration</title><description>Data Validation Tool is an open sourced python CLI based tool that compares heterogeneous data source tables and provides multi-level validation functions. DVT can connect to different types of source and target databases and create validation reports based on the criteria defined. In this blog, we will see how to set up the Data Validation Tool to compare data between Oracle and Spanner tables and create validation reports in BigQuery.</description><site_name>Google</site_name><url>https://cloud.google.com/blog/topics/developers-practitioners/data-validation-oracle-cloud-spanner-migration/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Logan Ramalingam</name><title>Strategic Cloud Engineer, Google Cloud</title><department></department><company></company></author></item><item><title>New Research: COVID-19 accelerates innovation in healthcare but tech adoption still lags</title><link>https://cloud.google.com/blog/topics/healthcare-life-sciences/google-cloud-covid-19-research/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Since the start of the COVID-19 pandemic, there’s been a rapid acceleration of digital transformation across the entire healthcare industry. Telehealth has become a more mainstream and safe way for patients and caregivers to connect. Machine learning modeling has helped speed up innovation and drug discovery. And new levels of integration and data portability have helped enable greater vaccine availability and equitable access to those who need it.&lt;/p&gt;&lt;p&gt;Data has been at the crux of this digital transformation -- helping people stay healthy, accelerating life sciences research and delivering more personalized and equitable care. We recently unveiled &lt;a href="https://cloud.google.com/blog/topics/healthcare-life-sciences/google-and-harris-poll-healthcare-interoperability-survey"&gt;partial results from our research&lt;/a&gt; with The Harris Poll, which revealed that nearly all physicians (95%) believe increased data interoperability will ultimately help improve patient outcomes. Today, we’re unveiling the second part of that research. &lt;/p&gt;&lt;p&gt;In February 2020, we commissioned The Harris Poll to survey 300 physicians in the U.S. about their biggest pain points -- this was just before the COVID-19 pandemic strained the entire healthcare system and made us all hyper-aware of the risks we take in going to the hospital. In June 2021, we followed-up with those same questions and more. What it unveiled was just how much COVID-19 reshaped technology’s role in the healthcare field and how it’s changing day-to-day operations for physicians. &lt;/p&gt;&lt;h3&gt;Here are some of the highlights: &lt;/h3&gt;&lt;p&gt;&lt;b&gt;Healthcare organizations accelerated technological upgrades over the course of the pandemic.&lt;/b&gt; After a year shaped primarily by the COVID-19 pandemic, use of telehealth saw substantial YOY growth, jumping nearly threefold from 32% in February 2020 to 90% this year. Forty-five percent of physicians say the COVID-19 pandemic accelerated the pace of their organization’s adoption of technology. In fact, more than 3 in 5 physicians (62%) say the pandemic has forced their healthcare organization to make technology upgrades that normally would have taken years. For example, 48% of physicians would like to have access to telehealth capabilities in the next five years. Before the COVID-19 pandemic, about half of physicians (53%) say their healthcare organization’s approach to the adoption of technology would best be described as “neutral” (i.e., willing to try new technologies only if they have been in the market for awhile or others have tried and recommended them). &lt;/p&gt;&lt;p&gt;&lt;b&gt;Despite the technological leaps this year, most physicians still believe the industry lags behind in technology adoption but recognize the opportunity for technological support and advancement.&lt;/b&gt; The majority of physicians don’t view the healthcare industry as a leader when it comes to digital adoption. More than half of physicians describe the healthcare industry as lagging behind the gaming (64%), telecommunications (56%), and financial services industries (53%). However, the healthcare industry is not seen to be trailing as much as it was last year behind retail (54% in 2020; 44% in 2021); hospitality and travel (53% in 2020; 43% in 2021); and the public sector (39% in 2020; 26% in 2021). &lt;/p&gt;&lt;p&gt;&lt;b&gt;Better interoperability alleviates physician burnout, improves health outcomes and speeds up diagnoses.&lt;/b&gt; The majority of physicians say increased data interoperability will cut the time to diagnosis for patients significantly (86%) and will ultimately help improve patient outcomes (95%.) In addition to better patient experiences and outcomes, more than half of physicians (54%) believe increased access to data via technology has had a positive impact on their healthcare organization overall. A majority believe that technology can alleviate the likelihood of physician “burn-out” (57%) and that efficient tools help decrease friction and stress (84%). And, as a result, 6 in 10 physicians say access to better technology and clinical data systems would allow them to have better work/life balance (60%) and that better access to/more complete patient data would reduce administrative burdens (61%). It is therefore not surprising that nearly 9 in 10 physicians (89%) say they are increasingly looking for ways to bring together all patient data into a single place for a more complete view of health. &lt;/p&gt;&lt;p&gt;&lt;b&gt;Familiarity with new Department of Health and Human Services (DHHS) interoperability rules grows, and many physicians are in favor.&lt;/b&gt; Most physicians (74%) say they have at least heard of the new DHHS rules (launched in 2019) to improve the interoperability of electronic health information. This is a clear rise from 2020 (64%), but deeper knowledge is fairly low. Only 30% of physicians say they are somewhat or very familiar with the new rules (though, again, this is a rise from 2020, when only 18% said they were very/somewhat familiar). Similar to in 2020, among those who have heard of the new rules, nearly half are in favor (48% in 2021; 45% in 2020) but a similar proportion remain unsure (46% in 2021; 50% in 2020). And like in 2020, by far the top potential benefit of the rules is thought to be forcing EHRs to be more interoperable with other systems (70%).&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="new interoperability rules electorinic health data.jpg" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/new_interoperability_rule.0981059417710934.max-1000x1000.jpg"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Google was founded on the idea that bringing more information to more people improves lives on a vast scale. In healthcare, that means creating tools and solutions that make data available in real time to help streamline operations and improve quality of care and patient outcomes. For example, our recently announced Healthcare Data Engine makes it easier for healthcare and life sciences leaders to make smart real-time decisions through clinical, operational, &amp;amp; groundbreaking scientific insights. To find out more about the Healthcare Data Engine, click &lt;a href="https://cloud.google.com/healthcare"&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;hr/&gt;&lt;p&gt;&lt;i&gt;&lt;sup&gt;Survey methodology: The 2021 survey was conducted online within the United States by The Harris Poll on behalf of Google Cloud from June 9 – 29, 2021 among 303 physicians who specialize in Family Practice, General Practice, or Internal Medicine, who treat patients, and are duly licensed in the state they practice. The 2020 survey was conducted from February 18 – 25, 2020 among 300 physicians who specialize in Family Practice, General Practice, or Internal Medicine, who treat patients, and are duly licensed in the state they practice. Physicians practicing in Vermont were excluded from the research. This online survey is not based on a probability sample and therefore no estimate of theoretical sampling error can be calculated. For complete survey methodology, including weighting variables and subgroup sample sizes, please contact press@google.com.&lt;/sup&gt;&lt;/i&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/healthcare-life-sciences/google-cloud-healthcare-interoperability-readiness-program/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/Google_Cloud_Healthcare.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Advancing healthcare with the Healthcare Interoperability Readiness Program&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;How Google Cloud is helping healthcare organizations achieve interoperability.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Mon, 09 Aug 2021 13:00:00 -0000</pubDate><guid>https://cloud.google.com/blog/topics/healthcare-life-sciences/google-cloud-covid-19-research/</guid><category>Google Cloud</category><category>Healthcare &amp; Life Sciences</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/images/2021_covid_researrch_google_cloud.max-600x600.jpg" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>New Research: COVID-19 accelerates innovation in healthcare but tech adoption still lags</title><description>Google Cloud and Harris Poll healthcare research reveals COVID-19 impacts on healthcare technology</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/images/2021_covid_researrch_google_cloud.max-600x600.jpg</image><site_name>Google</site_name><url>https://cloud.google.com/blog/topics/healthcare-life-sciences/google-cloud-covid-19-research/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Joe Miles</name><title>Managing Director, Healthcare and Life Sciences</title><department></department><company></company></author></item><item><title>Try a tutorial in the Google Cloud Console</title><link>https://cloud.google.com/blog/topics/developers-practitioners/try-tutorial-google-cloud-console/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Google Cloud Console" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_69.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;When it comes to learning how to implement some technology, we all have our own version of what I call the "tab game"—that is, your setup for all the tabs and windows you need open at once. You may have several monitors so you can see documentation, your IDE, and terminal windows at the same time. You may have several guides and references open at once in one window to get all the information you need.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Actual image of my computer while I'm trying to implement the code of a new product" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_68.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Personally, I like to work just from my laptop because I like to move around and work from various comfy spots. I think my tab game would probably enrage most devs because it involves a lot of swiping back and forth between windows *and* toggling tabs. It's not pretty. That is, it wasn't pretty until I discovered &lt;a href="https://console.cloud.google.com/?tutorial=firestore_add_data"&gt;tutorials in the Google Cloud Console&lt;/a&gt;!&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Here is one way to view some tutorials available in the Google Cloud Console." src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_67.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Jen really didn't know about tutorials in the Google Cloud Console?&lt;/h3&gt;&lt;p&gt;Yes, I honestly didn't know about them! I'm sharing about it because if I can work for Google and not know, then I can't be the only one, and it would be a shame to miss out on this because it's a brilliant idea. Also I wrote some pretty sweet tutorials for the console, but I swear that the main reason I'm telling you is because it's a cool thing!&lt;/p&gt;&lt;p&gt;There are several reasons that these tutorials are great:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;You can view the instructions and the console at the same time. No more playing the tab game!&lt;/li&gt;&lt;li&gt;The tutorials include links and highlights, making it easy to find the screens and buttons you're looking for&lt;/li&gt;&lt;li&gt;You can run code from Cloud Shell, so you don't need a separate window for an IDE&lt;/li&gt;&lt;li&gt;You can use the demo data provided to try things out, or you can apply the steps to your existing projects using data that suits your app's needs&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Here are some highlights of the highlights!" src="https://storage.googleapis.com/gweb-cloudblog-publish/original_images/neos_blog_firestore.gif"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Firestore tutorials&lt;/h3&gt;&lt;p&gt;I'm developing a series of tutorials in the Google Cloud Console designed to take you through everything you need to know about Firestore--from manually adding data in the Google Cloud Console to triggering Cloud Functions to make changes for you. Below are links and summaries for the currently available tutorials. Check back regularly to find the latest additions as they're released!&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="You, too, can see fun updates like these!" src="https://storage.googleapis.com/gweb-cloudblog-publish/original_images/unnamed_5.gif"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;&lt;a href="https://bit.ly/3gx32KR" target="_blank"&gt;Add Data to Firestore&lt;/a&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Enable Firestore on a project&lt;/li&gt;&lt;li&gt;Learn about the Firestore data model&lt;/li&gt;&lt;li&gt;Add a collection of documents&lt;/li&gt;&lt;li&gt;Add fields to a document&lt;/li&gt;&lt;li&gt;Delete documents and collections&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;a href="https://bit.ly/3iJTddG" target="_blank"&gt;Updating Data in Firestore using Node.js&lt;/a&gt; or &lt;a href="https://bit.ly/3eaABkb" target="_blank"&gt;using Python&lt;/a&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Add a collection of documents&lt;/li&gt;&lt;li&gt;Explore available data types&lt;/li&gt;&lt;li&gt;Replace the data of document&lt;/li&gt;&lt;li&gt;Replace fields in a document&lt;/li&gt;&lt;li&gt;Handle special cases: incrementing, timestamps, and arrays&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;a href="https://bit.ly/3ee5O5Z" target="_blank"&gt;Reading Data from Firestore using Node.js&lt;/a&gt; or &lt;a href="https://bit.ly/3r8hWdR" target="_blank"&gt;using Python&lt;/a&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Add a collection of documents&lt;/li&gt;&lt;li&gt;Explore available data types&lt;/li&gt;&lt;li&gt;Read a collection&lt;/li&gt;&lt;li&gt;Read a single document&lt;/li&gt;&lt;li&gt;Order documents&lt;/li&gt;&lt;li&gt;Query documents&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;a href="https://bit.ly/3ehR9GT" target="_blank"&gt;Transactions in Firestore using Node.js&lt;/a&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Add a collection of documents&lt;/li&gt;&lt;li&gt;Update data without a transaction to observe issue&lt;/li&gt;&lt;li&gt;Complete a transaction&lt;/li&gt;&lt;li&gt;Complete a batched write&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;a href="https://bit.ly/3B91zTh" target="_blank"&gt;Batched Writes in Firestore using Node.js&lt;/a&gt; or &lt;a href="https://bit.ly/3kbD6GI" target="_blank"&gt;using Python&lt;/a&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Use Cloud Shell and Cloud Shell Editor to write a Node.js or Python app&lt;/li&gt;&lt;li&gt;Complete a batched write&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;a href="https://bit.ly/3AZFpCQ" target="_blank"&gt;Firestore triggers for Cloud Functions&lt;/a&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Initialize Cloud Functions using the Firebase CLI&lt;/li&gt;&lt;li&gt;Write a Cloud Function triggered by a new document write to Firestore&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;a href="https://bit.ly/3eh2JC5" target="_blank"&gt;Offline Data in Firestore&lt;/a&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Add data to Firestore in the Cloud console Firestore dashboard&lt;/li&gt;&lt;li&gt;Create a web app that uses Firestore using the Firebase SDK&lt;/li&gt;&lt;li&gt;Deploy Firestore security rules that enable access to the required data&lt;/li&gt;&lt;li&gt;Enable data persistence in the web app&lt;/li&gt;&lt;li&gt;Observe app behavior with and without network connection&lt;/li&gt;&lt;/ul&gt;&lt;h3&gt;Chime in&lt;/h3&gt;&lt;p&gt;Is there a particular action or concept in Firestore that you'd like to see a tutorial for? Is there another Google Cloud product that you want to learn more about? Tweet &lt;a href="http://twitter.com/thatjenperson" target="_blank"&gt;@ThatJenPerson&lt;/a&gt; and you may just see your suggestion come to life in the Google Cloud Console!&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/databases/use-firestore-database-for-web-or-mobile-apps/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/Google_Cloud_Databases.max-500x500.jpg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Building scalable web applications with Firestore — new reference guide&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Google’s Firestore is a scalable, serverless document database that lets you build web or mobile apps. This new guide compares Firestore ...&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Fri, 06 Aug 2021 16:30:00 -0000</pubDate><guid>https://cloud.google.com/blog/topics/developers-practitioners/try-tutorial-google-cloud-console/</guid><category>Google Cloud</category><category>Developers &amp; Practitioners</category><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>Try a tutorial in the Google Cloud Console</title><description>You can follow tutorials and walkthroughs in the Google Cloud Console, allowing you to view the instructions and the console at the same time. The tutorials include links and highlights, making it easy to find the screens and buttons you're looking for.</description><site_name>Google</site_name><url>https://cloud.google.com/blog/topics/developers-practitioners/try-tutorial-google-cloud-console/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Jen Person</name><title>Senior Developer Relations Engineer</title><department></department><company></company></author></item><item><title>Introducing Unattended Project Recommender: discover, reclaim, or deprecate abandoned projects under your organization</title><link>https://cloud.google.com/blog/products/identity-security/google-cloud-launches-unattended-project-recommender/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;In fast-moving organizations, it's not uncommon for cloud resources, including entire projects, to occasionally be forgotten about. Not only such unattended resources can be difficult to identify, but they also tend to create a lot of headaches for product teams down the road, including unnecessary waste and security risks. &lt;/p&gt;&lt;p&gt;To help you prune your idle cloud resources, we’re excited to introduce &lt;a href="https://cloud.google.com/recommender/docs/unattended-project-recommender"&gt;Unattended Project Recommender&lt;/a&gt;. It’s a new feature of &lt;a href="https://cloud.google.com/solutions/active-assist"&gt;Active Assist&lt;/a&gt; that provides you with a one-stop shop for discovering, reclaiming, and shutting down unattended projects. With actionable and automatic recommendations, you no longer have to worry about wasting money or mitigating security risks presented by your idle resources. Unattended Project Recommender uses machine learning to identify, with a high degree of confidence, projects that are likely abandoned based on API and networking activity, billing, usage of cloud services, and other signals. This feature is available via the &lt;a href="https://cloud.google.com/recommender/docs/reference/rest"&gt;Recommender API&lt;/a&gt; today, making it easy for you to integrate with your company’s existing workflow management and communication tools, or &lt;a href="https://cloud.google.com/recommender/docs/bq-export/export-recommendations-to-bq"&gt;export results to a BigQuery table&lt;/a&gt; for custom analysis.&lt;/p&gt;&lt;h3&gt;Thousands of projects can be unattended in large organizations, presenting major security risks&lt;/h3&gt;&lt;p&gt;Your cloud projects can go abandoned or unattended for a number of reasons -- ranging from a test environment that’s no longer needed, to project cancellation, to project owner switching jobs, and more. Not only can such projects contribute to your cloud bill (waste) but they may contain security issues such as open firewalls or privileged service account keys that attackers can exploit to get a hold of your cloud resources for cryptocurrency mining or, worse, compromise your company’s sensitive data. These security risks tend to grow over time because the latest best practices and patches are usually not applied to unattended projects. &lt;/p&gt;&lt;p&gt;We experience this issue here at Google, too... In fact, it has been on Google’s internal security team’s radar for some time now, so we joined forces and looked into this problem together, starting with our very own &lt;b&gt;“google.com”&lt;/b&gt;organization cloud projects. We quickly found some projects that were unattended, but remediating this issue was easier said than done due to challenges in several areas:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Detection&lt;/b&gt;: With lots of signals available to you via sources like &lt;a href="https://cloud.google.com/monitoring/api/metrics"&gt;Cloud Monitoring&lt;/a&gt;, what are the right ones you should look at (e.g. API, networking, user activity)? How can you tell the difference between an unattended project and a project that has a low level of activity by design (e.g. a “shell” project that holds an auth token)?&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Remediation&lt;/b&gt;: Once you have identified a project that seems abandoned, how do you go about ensuring that it’s indeed an unattended project? How do you reduce the risk of deleting something that might be essential to a production workload, causing irreversible data loss? How do you solve this at the scale of your entire organization, beyond a one-time cleanup? &lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;Over the course of 2021 we built and tested a Google-internal prototype first, cleaning up many of our internal unattended projects, and then worked with a number of Google Cloud customers to build and tune this feature based on real-life data (thank you to all of our early adopters for working with us and your generous feedback that helped us shape this feature!) It was not uncommon for us to come across organizations with thousands of unattended projects, and we’re very excited to bring Unattended Project Recommender to all customers, in public preview.&lt;/p&gt;&lt;h3&gt;Discovering and acting on unattended project recommendations&lt;/h3&gt;&lt;p&gt;Unattended Project Recommender analyzes usage activity across all projects under your organization, including the following data:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;API activity (e.g. service accounts with authentication activity, API calls consumed)&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Networking activity (ingress and egress)&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Billing activity (e.g. services with billable usage)&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;User activity (e.g. active project owners)&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Cloud services usage (e.g. number of active VMs, BigQuery jobs, storage requests)&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;Based on these signals, it can generate recommendations to &lt;b&gt;clean up&lt;/b&gt; projects that have low usage activity (where “low usage” is defined using a machine learning model that ranks projects in your organization by level of usage), or recommendations to &lt;b&gt;reclaim&lt;/b&gt; projects that have high usage activity but no active project owners. Here’s what an example post-processed summary list of recommendations can look like for the “foobar” organization that has 3 projects:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;In addition to the recommendations, you can also examine the underlying project activity insights that the recommendations are based upon. The insights provide additional information that can be useful for integration with your organization’s existing workflows and automation (e.g. send an auto-generated email or chat message to project owners based on the list provided by the &lt;code&gt;owners&lt;/code&gt; field). Here’s an example insight payload:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;GCP projects are used in many different ways and for many different purposes. In case you get a recommendation to delete a project that's being used in a way that's out of the scope for this feature, you can dismiss the recommendation and it will stop showing up for the given project. &lt;/p&gt;&lt;h3&gt;Restoring deleted projects&lt;/h3&gt;&lt;p&gt;When you choose to shut down a project using the &lt;a href="https://cloud.google.com/resource-manager/reference/rest/v3/projects/delete"&gt;&lt;code&gt;projects.delete()&lt;/code&gt;&lt;/a&gt; method, it gets marked for deletion. After a project is marked for deletion, it becomes unusable, all resources within that project are shut down, and a 30-day wait period for the project and all of its data to get fully deleted begins.&lt;/p&gt;&lt;p&gt;In case a useful project is accidentally shut down, you have the &lt;a href="https://cloud.google.com/resource-manager/docs/creating-managing-projects#restoring_a_project"&gt;option to restore&lt;/a&gt; the project within that 30-day wait period. Since restoring allows you to recover most but &lt;a href="https://cloud.google.com/resource-manager/docs/creating-managing-projects#restoring_a_project"&gt;not necessarily all of your project data and resources&lt;/a&gt;, we recommend carefully examining the utilization insights associated with a project and considering any additional utilization signals that may not be captured by the Unattended Project Recommender before taking the cleanup action.&lt;/p&gt;&lt;h3&gt;Early customer success stories&lt;/h3&gt;&lt;p&gt;A number of enterprise customers are already using Unattended Project Recommender to keep their organizations clean of unattended projects and resources.&lt;/p&gt;&lt;p&gt;&lt;a href="https://www.decathlon.com/pages/our-story" target="_blank"&gt;&lt;b&gt;Decathlon&lt;/b&gt;&lt;/a&gt;, a French sporting goods retailer, is excited for the insight Unattended Project Recommender will bring to their environment, and are already deploying it as a part of their latest cloud security initiatives.&lt;/p&gt;&lt;p&gt;&lt;i&gt;"After a thorough test of this feature and the validation of our CISO, we ended up deleting our first 775 projects, and no one complained! A great help to improve our security. The next step for us will be to operationalize it at scale, and implement a company wide policy for unattended resource management."&lt;/i&gt; —&lt;b&gt;Adeline Villette, Cloud Security Officer&lt;/b&gt;&lt;/p&gt;&lt;p&gt;For &lt;a href="https://www.veolia.com/" target="_blank"&gt;&lt;b&gt;Veolia&lt;/b&gt;&lt;/a&gt;, one of the world’s largest water, waste and energy management companies, not only does this feature reduce security risks and waste, but also helps drive cultural shift and alignment with its ecological transformation strategy.&lt;/p&gt;&lt;p&gt;&lt;i&gt;“This feature allows us to reduce our costs and security debt on assets that are no longer in use, and is also fully in line with Veolia's philosophy of limiting its carbon footprint. After having tested Unattended Project Recommender on more than 3,000 projects throughout our organization, we are looking to bring it as proactive alerts to our project owners at scale.”&lt;/i&gt;—&lt;b&gt;Thomas Meriadec, Product Manager&lt;/b&gt;&lt;/p&gt;&lt;p&gt;&lt;a href="https://www.box.com/" target="_blank"&gt;&lt;b&gt;Box&lt;/b&gt;&lt;/a&gt;, a secure cloud content management provider, views it as a foundation for building a repeatable process to remediate unused resources.&lt;/p&gt;&lt;p&gt;&lt;i&gt;“Unattended Project Recommender is a great fit for us. It gives us a unified view of project usage across our entire organization and enables us to address security risks of legacy projects in a systematic and organized manner, ensuring an even safer environment.”&lt;/i&gt; —&lt;b&gt;Matt Bowes, Staff Security Engineer&lt;/b&gt;&lt;/p&gt;&lt;h3&gt;Getting started with the Unattended Project Recommender&lt;/h3&gt;&lt;p&gt;To help you get started, we’ve prepared a Cloud Shell tutorial (&lt;a href="https://github.com/GoogleCloudPlatform/cloud-shell-tutorials/blob/master/cloud-console-tutorials/active_assist_recommenders/unattended_project_recommender.md" target="_blank"&gt;source code&lt;/a&gt;) that you can use to find unattended project recommendations within your own Projects/Folders/Organization. Click this button to clone the tutorial from GitHub and run in your Cloud Shell environment:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;a href="https://console.cloud.google.com/?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2FGoogleCloudPlatform%2Fcloud-shell-tutorials&amp;amp;cloudshell_tutorial=cloud-console-tutorials%2Factive_assist_recommenders%2Funattended_project_recommender.md&amp;amp;page=shell" rel="external" target="_blank"&gt;&lt;img alt="google cloud shell.jpg" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/cloud_shell.1000014720000204.max-1000x1000.jpg"/&gt;&lt;/a&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;As you can see, listing recommendations for your projects only takes a few clicks with the tutorial (special thanks to &lt;a href="https://www.linkedin.com/in/olanrewaju-ogunmola/" target="_blank"&gt;Lanre Ogunmola&lt;/a&gt;, Security &amp;amp; Compliance Specialist, for making this look so easy)! For additional detail on using the gcloud CLI or API to discover unattended project recommendations, please refer to the &lt;a href="https://cloud.google.com/recommender/docs/unattended-project-recommender"&gt;documentation page&lt;/a&gt;.&lt;/p&gt;&lt;p&gt;You can also &lt;a href="https://cloud.google.com/recommender/docs/bq-export/export-recommendations-to-bq"&gt;automatically export all recommendations from your Organization to BigQuery&lt;/a&gt; and then investigate the recommendations with &lt;a href="https://support.google.com/datastudio/answer/6283323" target="_blank"&gt;DataStudio&lt;/a&gt; or &lt;a href="https://docs.looker.com/setup-and-management/database-config/google-bigquery" target="_blank"&gt;Looker&lt;/a&gt;, or use &lt;a href="https://cloud.google.com/bigquery/docs/connected-sheets"&gt;Connected Sheets&lt;/a&gt; that let you use Google Workspace Sheets to interact with the data stored in BigQuery without having to write SQL queries.&lt;/p&gt;&lt;p&gt;As with any other Recommender, you can choose to &lt;a href="https://cloud.google.com/recommender/docs/opting-out"&gt;opt out of data processing&lt;/a&gt; at any time by disabling the appropriate data groups in the &lt;b&gt;Transparency &amp;amp; control&lt;/b&gt; tab under &lt;a href="https://console.cloud.google.com/iam-admin/privacy"&gt;&lt;b&gt;Privacy &amp;amp; Security&lt;/b&gt;&lt;/a&gt; settings.&lt;/p&gt;&lt;p&gt;We hope that you can leverage Unattended Project Recommender to improve your cloud security posture and reduce cost, and can’t wait to hear your feedback and thoughts about this feature! Please feel free to reach us at &lt;a href="mailto:active-assist-feedback@google.com"&gt;active-assist-feedback@google.com&lt;/a&gt; and we also invite you to sign up for our &lt;a href="https://docs.google.com/forms/d/e/1FAIpQLSdRPpsVfSW5k-xO3B36B3iIcEWAg9rJF7OBZj_50Ge_oPRGBw/viewform" target="_blank"&gt;Active Assist Trusted Tester Group&lt;/a&gt; if you would like to get early access to the newest features as they are developed.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/products/management-tools/active-assist-comes-to-google-cloud/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/Google_Cloud-01_xyGPYQS.max-500x500.png')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Introducing Active Assist: Reduce complexity, maximize your cloud ROI&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Introducing Active Assist, a family of tools to help you easily optimize your Google Cloud environment.&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Fri, 06 Aug 2021 16:00:00 -0000</pubDate><guid>https://cloud.google.com/blog/products/identity-security/google-cloud-launches-unattended-project-recommender/</guid><category>Google Cloud</category><category>Identity &amp; Security</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/images/Unattended_Project_Recommender.max-600x600.jpg" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>Introducing Unattended Project Recommender: discover, reclaim, or deprecate abandoned projects under your organization</title><description>Save money and improve security by automating the discovery, management and reclamation of old projects with Unattended Project Recommender.</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/images/Unattended_Project_Recommender.max-600x600.jpg</image><site_name>Google</site_name><url>https://cloud.google.com/blog/products/identity-security/google-cloud-launches-unattended-project-recommender/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Dima Melnyk</name><title>Product Manager</title><department></department><company></company></author><author xmlns:author="http://www.w3.org/2005/Atom"><name>Bakh Inamov</name><title>Product Manager</title><department></department><company></company></author></item><item><title>New This Month in Data Analytics: Taking Home the Gold in Data</title><link>https://cloud.google.com/blog/products/data-analytics/new-month-data-analytics-taking-home-gold-data/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;As the Olympics kicked off in Tokyo at the end of July, we found ourselves reflecting on the beauty of diverse countries and cultures coming together to celebrate greatness and sportsmanship. For this month’s blog, we’d like to highlight some key data and analytics performances that should help inspire you to reach new heights in your data journey.&lt;/p&gt;&lt;p&gt;Let’s review the highlights!&lt;/p&gt;&lt;h2&gt;BigQuery ML Anomaly Detection: A perfect 10 for augmented analytics&lt;/h2&gt;&lt;p&gt;Identifying anomalous behavior at scale is a critical component of any analytics strategy. Whether you want to work with a single frame of data or a time series progression, &lt;a href="https://cloud.google.com/bigquery-ml/docs/introduction"&gt;BigQuery ML&lt;/a&gt; allows you to bring the power of machine learning to your data warehouse. &lt;/p&gt;&lt;p&gt;In &lt;a href="https://cloud.google.com/blog/products/data-analytics/bigquery-ml-unsupervised-anomaly-detection"&gt;this blog&lt;/a&gt; released at the beginning of last month, our team walked through both non-time series and time-series approaches to anomaly detection in BigQuery ML:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;Non-time series anomaly detection&lt;/p&gt;&lt;/li&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-create-autoencoder"&gt;Autoencoder model&lt;/a&gt; (now in Public Preview)&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-create-kmeans"&gt;K-means model&lt;/a&gt; (already in GA)&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;li&gt;&lt;p&gt;Time-series anomaly detection&lt;/p&gt;&lt;/li&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-create-time-series"&gt;ARIMA_PLUS time series model&lt;/a&gt; (already in GA)&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/ul&gt;&lt;p&gt;These approaches make it easy for your team to quickly experiment with data stored in &lt;a href="https://cloud.google.com/bigquery"&gt;BigQuery&lt;/a&gt; to identify what works best for your particular anomaly detection needs. Once a model has been identified as the right fit, you can easily port that model into the &lt;a href="https://cloud.google.com/vertex-ai"&gt;Vertex AI platform&lt;/a&gt; for real-time analysis or schedule it in BigQuery for continued batch processing.&lt;/p&gt;&lt;h2&gt;App Analytics: Winning the team event&lt;/h2&gt;&lt;p&gt;Google provides a broad ecosystem of technologies and services aimed at solving modern day challenges. Some of the best solutions come when those technologies are combined with our data analytics offerings to surface additional insights and provide new opportunities. &lt;/p&gt;&lt;p&gt;Firebase has deep adoption in the app development community and provides the technology backbone for many organization’s app strategy. This month we &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/creating-unified-analytics-platform-digital-natives"&gt;launched a design pattern&lt;/a&gt; that shows Firebase customers how to use &lt;a href="https://firebase.google.com/products/crashlytics" target="_blank"&gt;Crashlytics&lt;/a&gt; data, CRM, issue tracking, and support data in BigQuery and Looker to identify opportunities to improve app quality and enhance customer experiences.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Image 3" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/image3_ZTOwru6.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h2&gt;Crux on BigQuery: Taking gold in the all-around data competition&lt;/h2&gt;&lt;p&gt;&lt;a href="https://www.cruxinformatics.com/" target="_blank"&gt;Crux Informatics&lt;/a&gt; provides data services to many large companies to help their customers make smarter business decisions. While they were already operating on a modern stack and not on the hunt for a modern data warehouse, BigQuery became an enticing option due to performance and a more optimal pricing model. Crux also found advantages with lower-cost ingestion and processing engines like &lt;a href="https://cloud.google.com/dataflow"&gt;Dataflow&lt;/a&gt; that allow for streaming analytics.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-pull_quote"&gt;&lt;div class="uni-pull-quote h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;div class="uni-pull-quote__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;div class="uni-pull-quote__inner-wrapper h-c-copy h-c-copy"&gt;&lt;q class="uni-pull-quote__text"&gt;... when it came to building a centralized large-scale data cloud, we needed to invest in a solution that would not only suit our current data storage needs but also enable us to tackle what’s coming, supporting a massive ecosystem of data delivery and operations for thousands of companies.&lt;/q&gt; &lt;cite class="uni-pull-quote__author"&gt;&lt;span class="uni-pull-quote__author-meta"&gt;&lt;strong class="h-u-font-weight-medium"&gt;Mark Etherington&lt;/strong&gt;&lt;br/&gt; Chief Technology Office, Crux Informatics&lt;/span&gt;&lt;/cite&gt;&lt;/div&gt;&lt;/div&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Technology is a team sport, and Crux found our support team responsive and ready to help. This decision to more deeply adopt Google Cloud’s data analytics offerings provides Crux with the flexibility to manage a constantly evolving data ecosystem and stay competitive.&lt;/p&gt;&lt;p&gt;You can read more about Crux’s decision to adopt BigQuery in &lt;a href="https://cloud.google.com/blog/products/data-analytics/crux-chooses-bigquery-for-rock-solid-data-delivery"&gt;this blog&lt;/a&gt;.&lt;/p&gt;&lt;h2&gt;Google Trends: A classic emerges a champion&lt;/h2&gt;&lt;p&gt;Following up on the &lt;a href="https://cloud.google.com/blog/products/data-analytics/top-25-google-search-terms-now-in-bigquery"&gt;launch of our Google Trends dataset&lt;/a&gt; in June, we delivered some examples of how to use that data to augment your decision making. &lt;/p&gt;&lt;p&gt;As a quick recap of that dataset, Google Cloud, and in particular BigQuery, provide access to the top 25 trending terms by Nielsen’s &lt;a href="https://www.nielsen.com/us/en/contact-us/intl-campaigns/dma-maps/" target="_blank"&gt;Designated Market Area®&lt;/a&gt; (DMA) with a weekly granularity. These trending terms are based on search patterns and have historically only been available on the &lt;a href="https://trends.google.com/" target="_blank"&gt;Google Trends website&lt;/a&gt;.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-video"&gt;&lt;div class="article-module article-video "&gt;&lt;figure&gt;&lt;a class="h-c-video h-c-video--marquee" data-glue-modal-disabled-on-mobile="true" data-glue-modal-trigger="uni-modal-9FJAXMF0ASc-" href="https://youtube.com/watch?v=9FJAXMF0ASc"&gt;&lt;img alt="Google Trends" src="//img.youtube.com/vi/9FJAXMF0ASc/maxresdefault.jpg"/&gt;&lt;svg class="h-c-video__play h-c-icon h-c-icon--color-white" role="img"&gt;&lt;use xlink:href="#mi-youtube-icon"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/a&gt;&lt;/figure&gt;&lt;/div&gt;&lt;div class="h-c-modal--video" data-glue-modal="uni-modal-9FJAXMF0ASc-" data-glue-modal-close-label="Close Dialog"&gt;&lt;a class="glue-yt-video" data-glue-yt-video-autoplay="true" data-glue-yt-video-height="99%" data-glue-yt-video-vid="9FJAXMF0ASc" data-glue-yt-video-width="100%" href="https://youtube.com/watch?v=9FJAXMF0ASc" ng-cloak=""&gt;&lt;/a&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;The Google Trends &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/make-informed-decisions-google-trends-data"&gt;design pattern&lt;/a&gt; addresses some common business needs, such as identifying what’s trending geographically near your stores and how to match trending terms to products to identify potential campaigns. &lt;/p&gt;&lt;h2&gt;Dataflow GPU: More power than ever for those streaming sprints&lt;/h2&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/dataflow"&gt;Dataflow&lt;/a&gt; is our fully-managed data processing platform that supports both batch and streaming workloads. The ability of Dataflow to scale and easily manage unbounded data has made it the streaming solution of choice for large workloads with high-speed needs in Google Cloud. &lt;/p&gt;&lt;p&gt;But what if we could take that speed and provide even more processing power for advanced use cases? Our team, in partnership with &lt;a href="https://developer.nvidia.com/blog/accelerating-machine-learning-model-inference-on-google-cloud-dataflow-with-nvidia-gpus/" target="_blank"&gt;NVIDIA&lt;/a&gt;, did just that by &lt;a href="https://cloud.google.com/blog/products/data-analytics/give-your-data-pipelines-a-boost-with-gpus"&gt;adding GPU support to Dataflow&lt;/a&gt;. This allows our customers to easily accelerate compute-intensive processing like image analysis and predictive forecasting with amazing increases in efficiency and speed. &lt;/p&gt;&lt;p&gt;Take a look at the times below:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Table" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/image1_jiHQaRm.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h2&gt;Data Fusion: A play-by-play for data integration’s winning performance&lt;/h2&gt;&lt;p&gt;&lt;a href="https://cloud.google.com/data-fusion"&gt;Data Fusion&lt;/a&gt; provides Google Cloud customers with a single place to perform all kinds of data integration activities. Whether it’s ETL, ELT, or simply integrating with a cloud application, Data Fusion provides a clean UI and streamlined experience with deep integrations to other Google Cloud data systems. Check out &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/bridge-data-silos-data-fusion"&gt;our team’s review&lt;/a&gt; of this tool and the capabilities it can bring to your organization.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Table" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/image4_x5kaf59.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Fri, 06 Aug 2021 09:17:00 -0000</pubDate><guid>https://cloud.google.com/blog/products/data-analytics/new-month-data-analytics-taking-home-gold-data/</guid><category>Data Analytics</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/images/GCP_Data_Analytics_DpHUrLo.max-600x600.jpg" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>New This Month in Data Analytics: Taking Home the Gold in Data</title><description>Highlights from the July blogs for Data Analytics.</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/images/GCP_Data_Analytics_DpHUrLo.max-600x600.jpg</image><site_name>Google</site_name><url>https://cloud.google.com/blog/products/data-analytics/new-month-data-analytics-taking-home-gold-data/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Sudhir Hasbe</name><title>Sr. Director, Product Management</title><department></department><company></company></author></item><item><title>MySQL major version upgrade using Database Migration Service</title><link>https://cloud.google.com/blog/topics/developers-practitioners/mysql-major-version-upgrade-using-database-migration-service/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;As hopefully you all know by now, &lt;a href="https://cloud.google.com/blog/products/databases/database-migration-service-now-available-for-cloud-sql-and-more"&gt;migrating your SQL database&lt;/a&gt; got easier this past November when Google’s Database Migration Service went GA for MySQL. If you didn’t know, migrating your SQL database to Cloud SQL is now easier! Something you might have noticed, or I’ll happily point out if you haven’t noticed, is that Database Migration Service lets you pull from a number of different sources, including on premises, AWS, and even Cloud SQL for MySQL.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Source" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_57.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Side plug - we just announced &lt;a href="https://cloud.google.com/architecture/postgresql-migration-with-database-migration-service"&gt;DMS supports Cloud SQL for PostgreSQL&lt;/a&gt; as a source, so you can use DMS for your PostgreSQL major version upgrades too. But THIS post is about MySQL...&lt;/p&gt;&lt;p&gt;Because something ELSE I wanted to call out... notice this, it appears on the “Create a destination” page of the DMS process:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="MySQL 5.7" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_58.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;So if you’re following along here, I’m telling you that you can ALSO use DMS to do major version upgrades on your MySQL database. We’ll all still wait with baited breath for in-place upgrades of course, but while we’re waiting this is a great option.&lt;/p&gt;&lt;p&gt;There’s a catch, because of course there is. The key here is that because we’re using a migration tool to upgrade, there’s two dimensions of complications to deal with. We have the migration piece and the upgrade piece. DMS helps manage a good amount of this, but there’s still some things you need to consider. In this post, I’m going to pull together all the pieces you need to think about and link to everything you need to do a major version upgrade for MySQL using Database Migration Service.&lt;/p&gt;&lt;h3&gt;Why are we here?&lt;/h3&gt;&lt;p&gt;Just to cover why you might want to upgrade MySQL in the first place. Between performance upgrades and feature updates, there’s plenty of reasons to do it. If you need convincing, here’s a nice list of the &lt;a href="https://dev.mysql.com/doc/refman/8.0/en/mysql-nutshell.html" target="_blank"&gt;enhancements in MySQL 8.0&lt;/a&gt;. There’s also the elephant in the room: officially 5.6 was deprecated this past February. You might have been on 5.6 for the last 8 years, saw it’s officially end-of-lifed this year and are panicking a little bit. The good news here is that Cloud SQL will support 5.6 for a while longer, but that doesn’t mean now isn’t a good time to upgrade.&lt;/p&gt;&lt;h3&gt;Version compatibility&lt;/h3&gt;&lt;p&gt;The first thing to look at is what versions you’re upgrading between. So 5.6 to 5.7 or 5.7 to 8.0. 5.6 to 8.0 is right out. MySQL has significant changes between major versions that are likely to break compatibility, so you need to triple check your database for some of these incompatible changes.&lt;/p&gt;&lt;p&gt;For example, between 5.6 and 5.7 you need to keep an eye out for any system or status variables in the &lt;code&gt;INFORMATION_SCHEMA&lt;/code&gt; tables in your 5.6 database. Those were all replaced by the Performance Schema tables in 5.7.6. There are also a lot of little things–like if you have a column with the YEAR(2) data type, you’ll need to update all those values to a 4-digit YEAR column before you’ll be able to use those columns again. If you’re going 5.6 to 5.7, you can go over the full list of changes &lt;a href="https://dev.mysql.com/doc/refman/5.7/en/upgrading-from-previous-series.html" target="_blank"&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;p&gt;Between 5.7 and 8.0 of course there are yet more changes to watch out for. Big ones for me are that default flags were changed quite a bit between the two. While a lot of them might not break things with a segmentation fault, they could lead to some unintended behavior in your application. Also, you might want to take a peek at your AUTO_INCREMENT columns. It’s been deprecated for FLOAT and DOUBLE types. For a full list of what changed between these two versions, check out &lt;a href="https://dev.mysql.com/doc/refman/8.0/en/upgrading-from-previous-series.html" target="_blank"&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;h3&gt;Connecting the dots&lt;/h3&gt;&lt;p&gt;If you’re planning on doing the major version upgrade from one Cloud SQL instance to another, you can skip this section because you’ve already done what I’m going to talk about. If, however, you’re changing from an on premises database, or in some small edge cases, a GCE (virtual machine) instance to Cloud SQL, there’s a few extra things to watch out for.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="watch out for" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_59.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;#1 thing is latency. If you’ve gotten comfortable having a near-instantaneous response from your database because it lived right next to your application, be prepared for a bit of a wakeup call. Unless your application also lives in the Cloud, there’s now some extra time that will pass between your application and your database. This might be totally fine! Or, it could introduce some really difficult-to-debug re-entrancy bugs into your application. A colleague of mine wrote a great post that covers all the things you need to watch out and potentially change for when moving your database to Cloud SQL from another source &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/preparing-your-mysql-database-migration-database-migration-service"&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;p&gt;#2 is how do I even now get to my database?! Network connectivity can be really difficult. Firewalls, routing and DNS can all throw a wrench in the works. You had everything all nicely connected before you did this, and now all of a sudden your database lives (potentially) in an entirely different part of the internet with different requirements to connect to it. This particular rabbit hole I’ve gone very deep on already. I wrote an in-depth blog post on connecting source to destination instances with Database Migration Service &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/database-migration-service-connectivity-technical-introspective"&gt;here&lt;/a&gt;. If you run into problems with connectivity, that post should cover most situations.&lt;/p&gt;&lt;h3&gt;More migrating!&lt;/h3&gt;&lt;p&gt;DMS makes the moving your data piece of the process easy, but since we rely on the database engine’s native migration mechanisms, we’re also subject to its limitations. This means there’s some steps to complete once you’re done migrating your database regardless of where it came from. For example, you know, your IP address will change. Because we’re using a migration method to upgrade our version, we’re instantiating a new instance. Once we get in-place upgrades this is a non-issue, but in the meantime we have to remember to update any applications, load-balancers, etc that are pointing at the old IP address.&lt;/p&gt;&lt;p&gt;Another thing is, don’t forget, your users are stored in system tables that aren’t migrated either. Those will need to be brought over once the DMS portion is complete. There’s a few more bits and bobs, and there’s a complete post &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/closing-gap-migration-completeness-when-using-database-migration-service"&gt;here&lt;/a&gt; that lists the things you’ll need to finish off before your new instance is ready for use.&lt;/p&gt;&lt;h3&gt;What’s next?&lt;/h3&gt;&lt;p&gt;Now that you’ve got your newly upgraded instance all set, it’s time to get it ready for production. There are a few things that you’ll likely want that can’t be set up until the Cloud SQL instance is fully online.&lt;/p&gt;&lt;p&gt;By default, for example, SSL-only connections are off by default and we know many organizations enforce SSL-only connections. This will need to be enabled on the server and certificates will need to be set up. This can be done after the data is migrated but before the DMS target instance is promoted.&lt;/p&gt;&lt;p&gt;Disaster recovery is another must-have for production instances. It’s also a requirement for our SLA which guarantees uptime. Absolutely need to set this up. If you edit your instance it’ll be on the main Overview page:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="Single Zone" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_60.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Setting to “Multiple zones” enables this feature.&lt;/p&gt;&lt;p&gt;Also falling into this category are read replicas. Optimizing your data infrastructure to support your application is key and this is an integral part of that. If you have a read-heavy application, you’ll want to set up your read replicas at this stage of the game as well.&lt;/p&gt;&lt;p&gt;Setting up high availability and the read replicas can only be done after you’ve promoted the instance at the end of the DMS process.&lt;/p&gt;&lt;p&gt;One last bit before I leave you. You’ve done all this work to upgrade your instance to the next version. You’ve set up all the extra pieces you need. You’ve connected your application and it can totally talk to your database. Now is NOT the time to hit the push to prod button. This may absolutely come across as obvious to many of you reading this. But I’m going to say it anyway. TEST! Test all the things. Test all the edge cases you can think of. Test adding even more latency into your system arbitrarily to be sure that you can handle random internet lag spikes between application and database. The high availability feature once it’s on allows you to &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/testing-cloud-sql-failover-where-begin"&gt;test automatic failover&lt;/a&gt; to simulate your primary instance going down and the fallback instance taking over. So test. Test all these scenarios to ensure that you have a smooth launch of your new major version SQL instance.&lt;/p&gt;&lt;p&gt;Hopefully this all helped to guide you to successfully using Database Migration Service to upgrade your MySQL instance to a new version! If you have questions or comments, please feel free to reach out to me on &lt;a href="https://www.twitter.com/GabeWeiss_" target="_blank"&gt;Twitter&lt;/a&gt;, my DMs are open! Thanks for reading.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/developers-practitioners/closing-gap-migration-completeness-when-using-database-migration-service/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;Closing the gap: Migration completeness when using Database Migration Service&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;Learn what is and isn't included when migrating a MySQL database to Cloud SQL using Database Migration Service (DMS).&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Thu, 05 Aug 2021 22:00:00 -0000</pubDate><guid>https://cloud.google.com/blog/topics/developers-practitioners/mysql-major-version-upgrade-using-database-migration-service/</guid><category>Google Cloud</category><category>Databases</category><category>Developers &amp; Practitioners</category><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>MySQL major version upgrade using Database Migration Service</title><description>Google’s Database Migration Service gives us the tool we need to perform Major Version upgrades for MySQL.</description><site_name>Google</site_name><url>https://cloud.google.com/blog/topics/developers-practitioners/mysql-major-version-upgrade-using-database-migration-service/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Gabe Weiss</name><title>Developer Advocate</title><department></department><company></company></author></item><item><title>Beyond mainframe modernization: The art of possibilities</title><link>https://cloud.google.com/blog/topics/developers-practitioners/beyond-mainframe-modernization-art-possibilities/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Mainframe modernization has been a hot topic over the past decade or so. &lt;a href="https://analyticsindiamag.com/is-mainframes-a-dying-technology/" target="_blank"&gt;Many people&lt;/a&gt; have predicted the demise of mainframe times. But the mainframe is standing tall, strong and growing. Over time, the term "modernization" itself is manifested in &lt;a href="https://www.planetmainframe.com/2018/10/so-what-does-mainframe-modernization-really-mean/" target="_blank"&gt;many ways&lt;/a&gt;. So to even begin the modernization conversation, we need to define what modernization is. &lt;/p&gt;&lt;p&gt;As Henry Ford once said, “If I had asked people what they wanted, they would have said faster horses.” Instead, Ford invented an automobile for the masses to improve their lives. So modernization is akin to evolution, which in hindsight reflects the progress humanity made in lifestyle, resilience, knowledge, communication, and, in general, wisdom.&lt;/p&gt;&lt;p&gt;Similarly, modernization in mobile phones meant lighter weight, wider screen, faster speed, and more storage. And then the iPhone was released in 2007, which showed innovation and changed the way humans interact with digital devices. &lt;/p&gt;&lt;h3&gt;Deciding on mainframe modernization&lt;/h3&gt;&lt;p&gt;The decision to embark on a mainframe modernization journey typically lies with three stakeholders: infrastructure owners, application owners, and business owners. For a long time, mainframe modernization has focused on solving infrastructure and application problems like higher cost, lack of skillset availability, locked-in data, and poor integration with modern tools and products.In other words, mainframe modernization has been only focused on the infrastructure and application layer. &lt;/p&gt;&lt;p&gt;In this paper, we try to understand the importance of ‘business owner’, who is the most critical stakeholder and sponsor.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="table of experience" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/Screen_Shot_2021-08-03_at_1.45.02_PM.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Journey depends on disposition for data and applications&lt;/h3&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;a href="https://storage.googleapis.com/gweb-cloudblog-publish/images/journey.0813042115720619.max-1300x1300.png" rel="external" target="_blank"&gt;&lt;img alt="journey" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/journey.0800037916000703.max-1000x1000.png"/&gt;&lt;/a&gt;&lt;figcaption class="article-image__caption "&gt;&lt;div class="rich-text"&gt;Click to enlarge&lt;/div&gt;&lt;/figcaption&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;For a long time, the mainframe modernization journey has been defined as just moving the workload off the mainframe, with little or no focus on business amplification. As a result, there has not been much success in either commencing projects like these or in completing them. &lt;/p&gt;&lt;p&gt;But mainframe modernization should be looked at from a business imperative lens. You need to define a goal before embarking on the transformation path. This will help identify the right partner and trusted advisor, optimize the overall effort, and minimize the number of hops to get to that goal.&lt;/p&gt;&lt;p&gt;Google’s approach to &lt;a href="https://cloud.google.com/solutions/mainframe-modernization"&gt;mainframe modernization&lt;/a&gt; revolves around bringing business needs to the forefront of modernization initiatives. Let’s look at these one by one:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;One Google&lt;/b&gt;: As part of the larger Google ecosystem, it’s quite easy and seamless for Google Cloud services to integrate with products like Maps, Search, and Ads and hence be able to provide a very personalized and enhanced user experience. Most enterprise customers are existing subscribers to one or more of these services, so it’s also easy for them to integrate Google Cloud services.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Data-first modernization approach&lt;/b&gt;: Google offers a data-first approach to mainframe modernization that involves moving ‘system of records’ or ‘system of engagement’ from a mainframe to Google Cloud datastores. This approach enables you to leverage and unlock the data in many ways:&lt;/p&gt;&lt;/li&gt;&lt;ul&gt;&lt;li&gt;&lt;p&gt;Enabling business users, aka citizen developers, to build apps leveraging data and tools like &lt;a href="https://cloud.google.com/appsheet"&gt;AppSheet&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Reducing mainframe consumption by routing web and mobile apps to data on Google Cloud.&lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;Including the data that was moved from the mainframe to Google Cloud in advanced analytics to understand customer behavior and come up with market-relevant products.&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;AI/ML on Google Cloud&lt;/b&gt;: When it comes to extracting info and wisdom from raw data using artificial intelligence and machine learning, there’s no parallel to Google. And thereby our customers get a significant competitive advantage in understanding market trends and behavioral patterns. This enables them to meet and exceed customer requirements while providing a very personalized experience.  &lt;/p&gt;&lt;/li&gt;&lt;li&gt;&lt;p&gt;&lt;b&gt;Industry-specific solutions&lt;/b&gt;: Google has been investing heavily in business-specific industry solutions. With innovation deep rooted in its DNA and culture, Google has always been looking at things, starting from search, outside the box. One example is &lt;a href="https://deepmind.com/blog/article/AlphaFold-Using-AI-for-scientific-discovery" target="_blank"&gt;AlphaFold&lt;/a&gt;, which is an artificial intelligence program created by Google's &lt;a href="https://deepmind.com/" target="_blank"&gt;DeepMind&lt;/a&gt; that performs predictions of protein structure. Another example is &lt;a href="https://verily.com/" target="_blank"&gt;Verily&lt;/a&gt;, whose mission is to make the world's health data useful so that people enjoy longer and healthier lives. Similarly, &lt;a href="https://waymo.com/" target="_blank"&gt;Waymo&lt;/a&gt; is focused on fully autonomous driving, &lt;a href="https://wing.com/" target="_blank"&gt;Wing&lt;/a&gt; is focused on drone delivery, and &lt;a href="https://cloud.google.com/customers/route4me-route-optimization"&gt;Route4Me&lt;/a&gt; provides easy-to-use driving route optimization for businesses. In financial services, there are &lt;a href="https://pay.google.com/" target="_blank"&gt;Google Pay&lt;/a&gt;, &lt;a href="https://cloud.google.com/solutions/lending-doc-ai"&gt;Lending DocAI&lt;/a&gt;, &lt;a href="https://cloud.google.com/solutions/procurement-doc-ai"&gt;Procurement DocAI&lt;/a&gt;, &lt;a href="https://cloud.google.com/blog/products/data-analytics/how-to-build-a-fraud-detection-solution"&gt;Real-time credit card fraud detection&lt;/a&gt;, and so on.&lt;/p&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;All these business amplifications and enhancements come with state-of-the-art cloud infrastructure:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Hardware security&lt;/b&gt;: Google OpenTitan is the first open source project that builds a transparent, high-quality reference design and integration guidelines for silicon root of trust (RoT) chips.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Data security&lt;/b&gt;: The user owns the data. They set the rules and decide who needs to access what. Google Cloud encrypts data not only at rest and in motion but also during processing.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Latency&lt;/b&gt;: With an in-house network of optical fibers and undersea cables, there’s significantly less latency compared to others. Google search runs on the same platform.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Scalability&lt;/b&gt;: Alphabet has 9 products—Android, Chrome, Gmail, Drive, Maps, Search, the Play Store, YouTube and Photos—with more than a billion users each. These run on Google Cloud providing an uninterrupted experience to all users. &lt;/li&gt;&lt;li&gt;&lt;b&gt;100% carbon neutral footprint&lt;/b&gt;: Helps customers fulfill their obligations towards carbon neutrality and help save the planet earth.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;Many of the large deals like &lt;a href="https://cloud.google.com/press-releases/2020/0121/sabre-and-google-cloud-partnership"&gt;Sabre&lt;/a&gt;, &lt;a href="https://cloud.google.com/press-releases/2020/1204/deutsche-bank-google-cloud"&gt;Deutsche Bank&lt;/a&gt;, &lt;a href="https://cloud.google.com/press-releases/2021/0330/albertsons-google-partnership"&gt;Albertsons Companies&lt;/a&gt; and &lt;a href="https://cloud.google.com/press-releases/2021/0209/google-cloud-telus"&gt;Telus&lt;/a&gt; are testament of Google’s focus on amplifying and redefining business with our customers. With the advent of Google Cloud’s focus on mainframe modernization, business stakeholders are finding value in business transformation. Instead of looking at the modernization from an ‘IT lens’ and decommissioning the mainframe, they found a partner in Google who strives to not only solve the current problems but also define a ‘North Star’ for the enterprise and collaborate to achieve those goals. &lt;/p&gt;&lt;h3&gt;Conclusion&lt;/h3&gt;&lt;p&gt;Wrapping the thought with automobiles—Tesla showed the possibility that an automobile can be software driven, independent of fossil fuel, and contribute to reducing carbon footprint at the same time. Having said that, this has only been possible with the revolutionary or evolutionary journey started by Mr. Ford.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Thu, 05 Aug 2021 20:00:00 -0000</pubDate><guid>https://cloud.google.com/blog/topics/developers-practitioners/beyond-mainframe-modernization-art-possibilities/</guid><category>Google Cloud</category><category>Developers &amp; Practitioners</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/images/journey.0800037916000703.max-600x600.png" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>Beyond mainframe modernization: The art of possibilities</title><description>Mainframe modernization has been a hot topic over the past decade or so. Over time, the term "modernization" itself is manifested in many ways. So to even begin the modernization conversation, we need to define what modernization is.</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/images/journey.0800037916000703.max-600x600.png</image><site_name>Google</site_name><url>https://cloud.google.com/blog/topics/developers-practitioners/beyond-mainframe-modernization-art-possibilities/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Aman Gupta</name><title>Mainframe Solutions Specialist</title><department></department><company></company></author></item><item><title>BigQuery Admin reference guide: Query optimization</title><link>https://cloud.google.com/blog/topics/developers-practitioners/bigquery-admin-reference-guide-query-optimization/</link><description>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Last week in the BigQuery reference guide, we walked through query execution and how to leverage the query plan. This week, we’re going a bit deeper - covering more advanced queries and tactical optimization techniques.  Here, we’ll walk through some query concepts and describe techniques for optimizing related SQL. &lt;/p&gt;&lt;h2&gt;Filtering data&lt;/h2&gt;&lt;p&gt;From last week’s post, you already know that the execution details for a query show us how much time is spent reading data (either from persistent storage, federated tables or from the memory shuffle) and writing data (either to the memory shuffle or to persistent storage). Limiting the amount of data that is used in the query, or returned to the next stage, can be instrumental in making the query faster and more efficient. &lt;/p&gt;&lt;h3&gt;Optimization techniques&lt;/h3&gt;&lt;p&gt;&lt;b&gt;1. Necessary columns only:&lt;/b&gt; Only select the columns necessary, especially in inner queries. SELECT * is cost inefficient and may also hurt performance. If the number of columns to return is large, consider using SELECT * EXCEPT to exclude unneeded columns.&lt;/p&gt;&lt;p&gt;&lt;b&gt;2. Auto-pruning with partitions and clusters:&lt;/b&gt; Like we mentioned in our &lt;a href="https://cloud.google.com/blog/topics/developers-practitioners/bigquery-admin-reference-guide-storage"&gt;post on BigQuery storage&lt;/a&gt;, partitions and clusters are used to segment and order the data. Using a filter on columns that the data is partitioned or clustered on can drastically reduce the amount of data scanned. &lt;/p&gt;&lt;p&gt;&lt;b&gt;3. Expression order matters:&lt;/b&gt; BigQuery assumes that the user has provided the best order of expressions in the WHERE clause, and does not attempt to reorder expressions. Expressions in your WHERE clauses should be ordered with the most selective expression first.  The optimized example below is faster because it doesn’t execute the expensive LIKE expression on the entire column content, but rather only on the content from user, ‘anon’.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="anon" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_61.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;b&gt;4. Order by with limit:&lt;/b&gt;Writing results for a query with an ORDER BY clause can result in Resources Exceeded errors. Since the final sorting must be done on a single worker, ordering a large result set can overwhelm the slot that is processing the data. If you are sorting a large number of values use a LIMIT clause, which will filter the amount of data passed onto the final slot. &lt;br/&gt;&lt;h3&gt;Understanding aggregation&lt;/h3&gt;&lt;p&gt;In an aggregation query, GROUP BYs are done in individual workers and then &lt;b&gt;shuffled such that key value pairs of the same key are then in the same worker&lt;/b&gt;. Further aggregation then occurs and is passed into a single worker and served.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="served" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_62.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Repartitioning&lt;/h3&gt;&lt;p&gt;If too much data ends up on a single worker, BigQuery may re-partition the data. Let’s consider the example below. The sources start writing to Sink 1 and 2 (partitions within the memory shuffle tier). Next, the shuffle Monitor detects Sink 2 is over the limit. Now, the partitioning scheme changes and the sources stop writing to Sink 2, and instead start writing to Sink 3 and 4. &lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="sinks" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_63.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;Optimizations&lt;/h3&gt;&lt;b&gt;1. Late aggregation&lt;/b&gt;: Aggregate as late and as seldom as possible, because aggregation is very costly. The exception is if a table can be reduced drastically by aggregation in preparation for a join - more on this below.&lt;p&gt;For example, instead of a query like this, where you aggregate in both the subqueries and the final SELECT:&lt;br/&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;You should only aggregate once, in the outer query:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;b&gt;2. Nest repeated data&lt;/b&gt;: Let’s imagine you have a table showing retail transactions. If you model one order per row and nest line items in an &lt;a href="https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#array_type"&gt;ARRAY&lt;/a&gt; field - then you have cases where GROUP BY is no longer required. For example, looking at the total number of items in an order by using &lt;a href="https://cloud.google.com/bigquery/docs/reference/standard-sql/arrays#finding_lengths"&gt;ARRAY_LENGTH&lt;/a&gt;.&lt;br/&gt;&lt;br/&gt;&lt;p&gt;&lt;code&gt;{order_id1, [ {item_id1}, {item_id2} ] }&lt;/code&gt;&lt;/p&gt;&lt;h2&gt;Understanding joins&lt;/h2&gt;&lt;p&gt;One powerful aspect of BigQuery is the ability to combine data, and understand relationships and correlation information from disparate sources. Much of the JOIN syntax is about expressing how that data should be combined, and how to handle data when information is mismatched. However, once that relationship is encoded, BigQuery still needs to execute it. &lt;/p&gt;&lt;h3&gt;Hash-based joins&lt;/h3&gt;&lt;p&gt;Let's jump straight into large scale join execution.  When joining two tables on a common key, BigQuery favors a technique called the hash-based join, or more simply a hash join. With this technique, we can process a table using multiple workers, rather than moving data through a coordinating node.  &lt;/p&gt;&lt;p&gt;&lt;b&gt;So what does hashing actually involve?&lt;/b&gt; When we hash values, we're converting the input value into a number that falls in a known range. There's many properties of hash functions we care about for hash joins, but the important ones are that our function is deterministic (the same input always yields the same output value) and has uniformity (our output values are evenly spread throughout the allowed range of values).&lt;/p&gt;&lt;p&gt;With an appropriate hashing function, we can then use the output to bucket values.  For example, if our hash function yields an output floating point value between 0 and 1, we can bucket by dividing that key range into N parts, where N is the number of buckets we want.  Grouping data based on this hash value means our buckets should have roughly the same number of discrete values, but even more importantly, all duplicate values should end up in the same bucket. &lt;/p&gt;&lt;p&gt;Now that you understand what hashing does, let's talk through joining.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="joining" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_64.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;To perform the hash join, we're going to split up our work into three stages.&lt;/p&gt;&lt;p&gt;&lt;b&gt;Stage 1: Prepare the first table&lt;/b&gt;&lt;/p&gt;&lt;p&gt;In BigQuery, data for a table is typically split into multiple columnar files, but within those files there's no sorting guarantee that ensures that the columns that represent the join key are sorted and colocated.  So, what we do is apply our hashing function to the join key, and based on the buckets we desire we can write rows into different shuffle partitions. &lt;/p&gt;&lt;p&gt;In the diagram above, we have three columnar files in the first table, and we've using our hashing technique to split the data into four buckets (color coded).  Once the first stage is complete, the rows of the first table are effectively split into four "file-like" partitions in shuffle, with duplicates co-located.&lt;/p&gt;&lt;p&gt;&lt;b&gt;Stage 2:  Prepare the second table&lt;/b&gt;&lt;/p&gt;&lt;p&gt;This is effectively the same work as the first stage, but we're processing the other table we'll be joining data against. The important thing to note here is that we need to use the same hashing function and therefore the same bucket grouping, as we're aligning data. In the diagram above, the second table had four input files (and thus four units of work), and the data was written into a second set of shuffle partitions.&lt;/p&gt;&lt;p&gt;&lt;b&gt;Stage 3: consume the aligned data and perform the join&lt;/b&gt;&lt;/p&gt;&lt;p&gt;After the first two stages are completed, we've aligned the data in the two tables using a common hash function and bucketing strategy.  What this means is that we have a set of paired shuffle partitions that correspond to the same hash range, which means that rather than scanning potentially large sets of data, we can execute the join in pieces, as each worker is provided only the relevant data for doing it's subset of the join.&lt;/p&gt;&lt;p&gt;It's at this point that we care about the nature of the join operation again; depending on the desired join relationship we may yield no rows, a single row, or many rows for any particular input row from the original input tables.&lt;/p&gt;&lt;p&gt;Now, you can also get a better sense of how important having a good hashing function may be:  if the output values are poorly distributed, we have problems because we're much more likely to have a single worker that's slower and forced to do the majority of the work.  Similarly, if we picked our number of buckets poorly, we may have issues due to having split the work too finely or too coarsely.  Fortunately, these are not insurmountable problems, as we can leverage dynamic planning to fix this: we simply insert query stages to adjust the shuffle partitions.&lt;/p&gt;&lt;h3&gt;Broadcast joins&lt;/h3&gt;&lt;p&gt;Hash-based joins are an incredibly powerful technique for joining lots of data, but your data isn't always large enough to warrant it.  For cases where one of the tables is small, we can avoid all the alignment work altogether.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="altogether" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_65.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Broadcast joins work in cases where one table is small.  In these instances, it's easiest to replicate the small table into shuffle for faster access, and then simply provide a reference to that data for each worker that's responsible for processing the other table's input files.&lt;/p&gt;&lt;h3&gt;Optimization techniques&lt;/h3&gt;&lt;ol&gt;&lt;li&gt;Largest table first:  BigQuery best practice is to manually place the largest table first, followed by the smallest, and then by decreasing size. Only under specific table conditions does BigQuery automatically reorder/optimize based on table size.&lt;/li&gt;&lt;li&gt;Filter before joins: WHERE clauses should be executed as soon as possible, especially within joins, so the tables to be joined are as small as possible. We recommend reviewing the query execution details to see if filtering is happening as early as possible, and either fix the condition or use a subquery to filter in advance of a JOIN.&lt;/li&gt;&lt;li&gt;Pre-aggregate to limit table size: As mentioned above, aggregating tables before they are joined can help improve performance - but only if the amount of data being joined is drastically reduced and tables are aggregated to the same level (i.e., if there is only one row for every join key value).&lt;/li&gt;&lt;li&gt;Clustering on join keys: When you cluster a table based on the key that is used to join, the data is already co-located which makes it easier for workers to split the data into the necessary partitions within the memory shuffle. &lt;/li&gt;&lt;/ol&gt;&lt;h3&gt;A detailed query: finding popular libraries in Github&lt;/h3&gt;&lt;p&gt;Now that we understand some optimization techniques for filtering, aggregating and joining data - let’s look at a complex query with multiple SQL techniques. Walking through the execution details for this query should help you understand how data flows and mutates as it moves through the query plan - so that you can apply this knowledge and understand what’s happening behind the scenes in your own complex query.  &lt;/p&gt;&lt;p&gt;The&lt;a href="https://cloud.google.com/blog/topics/public-datasets/github-on-bigquery-analyze-all-the-open-source-code"&gt;public Github&lt;/a&gt;data has one table that contains information about source code filenames, while another contains the contents of these files.  By combining the two together, we can filter down to focus on interesting files and analyze them to understand which libraries are frequently used by developers. Here's an example of a query that does this for developers using the Go programming language. It scans files having the appropriate (.go) extensions, and looks for statements in the source code for importing libraries, then counts how often those libraries are used and how many distinct code repositories use them.&lt;/p&gt;&lt;p&gt;In SQL, it looks like this:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;We can see from a casual read that we've got lots of interesting bits here: subqueries, both a distributed join (the contents and files tables), array manipulation (cross join unnest), and powerful features such as regular expression filters and computing distinctness.&lt;/p&gt;&lt;h3&gt;Detailed stages and steps&lt;/h3&gt;&lt;p&gt;First, let's examine the full details of the plan in a graph format.  Here, we're looking at the low level details of how this query is run, as a set of stages.  Let's work through the query stages in detail.&lt;a href="https://github.com/shollyman/gcp-go-snippets/pull/1" target="_blank"&gt;If you want a graphical representation similar to the one we’re showing here, check out this code sample!&lt;/a&gt;&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-image_full_width"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid"&gt;&lt;figure class="article-image--large h-c-grid__col h-c-grid__col--6 h-c-grid__col--offset-3 "&gt;&lt;img alt="code sample" src="https://storage.googleapis.com/gweb-cloudblog-publish/images/unnamed_66.max-1000x1000.png"/&gt;&lt;/figure&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;&lt;b&gt;Stage S00-S01: Reading and filtering from the "files" table&lt;/b&gt;&lt;/p&gt;&lt;p&gt;The initial stage (corresponding to the inner subquery of the SQL) begins by processing the "files" table.  We can see the first task is to read the input, and immediately filter that to only pass through files with the appropriate suffix.  We then group based on the id and and repo name, as we're potentially working with many duplicates, and we only want to process each distinct pair once. In stage S01, we continue the GROUP BY operation; each worker in the first stage only deduplicated the repo/id pairs in their individual input file(s), the aggregate stage here is to combine those so that we've deduplicated across all input rows in the "files" table.&lt;/p&gt;&lt;p&gt;&lt;b&gt;Stage S02: Reading in the "contents" table&lt;/b&gt;&lt;/p&gt;&lt;p&gt;In this stage, we begin reading the source code in the "contents" table, looking for "import" statements (the syntax for referencing libraries in the Go language).  We collect information about the id (which will become the join key), and the content which has matches. You can also see that in both this stage and the previous (S01), the output is split based on a BY HASH operation.  This is the first part of starting the hash join, where we begin to align join keys into distinct shuffle partitions.  However, anytime we're dealing with data where we want to divide the work we'll be splitting it into shuffle buckets with this operation.&lt;/p&gt;&lt;p&gt;&lt;b&gt;Stages S03 - S0A: Repartitioning&lt;/b&gt;&lt;/p&gt;&lt;p&gt;This query invoked several repartitioning stages.  This is an example of the dynamic planner rebalancing data as it's working through the execution graph.  Much of the internals of picking appropriate bucketing is based on heuristics, as operations such as filtration can drastically change the amount of data flowing in and out of query stages. In this particular query, the query plan has chosen a non-optimal bucketing strategy, and is rebalancing the work as it goes.  Also note that this partitioning is happening on both sides of what will become the joined data, because we need to keep the partitioned data aligned as we enter the join.&lt;/p&gt;&lt;p&gt;&lt;b&gt;Stage S0B: Executing the join&lt;/b&gt;&lt;/p&gt;&lt;p&gt;Here's where we begin correlating the data between the two inputs.  You can see in this stage we have two input reads (one for each side of the join), and start computing counts.  There's also some overloaded work here; we consume the file contents to yield an array representing each individual library being imported, and make that available to future stages.&lt;/p&gt;&lt;p&gt;&lt;b&gt;Stages S0C - S0D: Partial Aggregations&lt;/b&gt;&lt;/p&gt;&lt;p&gt;These two stages are responsible for computing our top level statistics:  we wanted to count the total number of times each library was referenced, as well as the number of distinct repositories.  We end up splitting that into two stages.&lt;/p&gt;&lt;p&gt;&lt;b&gt;Stage S0E-S0F: Ordering and limiting&lt;/b&gt;&lt;/p&gt;&lt;p&gt;Our query requested only the top 1000 libraries ordered first by distinct repository count, and then total frequency of use.  The last two stages are responsible for doing this sorting and reduction to yield the final result.&lt;/p&gt;&lt;h2&gt;Other optimization techniques&lt;/h2&gt;&lt;p&gt;As a final thought, we’ll leave you with a few more optimization techniques that could help improve the performance of your queries.&lt;/p&gt;&lt;ol&gt;&lt;li&gt;Multiple WITH clauses: The &lt;a href="https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax#with_clause"&gt;WITH&lt;/a&gt; statement in BigQuery is like a Macro. At runtime the contents of the subquery will be inlined every place the alias is referenced. This can lead to query plan explosion as seen by the plan executing the same query stages multiple times. Instead, try using a &lt;a href="https://cloud.google.com/bigquery/docs/reference/standard-sql/scripting-concepts#temporary_tables"&gt;TEMP&lt;/a&gt; table.&lt;/li&gt;&lt;li&gt;String comparisons: &lt;a href="https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators"&gt;REGEXP_CONTAINS&lt;/a&gt; can offer more functionality, but it has a slower execution time compare to LIKE. Make LIKE when the full power of regex is not needed (e.g. wildcard matching): &lt;/li&gt;&lt;/ol&gt;&lt;p&gt;&lt;code&gt;regexp_contains(dim1, ‘.*test.*’)&lt;/code&gt; to &lt;code&gt;dim1&lt;/code&gt; &lt;code&gt;like&lt;/code&gt; &lt;code&gt;%test%&lt;/code&gt;&lt;/p&gt;&lt;p&gt;First or last record: When trying to calculate the first or last record in a subset of your data, using the &lt;a href="https://cloud.google.com/bigquery/docs/reference/standard-sql/analytic-function-concepts#numbering_function_concepts"&gt;ROW_NUMBER()&lt;/a&gt; function can fail with Resources Exceeded errors if there are too many elements to ORDER BY in a single partition. Instead, try using &lt;a href="https://cloud.google.com/bigquery/docs/reference/standard-sql/aggregate_functions#array_agg"&gt;ARRAY_AGG()&lt;/a&gt;- which runs more efficiently because the ORDER BY is allowed to drop everything except the top record on each GROUP BY. For example, this:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;p&gt;Becomes this:&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-code"&gt;&lt;div class="article-module h-c-page"&gt;&lt;div class="h-c-grid uni-paragraph-wrap"&gt;&lt;div class="uni-paragraph h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3"&gt;&lt;pre&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-paragraph"&gt;&lt;div class="rich-text"&gt;&lt;h3&gt;See you next week!&lt;/h3&gt;&lt;p&gt;Thanks again for tuning in this week! Next up is data governance, so be sure to keep an eye out for more in this series by following Leigha on &lt;a href="https://linkedin.com/in/leighajarett" target="_blank"&gt;LinkedIn&lt;/a&gt; and &lt;a href="https://twitter.com/leighajarett" target="_blank"&gt;Twitter&lt;/a&gt;.&lt;/p&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="block-related_article_tout"&gt;&lt;div class="uni-related-article-tout h-c-page"&gt;&lt;section class="h-c-grid"&gt;&lt;a class="uni-related-article-tout__wrapper h-c-grid__col h-c-grid__col--8 h-c-grid__col-m--6 h-c-grid__col-l--6 h-c-grid__col--offset-2 h-c-grid__col-m--offset-3 h-c-grid__col-l--offset-3 uni-click-tracker" data-analytics='{ "event": "page interaction", "category": "article lead", "action": "related article - inline", "label": "article: {slug}" }' href="https://gweb-cloudblog-publish.appspot.com/topics/developers-practitioners/bigquery-admin-reference-guide-query-processing/"&gt;&lt;div class="uni-related-article-tout__inner-wrapper"&gt;&lt;p class="uni-related-article-tout__eyebrow h-c-eyebrow"&gt;Related Article&lt;/p&gt;&lt;div class="uni-related-article-tout__content-wrapper"&gt;&lt;div class="uni-related-article-tout__image-wrapper"&gt;&lt;div class="uni-related-article-tout__image" style="background-image: url('https://storage.googleapis.com/gweb-cloudblog-publish/images/BQ_QueryPRoc.max-500x500.jpeg')"&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class="uni-related-article-tout__content"&gt;&lt;h4 class="uni-related-article-tout__header h-has-bottom-margin"&gt;BigQuery Admin reference guide: Query processing&lt;/h4&gt;&lt;p class="uni-related-article-tout__body"&gt;BigQuery is capable of some truly impressive feats, be it scanning billions of rows based on a regular expression, joining large tables, ...&lt;/p&gt;&lt;div class="cta module-cta h-c-copy uni-related-article-tout__cta muted"&gt;&lt;span class="nowrap"&gt;Read Article&lt;svg class="icon h-c-icon" role="presentation"&gt;&lt;use xlink:href="#mi-arrow-forward" xmlns:xlink="http://www.w3.org/1999/xlink"&gt;&lt;/use&gt;&lt;/svg&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/div&gt;&lt;/a&gt;&lt;/section&gt;&lt;/div&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;</description><pubDate>Thu, 05 Aug 2021 16:30:00 -0000</pubDate><guid>https://cloud.google.com/blog/topics/developers-practitioners/bigquery-admin-reference-guide-query-optimization/</guid><category>Google Cloud</category><category>Data Analytics</category><category>Developers &amp; Practitioners</category><media:content url="https://storage.googleapis.com/gweb-cloudblog-publish/images/BQA_QueryOpt_Thumb.max-600x600.jpeg" width="540" height="540"></media:content><og xmlns:og="http://ogp.me/ns#"><type>article</type><title>BigQuery Admin reference guide: Query optimization</title><description>Understand complex query execution and explore techniques to refine your SQL for faster, more efficient data processing.</description><image>https://storage.googleapis.com/gweb-cloudblog-publish/images/BQA_QueryOpt_Thumb.max-600x600.jpeg</image><site_name>Google</site_name><url>https://cloud.google.com/blog/topics/developers-practitioners/bigquery-admin-reference-guide-query-optimization/</url></og><author xmlns:author="http://www.w3.org/2005/Atom"><name>Leigha Jarett</name><title>Developer Advocate</title><department></department><company></company></author><author xmlns:author="http://www.w3.org/2005/Atom"><name>Seth Hollyman</name><title>Developer Relations Engineer</title><department></department><company>BigQuery</company></author></item></channel></rss>