CINXE.COM

<!doctype html><html lang=en-us><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1,maximum-scale=1,user-scalable=no"><title>Chapter 1: Combining Existing Transformations - MLIR</title><meta name=description content="Multi-Level IR Compiler Framework"><meta name=generator content="Hugo 0.119.0"><link href=https://mlir.llvm.org/index.xml rel=alternate type=application/rss+xml><link rel=canonical href=https://mlir.llvm.org/docs/Tutorials/transform/Ch1/><link rel=stylesheet href=https://mlir.llvm.org/css/theme.css><script src=https://use.fontawesome.com/releases/v5.0.6/js/all.js></script> <link rel=stylesheet href=https://mlir.llvm.org/css/chroma.min.css><script src=https://cdn.jsdelivr.net/npm/jquery@3.3.1/dist/jquery.min.js></script> <script src=https://cdn.jsdelivr.net/npm/jquery.easing@1.4.1/jquery.easing.min.js></script> <script src=https://mlir.llvm.org/js/bundle.js></script> <script type=text/javascript src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script> <script type=text/x-mathjax-config> MathJax.Hub.Config({ tex2jax: { inlineMath: [['$', '$'] ], displayMath: [ ['$$','$$'], ["\\[","\\]"] ] } }); </script><link rel=apple-touch-icon sizes=180x180 href="/apple-touch-icon.png?v=1"><link rel=icon type=image/png sizes=32x32 href="/favicon-32x32.png?v=1"><link rel=icon type=image/png sizes=16x16 href="/favicon-16x16.png?v=1"><link rel=manifest href="/site.webmanifest?v=1"><link rel=mask-icon href="/safari-pinned-tab.svg?v=1" color=#3775e0><link rel="shortcut icon" href="/favicon.ico?v=1"><meta name=msapplication-TileColor content="#2d89ef"><meta name=theme-color content="#ffffff"><link rel=icon href=/favicon.svg type=image/svg+xml sizes=any><style>:root{}</style></head><body><div class=container><header><h1><div><img src=https://mlir.llvm.org//mlir-logo.png width=40px align=absmiddle> MLIR</div></h1>Multi-Level IR Compiler Framework</header><div class=global-menu><nav><ul><li class=parent><a href>Community</a><ul class=sub-menu><li class=child><a href=https://llvm.discourse.group/c/mlir/31>Forums</a></li><li class=child><a href=https://discord.gg/xS7Z362>Chat</a></li></ul></li><li><a href=/getting_started/Debugging/>Debugging Tips</a></li><li><a href=/getting_started/Faq/>FAQ</a></li><li class=parent><a href=https://github.com/llvm/llvm-project/tree/main/mlir>Source</a><ul class=sub-menu><li class=child><a href=/doxygen/>Doxygen</a></li><li class=child><a href=https://github.com/llvm/llvm-project/tree/main/mlir>GitHub</a></li></ul></li><li><a href="https://bugs.llvm.org/buglist.cgi?bug_status=__open__&list_id=177877&order=changeddate%20DESC%2Cpriority%2Cbug_severity&product=MLIR&query_format=specific">Bugs</a></li><li><a href=https://github.com/llvm/mlir-www/tree/main/website/static/LogoAssets>Logo Assets</a></li><li><a href=https://www.youtube.com/MLIRCompiler>Youtube Channel</a></li></ul></nav></div><div class=content-container><main><h1>Chapter 1: Combining Existing Transformations</h1><h2 id=introduction>Introduction <a class=headline-hash href=#introduction>¶</a></h2>The Transform dialect allows one to precisely target transformations at specific operations in the IR and to chain them, that is to apply a transformation to operations produced by the previous transformation. To achieve this, transformations are expressed as other operations in the IR. We call these the IR containing these operations transform IR. And we call the IR that is being transformed payload IR.Transform IR operations operate on values that may be associated with payload IR operations, values or attributes. We call the first two kinds of values operation and value handles, respectively. We call the last kind of values parameters.The application of transform IR always starts from one top-level operation. In the C++ API, this operation is passed to the <code>applyTransforms</code> function. This top-level operation specifies if other transformations should be performed and how. The most common top-level operation, <code>transform.named_sequence</code> merely applies other transform operations listed in its body one after the other, similarly to a function or a macro.Let us illustrate this with a simple sequence of transformations on the common “fully connected + bias + ReLU” ML layer, which boils down to performing a matrix multiplication, followed by an (elementwise) matrix addition and taking an elementwise maximum with 0. This can be expressed using the following IR:<div class=highlight><pre tabindex=0 class=chroma><code class=language-mlir data-lang=mlir>func.func @fc_relu(%lhs: tensor<512x512xf32>, %rhs: tensor<512x512xf32>, %bias: tensor<512x512xf32>, %output: tensor<512x512xf32>) -> tensor<512x512xf32> { // Matrix-matrix multiplication. %matmul = linalg.matmul ins(%lhs, %rhs: tensor<512x512xf32>, tensor<512x512xf32>) outs(%output: tensor<512x512xf32>) -> tensor<512x512xf32> // Elementwise addition. %biased = linalg.elemwise_binary { fun = #linalg.binary_fn<add> } ins(%matmul, %bias : tensor<512x512xf32>, tensor<512x512xf32>) outs(%output : tensor<512x512xf32>) -> tensor<512x512xf32> // Elementwise max with 0 (ReLU). %c0f = arith.constant 0.0 : f32 %relued = linalg.elemwise_binary { fun = #linalg.binary_fn<max_signed> } ins(%biased, %c0f : tensor<512x512xf32>, f32) outs(%output : tensor<512x512xf32>) -> tensor<512x512xf32> func.return %relued : tensor<512x512xf32> } </code></pre></div><h2 id=top-level-sequence-operation>Top-Level Sequence Operation <a class=headline-hash href=#top-level-sequence-operation>¶</a></h2>For performance reasons, we would like to tile and fuse these operations to exploit cache locality. This is a sequence of transformations that need to be performed one after another, so we naturally start with the corresponding top-level transform operation.<div class=highlight><pre tabindex=0 class=chroma><code class=language-mlir data-lang=mlir>module attributes {transform.with_named_sequence} { transform.named_sequence @__transform_main( %arg0: !transform.any_op, %arg1: !transform.op<"linalg.matmul">, %arg2: !transform.op<"linalg.elemwise_binary">): transform.yield } } </code></pre></div>There are several aspects worth noticing in this operation.Its special name, <code>@__transform_main</code> and the first argument are mandated by the interpreter pass, similarly to how the entry point of C programs needs to be called <code>main</code> and may have the <code>int (int argc, char** argv)</code> signature. This argument will be associated with the top-level payload operation, most often the operation that the pass is applied to. Note that none of this is required when applying the transformation programmatically via <code>applyTransforms</code> or <code>applyNamedSequence</code>.The remaining entry block arguments are optional and can be associated with payload attributes, operations or values that are useful in the sequence. These are also specified when calling <code>applyTransforms</code>. In our case, we are interested in the matrix multiplication and elementwise operations that we are going to tile and fuse.All value handles have Transform dialect types. These types specify certain properties of the payload IR entities associated with them. In this example, <code>transform.any_op</code> indicates that the handle is associated with arbitrary payload operations. On the contrary, <code>transform.op<"X"></code> indicates that the handle is associated only with payload operations of kind <code>X</code>. These constraints are verified when the handle/payload association is created. For entry block arguments of top-level transform operations, this happens early in the <code>applyTransforms</code> function. If the constraints are not satisfied, the transform application fails and produces diagnostics for the user.Finally, the operation is wrapped in a module with the <code>transform.with_named_sequence</code> attribute that triggers all necessary verifications if multiple named sequences exist.<h2 id=failure-propagation>Failure Propagation <a class=headline-hash href=#failure-propagation>¶</a></h2>The Transform dialect infrastructure has a particular mechanism for handling diagnostics that supports recoverable errors. It is best understood by considering the (unnamed) sequence operation that has a mandatory attribute specifying the failure propagation mode. There are two options:<ul><li>“propagate” makes the sequence transformation fail if any of the nested transformation fails;</li><li>“suppress” makes the sequence succeed even if one of the nested transformations fails, but without attempting to perform the transformations following the failed one in the sequence.</li></ul>This latter allows the transformation script surrounding the sequence to continue despite errors within the sequence, assuming they are recoverable. As we are only building the transformation script, it is preferable to propagate failures so we know when something did not apply.To check or debug a transform sequence, it is possible to print various entities associated with the transform IR values. For example, we can print the operations associated with the handles:<div class=highlight><pre tabindex=0 class=chroma><code class=language-mlir data-lang=mlir>transform.sequence failures(propagate) { ^bb0(%arg0: !transform.any_op, %arg1: !transform.op<"linalg.matmul">, %arg2: !transform.op<"linalg.elemwise_binary">): transform.debug.emit_remark_at %arg1, "matmul" : !transform.op<"linalg.matmul"> transform.debug.emit_remark_at %arg2, "elemwise_binaries" : !transform.op<"linalg.elemwise_binary"> transform.yield } </code></pre></div><h2 id=transform-dialect-interpreter>Transform Dialect Interpreter <a class=headline-hash href=#transform-dialect-interpreter>¶</a></h2>Since we don’t want to recompile the compiler every time we change a transformation, we can use a Transform dialect interpreter pass to apply this transformation sequence to the payload IR. As we will see in the next chapter, it is possible to define custom passes or even integrate the transform interpreter into a larger pass. For now, we can use the existing test pass:<div class=highlight><pre tabindex=0 class=chroma><code class=language-sh data-lang=sh>$ mlir-opt sequence.mlir --pass-pipeline=" builtin.module(transform-interpreter{ debug-bind-trailing-args=linalg.matmul,linalg.elemwise_binary})" </code></pre></div>The <code>sequence.mlir</code> file contains both the payload IR function and the transform IR sequence nested in the same module. The transform interpreter pass will apply the <code>@__transform_main</code> named sequence to the anchor operation of the pass. In our case, we also asked the interpreter pass to associate the two extra arguments of the top-level sequence with all <code>linalg.matmul</code> and <code>linalg.elemwise_binary</code> payload operations through the respective pass options. Running this pass results in the expected remarks:<div class=highlight><pre tabindex=0 class=chroma><code class=language-sh data-lang=sh>sequence.mlir:7:13: remark: matmul %matmul = linalg.matmul ins(%lhs, %rhs: tensor<512x512xf32>, tensor<512x512xf32>) ^ sequence.mlir:7:13: note: see current operation: %0 = linalg.matmul ins(%arg0, %arg1 : tensor<512x512xf32>, tensor<512x512xf32>) outs(%arg3 : tensor<512x512xf32>) -> tensor<512x512xf32> sequence.mlir:10:13: remark: elemwise_binaries %biased = linalg.elemwise_binary { fun = #linalg.binary_fn<add> } ^ sequence.mlir:10:13: note: see current operation: %1 = linalg.elemwise_binary {fun = #linalg.binary_fn<add>} ins(%0, %arg2 : tensor<512x512xf32>, tensor<512x512xf32>) outs(%arg3 : tensor<512x512xf32>) -> tensor<512x512xf32> sequence.mlir:14:13: remark: elemwise_binaries %relued = linalg.elemwise_binary { fun = #linalg.binary_fn<max_signed> } ^ sequence.mlir:14:13: note: see current operation: %2 = linalg.elemwise_binary {fun = #linalg.binary_fn<max_signed>} ins(%1, %cst : tensor<512x512xf32>, f32) outs(%arg3 : tensor<512x512xf32>) -> tensor<512x512xf32> </code></pre></div>Note that <code>%arg2</code> is associated with both elementwise payload operations. Any handle is associated with a list of entities. Individual transformations may or may not care about the order of elements in that list.<h2 id=specifying-transformations>Specifying Transformations <a class=headline-hash href=#specifying-transformations>¶</a></h2>Now that we have handles to the operations we want to transform, we are ready to apply the transformations. Let us first try tiling the matmul operation itself.<div class=highlight><pre tabindex=0 class=chroma><code class=language-mlir data-lang=mlir>module attributes {transform.with_named_sequence} { transform.named_sequence @__transform_main( %arg0: !transform.any_op, %arg1: !transform.op<"linalg.matmul">, %arg2: !transform.op<"linalg.elemwise_binary">) { // The actual tiling transformation takes tile sizes as attributes. %loop, %tiled = transform.structured.tile_using_forall %arg1 tile_sizes [4, 32] : (!transform.op<"linalg.matmul">) -> (!transform.any_op, !transform.any_op) transform.yield } } </code></pre></div>The transformation returns two handles, as indicated in its <a href=https://mlir.llvm.org/docs/Dialects/Transform/#transformstructuredtile_using_forall-transformtileusingforallop>documentation</a>:<ul><li>A handle to <code>linalg.generic</code> operating on the subset of the original data.</li><li>A handle to the <code>scf.forall</code> “multi-for” loop around tensors.</li></ul>Running this transformation with the same command as above expectedly produces the tiled code.<div class=highlight><pre tabindex=0 class=chroma><code class=language-mlir data-lang=mlir>func.func @fc_relu(%arg0: tensor<512x512xf32>, %arg1: tensor<512x512xf32>, %arg2: tensor<512x512xf32>, %arg3: tensor<512x512xf32>) -> tensor<512x512xf32> { %cst = arith.constant 0.000000e+00 : f32 %0 = scf.forall (%arg4, %arg5) in (128, 16) shared_outs(%arg6 = %arg3) -> (tensor<512x512xf32>) { %3 = affine.apply affine_map<(d0) -> (d0 * 4)>(%arg4) %4 = affine.apply affine_map<(d0) -> (d0 * 32)>(%arg5) %extracted_slice = tensor.extract_slice %arg0[%3, 0] [4, 512] [1, 1] : tensor<512x512xf32> to tensor<4x512xf32> %extracted_slice_0 = tensor.extract_slice %arg1[0, %4] [512, 32] [1, 1] : tensor<512x512xf32> to tensor<512x32xf32> %extracted_slice_1 = tensor.extract_slice %arg6[%3, %4] [4, 32] [1, 1] : tensor<512x512xf32> to tensor<4x32xf32> %5 = linalg.matmul ins(%extracted_slice, %extracted_slice_0 : tensor<4x512xf32>, tensor<512x32xf32>) outs(%extracted_slice_1 : tensor<4x32xf32>) -> tensor<4x32xf32> scf.forall.in_parallel { tensor.parallel_insert_slice %5 into %arg6[%3, %4] [4, 32] [1, 1] : tensor<4x32xf32> into tensor<512x512xf32> } } %1 = linalg.elemwise_binary {fun = #linalg.binary_fn<add>} ins(%0, %arg2 : tensor<512x512xf32>, tensor<512x512xf32>) outs(%arg3 : tensor<512x512xf32>) -> tensor<512x512xf32> %2 = linalg.elemwise_binary {fun = #linalg.binary_fn<max_signed>} ins(%1, %cst : tensor<512x512xf32>, f32) outs(%arg3 : tensor<512x512xf32>) -> tensor<512x512xf32> return %2 : tensor<512x512xf32> } </code></pre></div>Besides producing new handles, the tiling transform operation consumes the operand handle. This means that the handle is invalidated after this operation, and is no longer supposed to be used. Transform operations are required to mark all their operands as either consumed or readonly. Transform operations usually consume the operand if the associated payload operations are erased or recreated (which means erased and created anew with similar structure). As handles are essentially references to payload operations, they would become dangling if the payload no longer exists.<h2 id=handle-invalidation-and-expensive-checks-mode>Handle Invalidation and Expensive Checks Mode <a class=headline-hash href=#handle-invalidation-and-expensive-checks-mode>¶</a></h2>Undefined behavior is difficult to grapple with when it does happen, so the Transform dialect interpreter defaults to performing a set of additional, potentially expensive, checks that detect most undefined behavior in the transform IR. For example, if we wanted to use the <code>%arg1</code> handle after it is consumed, it would cause undefined behavior that manifests as an assertion in the debug build, and likely as a segmentation fault in the release mode.<div class=highlight><pre tabindex=0 class=chroma><code class=language-mlir data-lang=mlir>module attributes {transform.with_named_sequence} { transform.named_sequence @__transform_main( %arg0: !transform.any_op, %arg1: !transform.op<"linalg.matmul">, %arg2: !transform.op<"linalg.elemwise_binary">) { // The actual tiling transformation takes tile sizes as attributes. %loop, %tiled = transform.structured.tile_using_forall %arg1 tile_sizes [4, 32] : (!transform.op<"linalg.matmul">) -> (!transform.any_op, !transform.any_op) // This is trying to use an invalidated handle leading to undefined behavior. transform.debug.emit_remark_at %arg1, "remark" : !transform.op<"linalg.matmul"> transform.yield } } </code></pre></div>However, with the expensive checks enabled in the interpreter, a nice diagnostic is produced:<div class=highlight><pre tabindex=0 class=chroma><code class=language-sh data-lang=sh>sequence.mlir:28:3: error: op uses a handle invalidated by a previously executed transform op transform.debug.emit_remark_at %mm, "elemwise_binaries" : !transform.any_op ^ sequence.mlir:26:9: note: handle to invalidated ops %mm = transform.cast %matmul : !transform.op<"linalg.matmul"> to !transform.any_op ^ sequence.mlir:27:19: note: invalidated by this transform op that consumes its operand #0 and invalidates all handles to payload IR entities associated with this operand and entities nested in them %loop, %tiled = transform.structured.tile_using_forall %mm tile_sizes [4, 32] </code></pre></div>When compile-time performance is a concern, and the transformation sequence is sufficiently stable, it is possible to disable expensive checks in the interpreter for improved performance by providing the <code>disable-expensive-checks</code> option to the pass or by setting the corresponding flag in the <code>TransformOptions</code> passed into <code>applyTransforms</code>.One may observe that some operations such as <code>transform.cast</code> do not consume the operand (because they don’t erase the corresponding operation). So what would happen if we tried to use that operand instead?<div class=highlight><pre tabindex=0 class=chroma><code class=language-mlir data-lang=mlir>module attributes {transform.with_named_sequence} { transform.named_sequence @__transform_main %arg0: !transform.any_op, %arg1: !transform.op<"linalg.matmul">, %arg2: !transform.op<"linalg.elemwise_binary">) { // We can cast one type to another as long as operations are compatible // with both types. This creates "aliasing" handles. %casted = transform.cast %arg1 : !transform.op<"linalg.matmul"> to !transform.any_op // The actual tiling transformation takes tile sizes as attributes. %loop, %tiled = transform.structured.tile_using_forall %arg1 tile_sizes [4, 32] : (!transform.op<"linalg.matmul">) -> (!transform.any_op, !transform.any_op) // Consuming an operand invalidates the consumed handle and any other handle // that is associated with the same payload operations, or payload // operations nested in them. transform.debug.emit_remark_at %casted, "remark" : !transform.any_op transform.yield } } </code></pre></div>Both <code>%arg1</code> and <code>%casted</code> reference the same payload operation. Extending the reference analogy, these references alias. Naturally, when the payload operation is erased, all references to it become dangling. This is also the case for handles. In fact, consuming an operand invalidates the operand handle as well as any other handle that is associated with any of the same payload operations. The payload IR consideration is recursive: a handle associated with a payload operation nested in the erased one is also invalidated (because erasing the operation also erases its regions and all contained operations). The expensive-checks mode can also handle this case.<div class=highlight><pre tabindex=0 class=chroma><code class=language-sh data-lang=sh>sequence.mlir:28:3: error: op uses a handle invalidated by a previously executed transform op transform.debug.emit_remark_at %matmul, "elemwise_binaries" : !transform.op<"linalg.matmul"> ^ sequence.mlir:21:29: note: handle to invalidated ops ^bb0(%root: !transform.any_op, %matmul: !transform.op<"linalg.matmul">, %elemwise: !transform.op<"linalg.elemwise_binary">): ^ sequence.mlir:27:19: note: invalidated by this transform op that consumes its operand #0 and invalidates all handles to payload IR entities associated with this operand and entities nested in them %loop, %tiled = transform.structured.tile_using_forall %mm tile_sizes [4, 32] </code></pre></div><h2 id=chaining-transformations-with-handles>Chaining Transformations with Handles <a class=headline-hash href=#chaining-transformations-with-handles>¶</a></h2>Going back to the transformation sequence, we have tiled the matrix multiplication, but we also want to tile and fuse the elementwise operations. The typical way of doing in the structured operations paradigm is to tile the last operation in some acyclic dataflow graph, and then progressively fuse the operations that produce its operands. This removes the need to explicitly tile all operations as fusion can adapt their sizes and inject recomputation if desired. So instead of tiling the matmul operation, we are going to tile the last operation in the chain, and then fuse the preceding operations into the loops produced by tiling.<div class=highlight><pre tabindex=0 class=chroma><code class=language-mlir data-lang=mlir>module attributes {transform.with_named_sequence} { transform.named_sequence @__transform_main( %arg0: !transform.any_op, %arg1: !transform.op<"linalg.matmul">, %arg2: !transform.op<"linalg.elemwise_binary">) { // Since the %arg2 handle is associated with both elementwise operations, // we need to split it into two handles so we can target only the second // elementwise operation. %add, %max = transform.split_handle %arg2 : (!transform.op<"linalg.elemwise_binary">) -> (!transform.any_op, !transform.any_op) // The actual tiling transformation takes tile sizes as attributes. It // produces a handle to the loop generated during tiling. %tiled_max, %loop = transform.structured.tile_using_forall %max tile_sizes [8, 32] : (!transform.any_op) -> (!transform.any_op, !transform.any_op) // We can now fuse the other operations into the loop. Here, we fuse // operations one by one. This requires the operation that is being fused to // define the value used within the loop, so the order of such fusions is // important. We could also use "transform.merge_handles" to obtain a single // handle to all operations and give it to `fuse_into_containing_op` that // would take care of the ordering in this case. %add_fused, %loop_0 = transform.structured.fuse_into_containing_op %add into %loop : (!transform.any_op, !transform.any_op) -> (!transform.any_op, !transform.any_op) %matmul_fused, %loop_1 = transform.structured.fuse_into_containing_op %arg1 into %loop_0 : (!transform.op<"linalg.matmul">, !transform.any_op) -> (!transform.any_op, !transform.any_op) transform.yield } } </code></pre></div>This achieves the desired tiling and fusion.<h2 id=more-handle-invalidation>More Handle Invalidation <a class=headline-hash href=#more-handle-invalidation>¶</a></h2>Finally, let us assume there exists an efficient microkernel, or a hardware instruction expressed as an intrinsic function, for a 4x4 matrix multiplication. For this purpose, we need to tile the fused operation to the desired size, and then outline it. The resulting function call can then be replaced with a call to the microkernel.<div class=highlight><pre tabindex=0 class=chroma><code class=language-mlir data-lang=mlir>module attributes {transform.with_named_sequence} { transform.named_sequence @__transform_main( %arg0: !transform.any_op, %arg1: !transform.op<"linalg.matmul">, %arg2: !transform.op<"linalg.elemwise_binary">) { // Since the %arg2 handle is associated with both elementwise operations, // we need to split it into two handles so we can target only the second // elementwise operation. %add, %max = transform.split_handle %arg2 : (!transform.op<"linalg.elemwise_binary">) -> (!transform.any_op, !transform.any_op) // The actual tiling transformation takes tile sizes as attributes. It // produces a handle to the loop generated during tiling. %tiled, %loop = transform.structured.tile_using_forall %max tile_sizes [8, 32] : (!transform.any_op) -> (!transform.any_op, !transform.any_op) // We can now fuse the other operations into the loop. Here, we fuse // operations one by one. This requires the operation that is being fused to // define the value used within the loop, so the order of such fusions is // important. We could also use "transform.merge_handles" to obtain a single // handle to all operations and give it to `fuse_into_containing_op` that // would take care of the ordering in this case. %add_fused, %loop_0 = transform.structured.fuse_into_containing_op %add into %loop : (!transform.any_op, !transform.any_op) -> (!transform.any_op, !transform.any_op) %matmul_fused, %loop_1 = transform.structured.fuse_into_containing_op %arg1 into %loop_0 : (!transform.op<"linalg.matmul">, !transform.any_op) -> (!transform.any_op, !transform.any_op) // Tile again to get the desired size. Note that this time this tiles the // "add" operation and fuses matmul into the loop, but doesn't affect the // "max" operation. This illustrates the precise targeting with the // transform dialect. Otherwise, it is difficult to differentiate "add" and // "max", both of which having the same kind. %tiled_2, %loop_2 = transform.structured.tile_using_forall %add_fused tile_sizes [4, 4] : (!transform.any_op) -> (!transform.any_op, !transform.any_op) %matmul_fused_2, %loop_3 = transform.structured.fuse_into_containing_op %matmul_fused into %loop_2 : (!transform.any_op, !transform.any_op) -> (!transform.any_op, !transform.any_op) // Since outlining is currently only implemented for region-holding // operations such as loops, use tiling to size 1 to materialize the outer // loop that is going to be outlined. %_, %outline_target = transform.structured.tile_using_forall %tiled_2 tile_sizes [1] : (!transform.any_op) -> (!transform.any_op, !transform.any_op) transform.structured.fuse_into_containing_op %matmul_fused_2 into %outline_target : (!transform.any_op, !transform.any_op) -> (!transform.any_op, !transform.any_op) %func, %call = transform.loop.outline %outline_target {func_name = "outlined"} : (!transform.any_op) -> (!transform.any_op, !transform.op<"func.call">) transform.yield } } </code></pre></div>This additional transformation also illustrates handle invalidation for nested operations. The <code>transform.loop.outline</code> operation consumes the handle to the loop, which invalidates it and all handles to any operations nested in it, such as <code>%2</code>. Attempting to use this handle will cause undefined behavior. (Note that it isn’t strictly necessary for this specific form of the outlining to consume the operand as the implementation only moves the region without recreating the operations, but the author of the transformation chose to invalidate the handle anyway.)Attempting to access the fusion result after outlining produces the following error<div class=highlight><pre tabindex=0 class=chroma><code class=language-sh data-lang=sh>test/Examples/transform/Ch1/invalidation-2.mlir:109:3: error: op uses a handle invalidated by a previously executed transform op transform.debug.emit_remark_at %outline_target, "outlined loop" : !transform.any_op ^ test/Examples/transform/Ch1/invalidation-2.mlir:102:25: note: handle to invalidated ops %outline_target, %_ = transform.structured.tile_using_forall %tiled_2 tile_sizes [1] ^ test/Examples/transform/Ch1/invalidation-2.mlir:106:18: note: invalidated by this transform op that consumes its operand #0 and invalidates all handles to payload IR entities associated with this operand and entities nested in them %func, %call = transform.loop.outline %outline_target {func_name = "outlined"} ^ test/Examples/transform/Ch1/invalidation-2.mlir:24:13: note: ancestor payload op %biased = linalg.elemwise_binary { fun = #linalg.binary_fn<add> } ^ test/Examples/transform/Ch1/invalidation-2.mlir:24:13: note: nested payload op %matmul = linalg.matmul ins(%lhs, %rhs: tensor<512x512xf32>, tensor<512x512xf32>) </code></pre></div>Note that the “add” elementwise operation is indicated as payload ancestor because it was used to produce the tile loop, and the loop therefore has its location.Finally, we would like to replace the call to the outlined function with a call to the microkernel. Unfortunately, the Transform dialect doesn’t have support for this transformation (and cannot have if the call is rewritten to a custom, out-of-tree operation). Therefore, we need to define new transform operations. The next chapters will describe how this can be done.<h2 id=tracking-ir-modifications>Tracking IR Modifications <a class=headline-hash href=#tracking-ir-modifications>¶</a></h2>The Transform dialect automatically tracks all IR changes that are made as part of transform ops. (Implementations must use the provided rewriter to modify IR.) If a payload op is erased, it is automatically removed from all handles that it is currently associated with. If a payload op is replaced, the transform dialect tries to find the replacement op and updates all handles accordingly. If a multi-result op is replaced with values that are defined by multiple ops, or if an op is replaced with an op of a different type, an error is produced. This is because it is unclear whether the direct replacements actually represent the computation of the original op. There are ways to customize this behavior. More details can be found at the documentation of <code>transform::TrackingListener</code>.<div class=edit-meta> </div><nav class=pagination><a class="nav nav-prev" href=https://mlir.llvm.org/docs/Tutorials/transform/Ch0/ title="Chapter 0: A Primer on “Structured” Linalg Operations"> Prev - Chapter 0: A Primer on “Structured” Linalg Operations</a> <a class="nav nav-next" href=https://mlir.llvm.org/docs/Tutorials/transform/Ch2/ title="Chapter 2: Adding a Simple New Transformation Operation">Next - Chapter 2: Adding a Simple New Transformation Operation </a></nav><footer>Powered by <a href=https://gohugo.io>Hugo</a>. Theme by <a href=https://themes.gohugo.io/hugo-theme-techdoc/>TechDoc</a>. Designed by <a href=https://github.com/thingsym/hugo-theme-techdoc>Thingsym</a>.</footer></main><div class=sidebar><nav class=slide-menu><ul><li><a href=https://mlir.llvm.org/>Home</a></li><li><a href=https://mlir.llvm.org/users/>Users of MLIR</a></li><li><a href=https://mlir.llvm.org/pubs/>MLIR Related Publications</a></li><li><a href=https://mlir.llvm.org/talks/>Talks</a></li><li><a href=https://mlir.llvm.org/deprecation/>Deprecations & Current Refactoring</a></li><li class=has-sub-menu><a href=https://mlir.llvm.org/getting_started/>Getting Started+</a><ul class=sub-menu><li><a href=https://mlir.llvm.org/getting_started/ReportingIssues/>Reporting Issues</a></li><li><a href=https://mlir.llvm.org/getting_started/Debugging/>Debugging Tips</a></li><li><a href=https://mlir.llvm.org/getting_started/Faq/>FAQ</a></li><li><a href=https://mlir.llvm.org/getting_started/Contributing/>How to Contribute</a></li><li><a href=https://mlir.llvm.org/getting_started/DeveloperGuide/>Developer Guide</a></li><li><a href=https://mlir.llvm.org/getting_started/openprojects/>Open Projects</a></li><li><a href=https://mlir.llvm.org/getting_started/Glossary/>Glossary</a></li><li><a href=https://mlir.llvm.org/getting_started/TestingGuide/>Testing Guide</a></li></ul></li><li class="parent has-sub-menu"><a href=https://mlir.llvm.org/docs/>Code Documentation-</a><ul class=sub-menu><li class=has-sub-menu><a href=https://mlir.llvm.org/docs/Bindings/>Bindings+</a><ul class=sub-menu><li><a href=https://mlir.llvm.org/docs/Bindings/Python/>MLIR Python Bindings</a></li></ul></li><li class=has-sub-menu><a href=https://mlir.llvm.org/docs/Tools/>Tools+</a><ul class=sub-menu><li><a href=https://mlir.llvm.org/docs/Tools/MLIRLSP/>MLIR : Language Server Protocol</a></li><li><a href=https://mlir.llvm.org/docs/Tools/mlir-reduce/>MLIR Reduce</a></li><li><a href=https://mlir.llvm.org/docs/Tools/mlir-rewrite/>mlir-rewrite</a></li></ul></li><li><a href=https://mlir.llvm.org/docs/QuantPasses/></a></li><li><a href=https://mlir.llvm.org/docs/ActionTracing/>Action: Tracing and Debugging MLIR-based Compilers</a></li><li><a href=https://mlir.llvm.org/docs/BufferDeallocationInternals/>Buffer Deallocation - Internals</a></li><li><a href=https://mlir.llvm.org/docs/Bufferization/>Bufferization</a></li><li><a href=https://mlir.llvm.org/docs/DataLayout/>Data Layout Modeling</a></li><li class=has-sub-menu><a href=https://mlir.llvm.org/docs/DefiningDialects/>Defining Dialects+</a><ul class=sub-menu><li><a href=https://mlir.llvm.org/docs/DefiningDialects/Constraints/>Constraints</a></li><li><a href=https://mlir.llvm.org/docs/DefiningDialects/AttributesAndTypes/>Defining Dialect Attributes and Types</a></li><li><a href=https://mlir.llvm.org/docs/DefiningDialects/Operations/>Operation Definition Specification (ODS)</a></li></ul></li><li><a href=https://mlir.llvm.org/docs/Diagnostics/>Diagnostic Infrastructure</a></li><li><a href=https://mlir.llvm.org/docs/DialectConversion/>Dialect Conversion</a></li><li class=has-sub-menu><a href=https://mlir.llvm.org/docs/Dialects/>Dialects+</a><ul class=sub-menu><li><a href=https://mlir.llvm.org/docs/Dialects/DLTITransformOps/></a></li><li><a href=https://mlir.llvm.org/docs/Dialects/OpenACCDialect/>'acc' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/Affine/>'affine' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/AMDGPU/>'amdgpu' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/AMX/>'amx' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/ArithOps/>'arith' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/ArmNeon/>'arm_neon' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/ArmSVE/>'arm_sve' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/ArmSME/>'ArmSME' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/AsyncDialect/>'async' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/BufferizationOps/>'bufferization' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/ControlFlowDialect/>'cf' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/ComplexOps/>'complex' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/DLTIDialect/>'dlti' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/EmitC/>'emitc' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/Func/>'func' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/GPU/>'gpu' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/IndexOps/>'index' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/IRDL/>'irdl' Dialect</a></li><li class=has-sub-menu><a href=https://mlir.llvm.org/docs/Dialects/Linalg/>'linalg' Dialect+</a><ul class=sub-menu><li><a href=https://mlir.llvm.org/docs/Dialects/Linalg/OpDSL/>Linalg OpDSL</a></li></ul></li><li><a href=https://mlir.llvm.org/docs/Dialects/LLVM/>'llvm' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/MathOps/>'math' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/MemRef/>'memref' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/Mesh/>'mesh' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/MLProgramOps/>'ml_program' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/MPI/>'mpi' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/NVGPU/>'nvgpu' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/NVVMDialect/>'nvvm' Dialect</a></li><li class=has-sub-menu><a href=https://mlir.llvm.org/docs/Dialects/OpenMPDialect/>'omp' Dialect+</a><ul class=sub-menu><li><a href=https://mlir.llvm.org/docs/Dialects/OpenMPDialect/ODS/>ODS Documentation</a></li></ul></li><li><a href=https://mlir.llvm.org/docs/Dialects/PDLInterpOps/>'pdl_interp' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/PDLOps/>'pdl' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/PolynomialDialect/>'polynomial' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/PtrOps/>'ptr' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/QuantDialect/>'quant' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/ROCDLDialect/>'rocdl' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/SCFDialect/>'scf' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/ShapeDialect/>'shape' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/SparseTensorOps/>'sparse_tensor' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/TensorOps/>'tensor' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/UBOps/>'ub' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/VCIXDialect/>'vcix' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/Vector/>'vector' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/X86Vector/>'x86vector' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/XeGPU/>'xegpu' Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/Builtin/>Builtin Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/MatchOpInterfaces/>OpInterface definitions</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/SPIR-V/>SPIR-V Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/TOSA/>Tensor Operator Set Architecture (TOSA) Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Dialects/Transform/>Transform Dialect</a></li></ul></li><li><a href=https://mlir.llvm.org/docs/Interfaces/>Interfaces</a></li><li><a href=https://mlir.llvm.org/docs/TargetLLVMIR/>LLVM IR Target</a></li><li><a href=https://mlir.llvm.org/docs/BytecodeFormat/>MLIR Bytecode Format</a></li><li><a href=https://mlir.llvm.org/docs/CAPI/>MLIR C API</a></li><li><a href=https://mlir.llvm.org/docs/LangRef/>MLIR Language Reference</a></li><li><a href=https://mlir.llvm.org/docs/ReleaseNotes/>MLIR Release Notes</a></li><li><a href=https://mlir.llvm.org/docs/Canonicalization/>Operation Canonicalization</a></li><li><a href=https://mlir.llvm.org/docs/OwnershipBasedBufferDeallocation/>Ownership-based Buffer Deallocation</a></li><li><a href=https://mlir.llvm.org/docs/PassManagement/>Pass Infrastructure</a></li><li><a href=https://mlir.llvm.org/docs/Passes/>Passes</a></li><li><a href=https://mlir.llvm.org/docs/PatternRewriter/>Pattern Rewriting : Generic DAG-to-DAG Rewriting</a></li><li><a href=https://mlir.llvm.org/docs/PDLL/>PDLL - PDL Language</a></li><li><a href=https://mlir.llvm.org/docs/Quantization/>Quantization</a></li><li class=has-sub-menu><a href=https://mlir.llvm.org/docs/Rationale/>Rationale+</a><ul class=sub-menu><li><a href=https://mlir.llvm.org/docs/Rationale/RationaleGenericDAGRewriter/>Generic DAG Rewriter Infrastructure Rationale</a></li><li><a href=https://mlir.llvm.org/docs/Rationale/RationaleLinalgDialect/>Linalg Dialect Rationale: The Case For Compiler-Friendly Custom Operations</a></li><li><a href=https://mlir.llvm.org/docs/Rationale/Rationale/>MLIR Rationale</a></li><li><a href=https://mlir.llvm.org/docs/Rationale/MLIRForGraphAlgorithms/>MLIR: Incremental Application to Graph Algorithms in ML Frameworks</a></li><li><a href=https://mlir.llvm.org/docs/Rationale/RationaleSimplifiedPolyhedralForm/>MLIR: The case for a simplified polyhedral form</a></li><li><a href=https://mlir.llvm.org/docs/Rationale/SideEffectsAndSpeculation/>Side Effects & Speculation</a></li><li><a href=https://mlir.llvm.org/docs/Rationale/UsageOfConst/>Usage of 'const' in MLIR, for core IR types</a></li></ul></li><li><a href=https://mlir.llvm.org/docs/ShapeInference/>Shape Inference</a></li><li><a href=https://mlir.llvm.org/docs/SPIRVToLLVMDialectConversion/>SPIR-V Dialect to LLVM Dialect conversion manual</a></li><li><a href=https://mlir.llvm.org/docs/SymbolsAndSymbolTables/>Symbols and Symbol Tables</a></li><li><a href=https://mlir.llvm.org/docs/DeclarativeRewrites/>Table-driven Declarative Rewrite Rule (DRR)</a></li><li class=has-sub-menu><a href=https://mlir.llvm.org/docs/Traits/>Traits+</a><ul class=sub-menu><li><a href=https://mlir.llvm.org/docs/Traits/Broadcastable/>The `Broadcastable` Trait</a></li></ul></li><li class="parent has-sub-menu"><a href=https://mlir.llvm.org/docs/Tutorials/>Tutorials-</a><ul class=sub-menu><li><a href=https://mlir.llvm.org/docs/Tutorials/CreatingADialect/>Creating a Dialect</a></li><li><a href=https://mlir.llvm.org/docs/Tutorials/QuickstartRewrites/>Quickstart tutorial to adding MLIR graph rewrite</a></li><li class=has-sub-menu><a href=https://mlir.llvm.org/docs/Tutorials/Toy/>Toy Tutorial+</a><ul class=sub-menu><li><a href=https://mlir.llvm.org/docs/Tutorials/Toy/Ch-1/>Chapter 1: Toy Language and AST</a></li><li><a href=https://mlir.llvm.org/docs/Tutorials/Toy/Ch-2/>Chapter 2: Emitting Basic MLIR</a></li><li><a href=https://mlir.llvm.org/docs/Tutorials/Toy/Ch-3/>Chapter 3: High-level Language-Specific Analysis and Transformation</a></li><li><a href=https://mlir.llvm.org/docs/Tutorials/Toy/Ch-4/>Chapter 4: Enabling Generic Transformation with Interfaces</a></li><li><a href=https://mlir.llvm.org/docs/Tutorials/Toy/Ch-5/>Chapter 5: Partial Lowering to Lower-Level Dialects for Optimization</a></li><li><a href=https://mlir.llvm.org/docs/Tutorials/Toy/Ch-6/>Chapter 6: Lowering to LLVM and CodeGeneration</a></li><li><a href=https://mlir.llvm.org/docs/Tutorials/Toy/Ch-7/>Chapter 7: Adding a Composite Type to Toy</a></li></ul></li><li class="parent has-sub-menu"><a href=https://mlir.llvm.org/docs/Tutorials/transform/>Transform Dialect Tutorial-</a><ul class=sub-menu><li><a href=https://mlir.llvm.org/docs/Tutorials/transform/Ch0/>Chapter 0: A Primer on “Structured” Linalg Operations</a></li><li class=active><a href=https://mlir.llvm.org/docs/Tutorials/transform/Ch1/>Chapter 1: Combining Existing Transformations</a></li><li><a href=https://mlir.llvm.org/docs/Tutorials/transform/Ch2/>Chapter 2: Adding a Simple New Transformation Operation</a></li><li><a href=https://mlir.llvm.org/docs/Tutorials/transform/Ch3/>Chapter 3: More than Simple Transform Operations</a></li><li><a href=https://mlir.llvm.org/docs/Tutorials/transform/Ch4/>Chapter 4: Matching Payload with Transform Operations</a></li><li><a href=https://mlir.llvm.org/docs/Tutorials/transform/ChH/>Chapter H: Reproducing Halide Schedule</a></li></ul></li><li><a href=https://mlir.llvm.org/docs/Tutorials/UnderstandingTheIRStructure/>Understanding the IR Structure</a></li><li><a href=https://mlir.llvm.org/docs/Tutorials/MlirOpt/>Using `mlir-opt`</a></li><li><a href=https://mlir.llvm.org/docs/Tutorials/DataFlowAnalysis/>Writing DataFlow Analyses in MLIR</a></li></ul></li></ul></li></ul></nav><div class=sidebar-footer></div></div></div><a href=# id=backtothetop-fixed class=backtothetop data-backtothetop-duration=600 data-backtothetop-easing=easeOutQuart data-backtothetop-fixed-fadein=1000 data-backtothetop-fixed-fadeout=1000 data-backtothetop-fixed-bottom=10 data-backtothetop-fixed-right=20> </a></div></body></html>