CINXE.COM

<!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Planet Raku</title> <link rel="alternate" type="application/atom+xml" title="Atom 0.3" href="/atom.xml"/> <style> @import url("style.css"); </style> <meta name="viewport" content="width=device-width, initial-scale=1"> </head> <body> <img align="left" hspace="8" text="camelia" src="200px-Camelia.svg.png"/> <h1>Planet Raku</h1> <p>Raku RSS Feeds</p> <address>Elizabeth Mattijsen (Libera: lizmat #raku) / 2025-03-20T12:59:42</address> <br/><br/> <div> <script> (function() { var cx = '004466830701554036267:yflzditg_ra'; var gcse = document.createElement('script'); gcse.type = 'text/javascript'; gcse.async = true; gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') + '//cse.google.com/cse.js?cx=' + cx; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(gcse, s); })(); </script> <gcse:search></gcse:search> </div> <h2><a href="https://rakudoweekly.blog/2025/03/17/2025-11-cuddlies-for-sale/">2025.11 Cuddlies for Sale</h2></a> <p><h3>Published by Elizabeth Mattijsen on 2025-03-17T19:43:59</h3></p> <p>Following up on a long-standing request by many, <em>Wendy van Dijk</em> has made it possible for anybody to get a Camelia cuddly toy (well, at a price) in <a href="https://www.lastdodo.com/en/shops/vandijkwendyga/items/159297791-camelia-butterfly?item_id=11047443">their shop</a> without actually having to go to a conference where they’d be selling these at special conference prices. Lovely stuff, especially if you have kids, or are still a kid at heart!</p> <h3 class="wp-block-heading">Paweł’s Corner</h3> <p><em>Paweł bbkr Pabian</em> published about the problems when extending existing code in: <a href="https://dev.to/bbkr/guard-state-transitions-with-proto-methods-3b1n">Guard state transitions with proto methods</a> (<a href="https://www.reddit.com/r/rakulang/comments/1j88m7d/guard_state_transitions_with_proto_methods_paweł/">/r/rakulang</a> comments).</p> <h3 class="wp-block-heading">Elizabeth’s Corner</h3> <p><em>Elizabeth Mattijsen</em> published two blog posts this week: one about the work they did on <code>.assuming</code> (<a href="https://dev.to/lizmat/quicker-to-assume-2cpj">Quicker to assume</a>) and the second part about their work on the <code><a href="https://raku.land/zef:lizmat/REPL">REPL</a></code> distribution: <a href="https://dev.to/lizmat/debugging-by-repl-43ff">Debugging by REPL</a>.</p> <h3 class="wp-block-heading">Fernando’s Corner</h3> <p><em>Fernando Correa de Oliveira</em> continued their series of blog posts about Event Sourcing in a blog post about making an Event Store using <a href="https://raku.land/zef:FCO/Red">Red</a>: <a href="https://dev.to/fco/redeventstore-3o9o">RedEventStore</a>.</p> <h3 class="wp-block-heading">Weeklies</h3> <p><a href="https://theweeklychallenge.org/blog/perl-weekly-challenge-313/">Weekly Challenge #313</a> is available for your perusal.</p> <h3 class="wp-block-heading">New Problem Solving Issues</h3> <ul class="wp-block-list"> <li><a href="https://github.com/raku/problem-solving/issues/468">Consider adding a leading modifier to a string in the manner of “0b10” to identify numbers of any allowed base</a></li> <li><a href="https://github.com/raku/problem-solving/issues/469">Provide an easy way to specify argument values if an argument is not defined, rather than absent</a></li> <li><a href="https://github.com/raku/problem-solving/issues/470">Precedence on methodcall-assign: <code>$foo .= bar.baz</code> doesn’t do what most people expect</a></li> </ul> <h3 class="wp-block-heading">New Pull Requests</h3> <ul class="wp-block-list"> <li><a href="https://github.com/MoarVM/MoarVM/pull/1923">Add 32bit/64bit native int write support</a></li> <li><a href="https://github.com/rakudo/rakudo/pull/5811">Add support for <code>RAKUDO_NOPATH_STACKFRAME</code> envvar</a></li> <li><a href="https://github.com/rakudo/rakudo/pull/5817">RakuAST: improve type check, there’s still a lot of improvement possible</a></li> </ul> <h3 class="wp-block-heading">Core Developments</h3> <ul class="wp-block-list"> <li><em>Daniel Green</em> and <em>Timo Paulssen</em> fixed an issue with mimalloc at exit time on MoarVM.</li> <li><em>Stefan Seifert</em> fixed a potential segfault in generating object IDs, and an issue with signatures containing multiple slurpies, and an issue with the <code>will</code> trait.</li> <li><em>Timo Paulssen</em> fixed an issue with nameless compilation units and precompilation.</li> <li><em>Elizabeth Mattijsen</em> removed the leap-second check from the release process, made the error message of using a regex as sub a bit clearer.</li> </ul> <p>In RakuAST developments this week:</p> <p><em>Stefan Seifert</em> started focusing on bootstrapping the new Raku grammar from scratch (whereas until now it assumed there was a working Raku available) as opposed to try fixing errors in roast. This work is available in a branch as of this writing, and the number of passing spectest files in this fully bootstrapped implementation of the Raku Programming Language is now already <strong>1228</strong> (out of 1345, as opposed to 1279 in the non-bootstrapped version). Another major step forward to making RakuAST mainstream!</p> <ul class="wp-block-list"> <li><em>Jimmy Zhuo</em> and <em>Daniel Green</em> fixed several error reporting issues.</li> <li><em>Elizabeth Mattijsen</em> added support for literal expression using superscripts such as <code>2³⁺²ⁱ</code>, added support for <code>unit sub MAIN</code>, and using colonpairs as a list of <code>Pair</code>s in an assignment.</li> <li><em>Stefan Seifert</em> also fixed quite a few issues (and that’s an understatement!) in the non-bootstrapped RakuAST as well.</li> <li>The number of passing test-files with the new Raku grammar are now 141/153 (<code>make test</code> +0) and 1228/1279/1345 (<code>make spectest</code> +1228/+4).</li> </ul> <h3 class="wp-block-heading">Questions about Raku</h3> <ul class="wp-block-list"> <li><a href="https://stackoverflow.com/questions/79500484/writing-async-routines-use-start-or-promise-then">Writing async routines: use <code>start</code> or <code>Promise.then</code>?</a> by <em>fingolfin</em>.</li> </ul> <h3 class="wp-block-heading">Comments about Raku</h3> <ul class="wp-block-list"> <li><a href="https://news.ycombinator.com/item?id=43355120">Elegant way of decomposition</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://perlmonks.org/?node_id=11164241">Raku is one of them</a> by <em>LanX</em>.</li> </ul> <h3 class="wp-block-heading">New Raku Modules</h3> <ul class="wp-block-list"> <li><a href="https://raku.land/zef:lizmat/Slang::NumberBase">Slang::NumberBase</a> “Provide number base literals” by <em>Elizabeth Mattijsen</em>.</li> </ul> <h3 class="wp-block-heading">Updated Raku Modules</h3> <ul class="wp-block-list"> <li><a href="https://raku.land/zef:tbrowder/Number::More">Number::More</a> by <em>Tom Browder</em>.</li> <li><a href="https://raku.land/zef:librasteve/App::Crag">App::Crag</a>, <a href="https://raku.land/zef:librasteve/Air::Play">Air::Play</a>, <a href="https://raku.land/zef:librasteve/Air">Air</a>, <a href="https://raku.land/zef:librasteve/CLI::AWS::EC2-Simple">CLI::AWS::EC2-Simple</a> by <em>Steve Roe</em>.</li> <li><a href="https://raku.land/zef:melezhik/Sparrow6">Sparrow6</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://raku.land/zef:lizmat/REPL">REPL</a> by <em>Elizabeth Mattijsen</em>.</li> <li><a href="https://raku.land/zef:jjatria/PublicSuffix">PublicSuffix</a> by <em>JJ Atria</em>.</li> <li><a href="https://raku.land/zef:antononcube/Math::NumberTheory">Math::NumberTheory</a> by <em>Anton Antonov</em>.</li> <li><a href="https://raku.land/zef:ingy/YAMLScript">YAMLScript</a> by <em>Ingy döt Net</em>.</li> </ul> <h3 class="wp-block-heading"><a href="https://github.com/lizmat/articles/blob/main/weekly-2022.51.md#winding-down"></a>Winding down</h3> <p>Wow! So much core development progress, powered by <em>Stefan Seifert</em>! Please keep staying safe and healthy, and keep up the good work! Still even after this even crazier week 8 of hopefully only 209.</p> <p> Meanwhile, still: Слава Україні!  Героям слава!</p> <p class="has-small-font-size">If you like what I’m doing, committing to a <a href="https://github.com/sponsors/lizmat/">small sponsorship</a> would mean a great deal!</p> <h2><a href="https://dev.to/fco/redeventstore-3o9o">RedEventStore</h2></a> <p><h3>Published by Fernando Correa de Oliveira on 2025-03-15T01:24:47</h3></p> <p>Recently, I wrote my <a href="https://dev.to/fco/sourcing-part-1-exploring-event-sourcing-in-raku-5e3">first blog post on this blog</a> about <a href="https://github.com/FCO/Sourcing" rel="noopener noreferrer">Sourcing</a>, a Raku framework I'm developing for Event Sourcing. One crucial component of Event Sourcing is the Event Store, which is responsible for storing all domain events in an append-only manner. This allows for event replay, debugging, auditing, and system reconstruction.</p> <p>The goal of Sourcing is to be Event Store agnostic. However, to demonstrate its functionality, it is useful to provide a small reference implementation. To achieve this, I decided to create a simple and lightweight Event Store using a database. The easiest way to do this in Raku (in my opinion) is by leveraging the <a href="https://github.com/FCO/Red" rel="noopener noreferrer">Red ORM</a>.</p> <h2> What is an ORM, and how is Red different? </h2> <p>An Object-Relational Mapper (ORM) is a tool that allows developers to interact with databases using objects and methods instead of raw SQL queries. ORMs typically map database tables to classes, making it easier to manage and query data programmatically.</p> <p>Red is unique compared to most ORMs because instead of merely implementing SQL within Raku, it integrates SQL seamlessly with Raku's syntax and capabilities. This approach provides more expressive and powerful query-building capabilities while maintaining the idiomatic feel of Raku.</p> <h2> Query by ID and Type </h2> <p>An Event Store needs to store events, so let's start with a basic definition:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>use Red:api<2>; use Red::Type::Json; unit model Event; has UInt $.id is serial; has Str $.type is column; has Json $.data is column; </code></pre> </div> <p>This defines a table with three columns: <code>id</code> (a primary key with auto-increment), <code>type</code> (a string indicating the event type), and <code>data</code> (a JSON column storing event details). To create a new event, you can run:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>Event.^create: :type<EventType>, :data{bla => 1, ble => 2}; </code></pre> </div> <p>To retrieve all events after a specific ID (e.g., 42):<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>Event.^all.grep: *.id > 42; </code></pre> </div> <p>To filter events by a specific type or a set of types:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>Event.^all.grep: *.type ⊂ <Type1 Type2 Type3>; </code></pre> </div> <p>In Raku, the <code>.grep</code> method returns a <code>Seq</code>, which lazily processes data. Red extends this behavior by returning a <code>ResultSeq</code>, which generates SQL queries without executing them immediately. This allows additional filtering before executing the query. Given this, we can define a method to retrieve events with optional filters for a minimum ID and a set of event types:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>method get-events(UInt $seq = 0, :@types) { my $events = Event.^all; $events .= grep(*.id > $seq) if $seq; $events .= grep(*.type ⊂ @types) if @types; $events; } </code></pre> </div> <h2> Query by Parent Types </h2> <p>An Event Store should also support querying events based on their parent types. To achieve this, we create a related table to store type hierarchies:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>use Red:api<2>; use Red::Type::Json; unit model Event; has UInt $.id is serial; has Str $.type is column; has Json $.data is column; has @.parents is relationship( *.type, :model<EventType> ); </code></pre> </div> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>use Red:api<2>; unit model EventType; has Str $.type is id; has Str $.parent is id; has @.events is relationship( *.type, :model<Event> ); </code></pre> </div> <p>To add an event and store its type hierarchy:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>method add-event($event) { Event.^create: |$event.to-data, :parents[ $event.^mro.map: { %( parent => .^name ) } ] } </code></pre> </div> <p>This inserts the event into the <code>Event</code> table and its <code>^mro</code> parents as <code>(type, parent)</code> pairs into the <code>EventType</code> table.</p> <p>Now, we update the event retrieval method to support filtering by parent types:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>method get-events(UInt $seq = 0, :@types) { my $events = Event.^all; $events .= grep(*.id > $seq) if $seq; $events .= grep(*.parents.parent ⊂ @types) if @types; $events; } </code></pre> </div> <h2> Filtering by Data </h2> <p>Filtering events based on specific data fields is another essential feature. Red supports filtering JSON column data directly. For example, to find events where <code>field1</code> starts with "bla":<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>Event.^all.grep: *.data<field1>.starts-with: "bla" </code></pre> </div> <p>However, this approach checks each row individually, which may be inefficient. To optimize this, we store indexed event data separately:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>use Red:api<2>; use Red::Type::Json; unit model Event; has UInt $.id is serial; has Str $.type is column; has Json $.data is column; has @.parents is relationship( *.type, :model<EventType> ); has @.fields is relationship( *.event-id, :model<EventData> ); </code></pre> </div> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>use Red:api<2>; unit model EventData; has UInt $.event-id is column{ :id, :references{ .id }, :model-name<Event> }; has Str $.field is required is id; has $.value is column{ :id, :nullable }; has $.event is relationship( *.event-id, :model<Event> ); </code></pre> </div> <p>To add events and store indexed data fields:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>method add-event($event) { Event.^create: |$event.to-data, :parents[$event.^mro.map: { %( parent => .^name ) }], :fields[$event.get-data.kv.map: -> $k, $v { next if $v ~~ Positional | Associative; %( field => $k, value => $v ) }] } </code></pre> </div> <p>Now, when querying events based on data fields:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>my $events = do if %pars { my $fields = EventData.^all; for %pars.kv -> $key, $value { FIRST { $fields .= grep: { .field eq $key && .value eqv $value } next; } $fields .= join-model: :name("field_$key"), EventData, -> $prev, $field { $prev.event-id == $field.event-id && $field.field eq $key && $field.value eqv $value } } $fields.map: { .event }; } else { Event.^all; }; </code></pre> </div> <h2> Conclusion </h2> <p>By implementing an Event Store using Red, we gain a flexible, efficient, and Raku-idiomatic way to manage and query event data.<br> With its powerful ORM features and SQL integration, Red simplifies event storage, filtering, and retrieval.<br> This approach ensures that Event Sourcing in Raku remains both accessible and performant.<br> Future enhancements could include support for more advanced queries, indexing strategies, and additional database backends.</p> <h2><a href="https://dev.to/lizmat/debugging-by-repl-43ff">Debugging by REPL</h2></a> <p><h3>Published by Elizabeth Mattijsen on 2025-03-12T18:17:07</h3></p> <blockquote> <p>This is part 2 in the <a href="https://dev.to/lizmat/series/30566">REPL Avalanche</a> blog series.</p> </blockquote> <p>The <a href="https://raku.land/zef:lizmat/REPL" rel="noopener noreferrer"><code>REPL</code></a> distribution also offers a <code>repl</code> subroutine apart from a command-line interface (CLI). A call to this subroutine can be placed in source code during development of a code-base, and then allows for some ad-hoc interactive debugging features.</p> <p>For the purpose of fun and taste, I will be calling such a (temporary) placement of a call to the <code>repl</code> sub a <a href="https://en.wikipedia.org/wiki/Sprinkles" rel="noopener noreferrer">"<strong>sprinkle</strong>"</a>.</p> <blockquote> <p>All of the examples in this blog post assume that the <code>REPL</code> distribution has been installed, and has been loaded either explicitely with a <code>use REPL</code> command in your code, or by setting the <code>RAKUDO_OPT</code> environment variable to include <code>-MREPL</code>.</p> </blockquote> <h2> Sprinkling differently </h2> <p>One of the ways you can use the <a href="https://raku.land/zef:lizmat/REPL" rel="noopener noreferrer"><code>REPL</code></a> distribution is as a debugging tool. Instead of putting <code>print</code> or <code>dd</code> statements in your code, you can sprinkle <code>repl</code> statements in your code. And then interactively check out the situation when that <code>repl</code> statement is executed. A very contrived and simplified example:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="c1">#line 666 foo.raku</span> <span class="k">my</span> <span class="nv">$answer</span> <span class="o">=</span> <span class="mi">42</span><span class="p">;</span> <span class="k">if</span> <span class="nv">$answer</span> <span class="o">==</span> <span class="mi">42</span> <span class="p">{</span> <span class="nv">repl</span> <span class="p">}</span> </code></pre> </div> <p>would open the REPL with:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>block at foo.raku line 666 [0] > </code></pre> </div> <p>Note that it does <strong>not</strong> show the standard REPL header, which would look like this:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>Welcome to Rakudo™ v2025.02. Implementing the Raku® Programming Language v6.d. Built on MoarVM version 2025.02. To exit type '=quit' or '^D'. Type '=help' for help. </code></pre> </div> <p>because that would become <em>very</em> annoying for regular users of the <code>repl</code> subroutine.</p> <p>Instead of that, it shows the location of the block in which the <code>repl</code> subroutine was called. This should give you a rough idea of where you are in the code at the moment the <code>repl</code> is called. Which can be handy if you have a number of <code>repl</code> calls sprinkled in your code.</p> <p>If you would like to know the value of the variable <code>$answer</code> once in the REPL, then you can enter that variable as you could any other expression:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>[0] > $answer 42 [1] > </code></pre> </div> <p>Because the <code>repl</code> subroutine knows about the runtime context it is being called, it can look up any variable that could be referenced in the location of the code where the <code>repl</code> subroutine was called.</p> <p>Not only can you look up values in variables, you can also change them! Continiuing the above sprinkle:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>[1] > $answer = 137 137 [2] > $answer 137 [3] > </code></pre> </div> <p>This allows one to explore "what-if" scenarios, and the like. Of course, changes are only allowed on things that are changeable, just like they would be in normal execution of code.</p> <h2> Conditional sprinkles </h2> <p>If you would only like to open a REPL if something unexpected happens, you can of course do that as well. Again, a contrived example:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="k">if</span> <span class="mi">42</span> <span class="o">-></span> <span class="nv">$answer</span> <span class="p">{</span> <span class="nv">repl</span> <span class="k">if</span> <span class="nv">$answer</span> <span class="o">!=</span> <span class="mi">42</span><span class="p">;</span> <span class="p">}</span> </code></pre> </div> <p>would only open up a REPL if the answer was wrong. In this example, that will of course be never. But I hope you get the picture!</p> <h2> Identifying sprinkles </h2> <p>Sometimes you are not so much interested in the exact location in the code, but are more interested in a logical step in your code. To allow easy identification of the sprinkle, you can pass a string in the call to the <code>repl</code> subroutine:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="k">if</span> <span class="mi">42</span> <span class="o">-></span> <span class="nv">$answer</span> <span class="p">{</span> <span class="nv">repl</span> <span class="p">"</span><span class="s2">What did Deep Thought say?</span><span class="p">";</span> <span class="p">}</span> </code></pre> </div> <p>would then open the REPL with:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>What did Deep Thought say? [0] > </code></pre> </div> <p>If you want to get a little more whimsical: all of the expansions that are possible in the prompt are also possible in this string. And since we could always use a little fun, why not identify your sprinkle with:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="k">if</span> <span class="mi">42</span> <span class="o">-></span> <span class="nv">$answer</span> <span class="p">{</span> <span class="nv">repl</span> <span class="p">"</span><span class="s2">:clown-face:</span><span class="p">";</span> <span class="p">}</span> </code></pre> </div> <p>which would then open the REPL with:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>🤡 [0] > </code></pre> </div> <p>Or you could express your feeling about this debugging session with "😕" (😕) or ":exploding-head:" (🤯).</p> <blockquote> <p>See the <a href="https://raku.land/zef:lizmat/Text::Emoji#people--body" rel="noopener noreferrer"><code>Text::Emoji</code></a> documentation for some inspiration.</p> </blockquote> <p>Once in the REPL again you could enter "$answer" to find out the answer. But that would get pretty boring in a loop pretty quickly!</p> <h2> Priming sprinkles </h2> <p>If you'd like to know the value of one or more variables or expressions at a specific sprinkle, you can also prime the sprinkle with named arguments. For instance:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="k">if</span> <span class="mi">42</span> <span class="o">-></span> <span class="nv">$answer</span> <span class="p">{</span> <span class="nv">repl</span> <span class="p">"</span><span class="s2">Ready!</span><span class="p">",</span> <span class="p">:</span><span class="nv">$answer</span><span class="p">,</span> <span class="p">:</span><span class="nb">time</span><span class="o"><</span><span class="p">:</span><span class="nv">HHMM:</span><span class="o">></span><span class="p">;</span> <span class="p">}</span> </code></pre> </div> <p>would then open the REPL with:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>Ready! answer: 42 time: 15:10 [0] > </code></pre> </div> <p>which not only would give you the value of the "$answer" variable immediately so you wouldn't have to ask for the answer every time you enter that sprinkle.</p> <p>And you'd know the time: if the value of a primed expression in the call to <code>repl</code> is a string, it will allow for all the expansions that are possible in the prompt as well. In this example ":HHMM:" expands to the current hour and minute.</p> <blockquote> <p>Check the <a href="https://raku.land/zef:lizmat/DateTime::strftime" rel="noopener noreferrer"><code>DateTime::strftime</code></a> documentation for more inspiration.</p> </blockquote> <h2> Fast consumption of sprinkles </h2> <p>Generally, once you're done with a specific sprinkle, you'd like to continue executing your code. So you'd need to leave the REPL with "=quit" (or its shortening: "=q"). That also can get annoying after a while.</p> <p>That's why the REPL gets in a special mode when called as a sprinkle. The "empty" command is then the same as "=quit", so you only need to press ENTER without having entered anything to continue execution.</p> <h2> Conclusion </h2> <p>By using sprinkles in your code, you can both interactively look up values in your code, as well as immediately see any values / expressions that you would be interested in for a specific sprinkle.</p> <p>This is the second part of a series of blog posts about the <a href="https://raku.land/zef:lizmat/REPL" rel="noopener noreferrer"><code>REPL</code></a> distribution. Future installments will look at the available commands, and the completions logic.</p> <p>Image courtesy of <em>Salve J. Nilsen</em>.</p> <p><em>If you like what I'm doing, committing to a <a href="https://github.com/sponsors/lizmat/" rel="noopener noreferrer">small sponsorship</a> would mean a great deal to me!</em></p> <h2><a href="https://dev.to/lizmat/quicker-to-assume-2cpj">Quicker to assume</h2></a> <p><h3>Published by Elizabeth Mattijsen on 2025-03-10T20:30:52</h3></p> <p>The Raku Programming Language has a nice feature that goes by different names: <a href="https://en.wikipedia.org/wiki/Currying" rel="noopener noreferrer">"currying"</a>, <a href="https://docs.raku.org/syntax/priming" rel="noopener noreferrer">"priming"</a> or <a href="https://en.wikipedia.org/wiki/Partial_application" rel="noopener noreferrer">"partial application"</a>. Oddly enough, the method name associated with the feature is called <a href="https://docs.raku.org/routine/assuming" rel="noopener noreferrer"><code>assuming</code></a>.</p> <h2> Some examples </h2> <p>So what does the <code>.assuming</code> method do? In short, it creates a subroutine from an existing subroutine (or method, or pointy block) with one or more arguments already "filled in". For instance:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="k">sub </span><span class="nf">hello</span><span class="p">($firstname, $lastname) {</span> <span class="nv">say</span> <span class="p">"</span><span class="s2">Hello, </span><span class="si">$firstname</span><span class="s2"> </span><span class="si">$lastname</span><span class="p">";</span> <span class="p">}</span> <span class="nv">hello</span> <span class="p">"</span><span class="s2">Joe</span><span class="p">",</span> <span class="p">"</span><span class="s2">Smith</span><span class="p">";</span> <span class="c1"># Hello, Joe Smith</span> <span class="k">my</span> <span class="nv">&hi</span><span class="o">-</span><span class="nv">smith</span> <span class="o">=</span> <span class="nv">&hello</span><span class="o">.</span><span class="nv">assuming</span><span class="p">(</span><span class="o">*</span><span class="p">,</span> <span class="p">'</span><span class="s1">Smith</span><span class="p">');</span> <span class="c1"># fill in 2nd positional</span> <span class="nv">hi</span><span class="o">-</span><span class="nv">smith</span><span class="p">("</span><span class="s2">Joe</span><span class="p">");</span> <span class="c1"># Hello, Joe Smith</span> </code></pre> </div> <p>This is a simple example in which one positional parameter is replaced. Replacing named arguments is also possible, and adding values to slurpy arrays as well.</p> <p>Let's take the <a href="https://docs.raku.org/routine/max#(Operators)_infix_max" rel="noopener noreferrer"><code>max</code></a> subroutine that returns the maximum of the given values. In this case we make sure that the maximum value of any number of values given, is at least 0. By making sure that the value <code>0</code> is always added to any list of values specified. Which also handles the case if called without any arguments.<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="k">my</span> <span class="nv">&max0</span> <span class="o">=</span> <span class="nv">&max</span><span class="o">.</span><span class="nv">assuming</span><span class="p">(</span><span class="mi">0</span><span class="p">);</span> <span class="c1"># always add 0 as a value to be checked</span> <span class="nv">say</span> <span class="nv">max</span> <span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">-</span><span class="mi">2</span><span class="p">,</span> <span class="o">-</span><span class="mi">3</span><span class="p">;</span> <span class="c1"># -1</span> <span class="nv">say</span> <span class="nv">max0</span> <span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">-</span><span class="mi">2</span><span class="p">,</span> <span class="o">-</span><span class="mi">3</span><span class="p">;</span> <span class="c1"># 0, because 0 is greater than -1;</span> <span class="nv">say</span> <span class="nv">max</span><span class="p">;</span> <span class="c1"># -Inf, smallest possible numeric value</span> <span class="nv">say</span> <span class="nv">max0</span><span class="p">;</span> <span class="c1"># 0, because max(0) is 0</span> </code></pre> </div> <h2> Original implementation </h2> <p>The original implementation of <code>.assuming</code> was done by <em>Brian S. Julin</em> in July 2015. The commit message mentioned:</p> <blockquote> <p>This currently uses EVAL to construct the closure, which is LTA, but it gives us something functional/testable to work forward from.</p> </blockquote> <p>In the close to 10 years since then, only small tweaks and fixes have been applied by a group of Rakudo core developers. But the essentially hacky approach of building a string with Raku code, and then <a href="https://docs.raku.org/routine/EVAL" rel="noopener noreferrer"><code>EVAL</code></a> it, did not change. Simply because there were no alternative solutions.</p> <p>Apart from being hacky, the <code>EVAL</code> approach added quite a bit of <em>runtime</em> overhead. Not only would it take time to create the string with Raku code to be evaluated, and run the evaluation itself (which could potentially happen at compile time, so not a real worry for modules). It would also process arguments at runtime. That runtime overhead also caused an issue to be made in early 2019: <a href="https://github.com/rakudo/rakudo/issues/2599" rel="noopener noreferrer">.assuming is painfully slow</a>. But since there was no alternative approach available, the issue remained dormant for more than 6 years.</p> <h2> RakuAST </h2> <p>Fastforward to today. There is now an alternative way of building code that is to be executed: <a href="https://docs.raku.org/type/RakuAST" rel="noopener noreferrer"><code>RakuAST</code></a>.</p> <p>The <a href="https://news.perlfoundation.org/post/gp_rakuast" rel="noopener noreferrer">RakuAST project</a> was started by <em>Jonathan Worthington</em>. From the grant proposal:</p> <blockquote> <p>The goal of RakuAST is to provide an <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree" rel="noopener noreferrer">AST</a> that is part of the Raku language specification, and thus can be relied upon by the language user. Such an AST is a prerequisite for a useful implementation of macros that actually solve practical problems, but also offers further powerful opportunities for the module developer.</p> </blockquote> <p>About two years ago yours truly blogged a little about it already: <a href="https://dev.to/lizmat/rakuast-for-early-adopters-576n">RakuAST for Early Adopters</a>. Since then, development has continued to a point where almost all features of Raku can be expressed using the new Raku grammar, which builds its AST using RakuAST (now at 94.9% of test files completely passing).</p> <p>Having looked at making <code>.assuming</code> produce faster code in 2019 already, it felt like a good time to do another attempt. But this time using RakuAST!</p> <h2> Prototyping </h2> <p>The goal would be that for a simple case like:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="k">sub </span><span class="nf">foo</span><span class="p">(Int $a) {</span> <span class="o">...</span> <span class="p">}</span> <span class="k">my</span> <span class="nv">&bar</span> <span class="o">=</span> <span class="nv">&foo</span><span class="o">.</span><span class="nv">assuming</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span> </code></pre> </div> <p><code>.assuming</code> would effectively create a subroutine <code>bar</code> that would just be:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="k">sub </span><span class="nf">bar</span><span class="p">()</span> <span class="p">{</span> <span class="nv">foo</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span> <span class="p">}</span> </code></pre> </div> <p>without any additional runtime overhead.</p> <p>Just over two weeks ago a prototype was created. Outside of the core setting, just in a simple script. And the initial results turned out to be promising: some simple examples turned out to be already 40x as fast as the original implementation. As more features were added, that number went down for some cases as was to be expected.</p> <p>As for the "more features" bit: wow, quite a number of features that you can have in the signature of a subroutine!</p> <h2> Nooks and crannies </h2> <p>Or how I learned more about signatures than I would ever like to know. Apart from positional and named arguments, you can also have slurpy arrays (3 types) and slurpy hashes. And captures (named or nameless), without or without sub-signatures. And then you can specify a <em>variable</em> as a value, which can be changed by the subroutine if it was defined with <code>is raw</code>. Various constraints can be applicable: as code blocks, or as types: possibly parameterized, and/or coerced and/or with a type smiley. And then there are generics (aka type object placeholders). Put them all together in a mix and let the fun begin!</p> <p>That was actually the most work: getting all of the interactions right. And while doing that, 3 bugs were found that had to do with the use of generics in parameterization, and role composition. One of them is fortunately already fixed.</p> <p>In any case, as of this writing it appears that every possible combination of signature features are supported by the new implementation of <code>.assuming</code>.</p> <h2> Introspection </h2> <p>In Raku one can easily introspect aspects of code: given a <code>Sub</code> object, one can ask for its signature (which is a <a href="https://docs.raku.org/type/Signature" rel="noopener noreferrer"><code>Signature</code></a> object). And given a <code>Signature</code> object, one can ask for its parameters (which would be <a href="https://docs.raku.org/type/Parameter" rel="noopener noreferrer"><code>Parameter</code></a> objects), and one can ask for the return value / type constraint (which would generally be a <a href="https://docs.raku.org/syntax/Type%20Objects" rel="noopener noreferrer">type object</a>. And given a <code>Parameter</code> object, one can ask whether it is a named or positional parameter, whether it optional or not, its constraint and many other aspects.</p> <p>To be able to build a new <code>Sub</code> object, these aspects of signatures, parameters and types need to be converted to <code>RakuAST</code> objects. Any code trying to do similar things as <code>.assuming</code>, could benefit from this logic. Therefore this part of the work on <code>.assuming</code> has also been published as the <a href="https://raku.land/zef:lizmat/RakuAST::Utils" rel="noopener noreferrer"><code>RakuAST::Utils</code></a> distribution.</p> <p>To give you an idea how this looks like:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="k">use</span> <span class="nn">RakuAST::</span><span class="nv">Utils</span><span class="p">;</span> <span class="k">sub </span><span class="nf">foo</span><span class="p">(Int $a, Int(Str) $b --> Str:D) {</span> <span class="p">"</span><span class="s2">foo</span><span class="p">"</span> <span class="p">}</span> <span class="nv">say</span> <span class="nv">SignatureAST</span><span class="p">(</span><span class="nv">&foo</span><span class="o">.</span><span class="nv">signature</span><span class="p">);</span> <span class="c1"># RakuAST::Signature.new(</span> <span class="c1"># parameters => (</span> <span class="c1"># RakuAST::Parameter.new(</span> <span class="c1"># type => RakuAST::Type::Simple.new(</span> <span class="c1"># RakuAST::Name.from-identifier("Int")</span> <span class="c1"># ),</span> <span class="c1"># target => RakuAST::ParameterTarget::Var.new(</span> <span class="c1"># name => "\$a"</span> <span class="c1"># )</span> <span class="c1"># ),</span> <span class="c1"># RakuAST::Parameter.new(</span> <span class="c1"># type => RakuAST::Type::Coercion.new(</span> <span class="c1"># base-type => RakuAST::Type::Simple.new(</span> <span class="c1"># RakuAST::Name.from-identifier("Int")</span> <span class="c1"># ),</span> <span class="c1"># constraint => RakuAST::Type::Simple.new(</span> <span class="c1"># RakuAST::Name.from-identifier("Str")</span> <span class="c1"># )</span> <span class="c1"># ),</span> <span class="c1"># target => RakuAST::ParameterTarget::Var.new(</span> <span class="c1"># name => "\$b"</span> <span class="c1"># )</span> <span class="c1"># ),</span> <span class="c1"># ),</span> <span class="c1"># returns => RakuAST::Type::Definedness.new(</span> <span class="c1"># base-type => RakuAST::Type::Simple.new(</span> <span class="c1"># RakuAST::Name.from-identifier("Str")</span> <span class="c1"># ),</span> <span class="c1"># definite => True</span> <span class="c1"># )</span> <span class="c1"># )</span> </code></pre> </div> <p>The functionality offered by this distribution could possibly become core at some point, but the interface / API should mature a bit before that. Also, the RakuAST interface itself isn't fully stable yet, so this module can provide a more stable interface as a central place to handle any RakuAST interface changes.</p> <h2> Testing </h2> <p>There were quite a few tests for the <code>.assuming</code> logic in the <a href="https://github.com/raku/roast?tab=readme-ov-file#the-official-raku-test-suite" rel="noopener noreferrer">official Raku test suite (aka "roast")</a>. Alas, many of them were testing implementation details of the signatures created by the old (incomplete) implementation. They have either been adjusted, or removed altogether. And some new tests have been added, to cover some features that were not supported yet.</p> <h2> Performance </h2> <p>Benchmarking during the development process showed that the <code>RakuAST</code> approach to <code>.assuming</code> was between 2.5x and 50x faster than the old string <code>EVAL</code>. However, the slowdown that was mentioned in the <a href="https://github.com/rakudo/rakudo/issues/2599" rel="noopener noreferrer">issue</a> turned out to be underestimated: quite a few tests showed an actual slowdown of up to 80x. So even being 50x faster on a 80x slower case, is still significantly slower :-(</p> <p>A small benchmark on a 2020 M1 Apple Silicon processor (no JIT available):<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="k">sub </span><span class="nf">a</span><span class="p">($a) {</span> <span class="nv">$a</span> <span class="p">}</span> <span class="k">sub </span><span class="nf">b</span><span class="p">()</span> <span class="p">{</span> <span class="nv">a</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span> <span class="p">}</span> <span class="k">BEGIN</span> <span class="k">my</span> <span class="nv">&c</span> <span class="o">=</span> <span class="nv">&a</span><span class="o">.</span><span class="nv">assuming</span><span class="p">(</span><span class="mi">42</span><span class="p">);</span> <span class="k">my</span> <span class="nv">&d</span> <span class="o">=</span> <span class="nv">&a</span><span class="o">.</span><span class="nv">assuming</span><span class="p">(</span><span class="mi">42</span><span class="p">);</span> <span class="p">{</span> <span class="nv">a</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span> <span class="k">for</span> <span class="o">^</span><span class="mi">10_000_000</span><span class="p">;</span> <span class="nv">say</span> <span class="nv">now</span> <span class="o">-</span> <span class="nv">ENTER</span> <span class="nv">now</span> <span class="p">}</span> <span class="c1"># 0.352358404</span> <span class="p">{</span> <span class="nv">b</span><span class="p">()</span> <span class="k">for</span> <span class="o">^</span><span class="mi">10_000_000</span><span class="p">;</span> <span class="nv">say</span> <span class="nv">now</span> <span class="o">-</span> <span class="nv">ENTER</span> <span class="nv">now</span> <span class="p">}</span> <span class="c1"># 0.415636707</span> <span class="p">{</span> <span class="nv">c</span><span class="p">()</span> <span class="k">for</span> <span class="o">^</span><span class="mi">10_000_000</span><span class="p">;</span> <span class="nv">say</span> <span class="nv">now</span> <span class="o">-</span> <span class="nv">ENTER</span> <span class="nv">now</span> <span class="p">}</span> <span class="c1"># 1.234486065</span> <span class="p">{</span> <span class="nv">d</span><span class="p">()</span> <span class="k">for</span> <span class="o">^</span><span class="mi">10_000_000</span><span class="p">;</span> <span class="nv">say</span> <span class="nv">now</span> <span class="o">-</span> <span class="nv">ENTER</span> <span class="nv">now</span> <span class="p">}</span> <span class="c1"># 1.404967796</span> </code></pre> </div> <p>Which shows that the new <code>.assuming</code> approach (<code>c</code>) is still almost 3x as slow (1.234486065 / 0.415636707 = 2.97) as a handmade intermediate subroutine (<code>b</code>).</p> <p>Also note that running <code>.assuming</code> at runtime makes it even slower still (3.38x): probably because it then misses some static optimizations that are run at compile time.</p> <p>With more work done on RakuAST, specifically with regards to optimizations, it should be possible to eliminate the difference between <code>b()</code> and <code>c()</code>.</p> <p>On the other hand, please keep in perspective that these benchmarks were run for <strong>10 million times</strong>, where the runtime was mostly spent in calling a subroutine, and not much else. In any fleshed out subroutine, the <code>.assuming</code> overhead will probably be drowned away in the time needed to execute the rest of the code in the subroutine.</p> <h2> Conclusion </h2> <p>The RakuAST project has reached such a maturity that it could be used to re-imagine the currying / priming / partial applocation logic of the Raku Programming Language. And that this re-imagination has made that functionality an order of magnitude faster, while still not being fully optimized yet.</p> <p>Thanks to the <a href="https://raku.foundation" rel="noopener noreferrer">Raku Foundation</a> for supporting this work on <code>.assuming</code>.</p> <p><em>If you like what I'm doing, committing to a <a href="https://github.com/sponsors/lizmat/" rel="noopener noreferrer">small sponsorship</a> would mean a great deal to me!</em></p> <h2><a href="https://rakudoweekly.blog/2025/03/10/2025-10-final-cfp-week/">2025.10 Final CFP Week</h2></a> <p><h3>Published by Elizabeth Mattijsen on 2025-03-10T13:05:12</h3></p> <p>The <a href="https://www.papercall.io/tprcgsp2025">Call For Presentations</a> for the <a href="https://tprc.us/tprc-2025-gsp/">Conference in Greenville, SC on June 27-29 2025 </a>closes on 15 March. If you plan to do a presentation, make sure it is submitted before the deadline!</p> <h3 class="wp-block-heading">Foundation News</h3> <p><em>Saif Ahmed</em> has posted an update to the <a href="https://news.perlfoundation.org/post/raku-ecosystem-tonyo-final">Raku Ecosystem Grant Report</a>.</p> <h3 class="wp-block-heading">Anton’s Corner</h3> <p><em>Anton Antonov</em> published two blog posts in the past week, both based on their <a href="https://raku.land/zef:antononcube/Graph">Graph</a> module:</p> <ul class="wp-block-list"> <li><a href="https://rakuforprediction.wordpress.com/2025/03/03/rock-paper-scissors-extensions/">Rock-Paper-Scissors extensions</a></li> <li><a href="https://rakuforprediction.wordpress.com/2025/03/06/military-forces-interactions-graphs/">Military forces interactions graphs</a></li> </ul> <h3 class="wp-block-heading">Weeklies</h3> <p><a href="https://theweeklychallenge.org/blog/perl-weekly-challenge-312/">Weekly Challenge #312</a> is available for your perusal.</p> <h3 class="wp-block-heading">New Pull Requests</h3> <ul class="wp-block-list"> <li><a href="https://github.com/Raku/doc/pull/4564">Added initial revision of Slangs tutorial</a></li> <li><a href="https://github.com/MoarVM/MoarVM/pull/1922">Bump <code>mimalloc</code> to v2.2.2</a></li> <li><a href="https://github.com/rakudo/rakudo/pull/5800">Test and fix that <code>Instant.from-posix-nanos</code> and <code>Instant.from-posix</code> match.</a></li> </ul> <h3 class="wp-block-heading">Core Developments</h3> <ul class="wp-block-list"> <li><em>Daniel Green</em> significantly reduced the number of lexical lookups by skipping frames that don’t have any.</li> <li><em>Timo Paulssen</em> fixed an issue with huge hash literals hogging up much more memory than they should, and fixed an issue with type captures.</li> <li><em>Elizabeth Mattijsen</em> fixed an issue with comparing type objects with <code>===</code>, an issue with memory reporting with <code>snapper</code> on MacOS, and re-imagined <code>.assuming</code> using <code>RakuAST</code> (making the resulting <code>Callable</code> between 2.5x and 50x faster).</li> </ul> <p>In RakuAST developments this week:</p> <ul class="wp-block-list"> <li><em>Stefan Seifert</em> continued working on RakuAST. The most significant fixes: <ul class="wp-block-list"> <li><code>BEGIN</code> time call for non-simple constructs</li> <li>support for <code>%?RESOURCES</code> and <code>$?DISTRIBUTION</code></li> <li>blocks as defaults for parameters</li> <li>many attribute and package stub issues</li> <li>added several warnings</li> <li>and many smaller fixes!</li> </ul> </li> <li><em>Daniel Green</em> and <em>Jimmy Zhuo</em> fixed a number of warnings.</li> <li><em>Elizabeth Mattijsen</em> fixed an issue with literal values.</li> <li>The number of passing test-files with the new Raku grammar are now 141/153 (<code>make test</code> +0) and 1275/1345 (<code>make spectest</code> +31).</li> </ul> <h3 class="wp-block-heading"> Meanwhile on Mastodon</h3> <ul class="wp-block-list"> <li><a href="https://fosstodon.org/@melezhik/114105787125567245">Sparrowdo vs Ansible, in Russian</a> by <em>Alexey Melezhik</em>.</li> </ul> <h3 class="wp-block-heading">Meanwhile on BlueSky</h3> <ul class="wp-block-list"> <li><a href="https://bsky.app/profile/colomon.bsky.social/post/3ljhzsxqiy22d">Awesome!</a> by <em>Solomon Foster</em>.</li> </ul> <h3 class="wp-block-heading">Meanwhile on the mailing list</h3> <ul class="wp-block-list"> <li><a href="https://www.nntp.perl.org/group/perl.perl6.users/2025/03/msg11385.html">Raku Study Group on 23 March</a> by <em>Joseph Brenner</em>.</li> </ul> <h3 class="wp-block-heading">Questions about Raku</h3> <ul class="wp-block-list"> <li><a href="https://bitbang.social/@profoundlynerdy/114127380438905173">Can Cro be used to create a simple website?</a> by <em>Profoundly Nerdy</em>.</li> </ul> <h3 class="wp-block-heading">Comments about Raku</h3> <ul class="wp-block-list"> <li><a href="https://news.ycombinator.com/item?id=43250779">Loved the grammars</a> by <em>harry_ord</em>.</li> <li><a href="https://news.ycombinator.com/item?id=43258315">Raised to the power of the swiss army chainsaw</a> by <em>creer</em>.</li> <li><a href="https://www.buidtech.com/techblogs/niche-programming-languages-2025/">Top Niche Programming Languages in 2025</a> at BUiDTECH.</li> <li><a href="https://www.reddit.com/r/ProgrammingLanguages/comments/1j2g4cs/comment/mfvw2dw/?context=3">Fairly opinionated about being fairly non-opinionated</a> by <em>Ralph Mellor</em>.</li> <li><a href="https://news.ycombinator.com/item?id=43290250">A true pleasure</a> by <em>Mark Devine</em>.</li> <li><a href="https://news.ycombinator.com/item?id=43295959">What about Sanskrit?</a> by <em>fuzztester</em>.</li> </ul> <h3 class="wp-block-heading">New Raku Modules</h3> <ul class="wp-block-list"> <li><a href="https://raku.land/zef:lizmat/RakuAST::Utils">RakuAST::Utils</a> “Provide basic RakuAST utilities” by <em>Elizabeth Mattijsen</em>.</li> </ul> <h3 class="wp-block-heading">Updated Raku Modules</h3> <ul class="wp-block-list"> <li><a href="https://raku.land/zef:tbrowder/Number::More">Number::More</a> by <em>Tom Browder</em>.</li> <li><a href="https://raku.land/zef:librasteve/Air">Air</a>, <a href="https://raku.land/zef:librasteve/Air::Play">Air::Play</a> by <em>Steve Roe</em>.</li> <li><a href="https://raku.land/zef:ingy/YAMLScript">YAMLScript</a> by <em>Ingy döt Net</em>.</li> <li><a href="https://raku.land/zef:jnthn/Dev::ContainerizedService">Dev::ContainerizedService</a> by <em>Jonathan Worthington</em>.</li> <li><a href="https://raku.land/zef:jjatria/PublicSuffix">PublicSuffix</a>, <a href="https://raku.land/zef:jjatria/HTTP::Tiny">HTTP::Tiny</a> by <em>JJ Atria</em>.</li> <li><a href="https://raku.land/zef:antononcube/Graph">Graph</a>, <a href="https://raku.land/zef:antononcube/JavaScript::D3">JavaScript::D3</a> by <em>Anton Antonov</em>.</li> <li><a href="https://raku.land/zef:lizmat/Text::Emoji">Text::Emoji</a> by <em>Elizabeth Mattijsen</em>.</li> <li><a href="https://raku.land/zef:jforget/Date::Calendar::CopticEthiopic">Date::Calendar::CopticEthiopic</a> by <em>Jean Forget</em>.</li> <li><a href="https://raku.land/zef:raku-community-modules/Redis">Redis</a> by <em>Brian Duggan</em>.</li> </ul> <h3 class="wp-block-heading"><a href="https://github.com/lizmat/articles/blob/main/weekly-2022.51.md#winding-down"></a>Winding down</h3> <p>Please keep staying safe and healthy, and keep up the good work! Even after this even crazier week 7 of hopefully only 209.</p> <p> Meanwhile, still: Слава Україні!  Героям слава!</p> <p class="has-small-font-size">If you like what I’m doing, committing to a <a href="https://github.com/sponsors/lizmat/">small sponsorship</a> would mean a great deal!</p> <h2><a href="https://rakuforprediction.wordpress.com/2025/03/06/military-forces-interactions-graphs/">Military forces interactions graphs</h2></a> <p><h3>Published by Anton Antonov Antonov on 2025-03-06T15:25:17</h3></p> <h2 class="wp-block-heading"><a></a>Introduction</h2> <p>Interesting analogies of <a href="https://en.wikipedia.org/wiki/Rock_paper_scissors">Rock-Paper-Scissors</a> (RPS) hand games can be made with military forces interactions; see [AAv1]. Those analogies are easily seen using graphs. For example, the extension of the graph of Rock-Paper-Scissors-Lizard-Spock, [Wv1], into the graph “Chuck Norris defeats all” is analogous to the extension of “older” (say, WWII) military forces interactions graphs with drones.</p> <p>Here is the graph of <em>Rock-Paper-Scissors-Lizard-Spock-ChuckNorris</em>, [<a href="https://rakuforprediction.wordpress.com/2025/03/03/rock-paper-scissors-extensions/" target="_blank" rel="noreferrer noopener">AA1</a>]:</p> <figure class="wp-block-image"><img src="https://raw.githubusercontent.com/antononcube/RakuForPrediction-blog/refs/heads/main/Articles/Diagrams/Rock-Paper-Scissors-extensions/04-RPS-Lizard-Spock-ChuckNorris.png" alt="Chuck Norris defeats all" /></figure> <p>In this document (<a href="https://github.com/antononcube/RakuForPrediction-blog/blob/main/Notebooks/Jupyter/Military-forces-interactions-graphs.ipynb" target="_blank" rel="noreferrer noopener">notebook</a>), we use Raku to create graphs that show how military forces interact. We apply the know-how for making graphs for RPS-games detailed in the blog post <a href="https://rakuforprediction.wordpress.com/2025/03/03/rock-paper-scissors-extensions/" target="_blank" rel="noreferrer noopener">“Rock-Paper-Scissors extensions”</a>, [AA1].</p> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Setup</h2> <p><em>The setup is the same as in [AA1] (<a href="https://github.com/antononcube/RakuForPrediction-blog/blob/main/Notebooks/Jupyter/Rock-Paper-Scissors-extensions.ipynb">notebook</a>).</em></p> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Convenient LLM function</h2> <p>We can define an LLM function that provides the graph edges dataset for different RPS variants. Here is such an LLM function using “LLM::Functions”, [AAp1], and “LLM::Prompts”, [AAv2]:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my sub rps-edge-dataset($description, Str:D $game-name = 'Rock-Paper-Scissors', *%args) { llm-synthesize([ "Give the edges the graph for this $game-name variant description", 'Give the edges as an array of dictionaries. Each dictionary with keys "from", "to", "label",', 'where "label" has the action of "from" over "to".', $description, llm-prompt('NothingElse')('JSON') ], e => %args<llm-evaluator> // %args<e> // %args<conf> // $conf4o-mini, form => sub-parser('JSON'):drop ) } </pre></div> <p><strong>Remark:</strong> We reuse the sub definition  <code>rps-edge-dataset</code> from [AA1].</p> <p><strong>Remark::</strong> Both <a href="https://raku.land/zef:antononcube/LLM::Functions">“LLM::Functions”</a> and <a href="https://raku.land/zef:antononcube/LLM::Prompts" target="_blank" rel="noreferrer noopener">“LLM::Prompts”</a> are pre-loaded in <a href="https://raku.land/zef:antononcube/Jupyter::Chatbook" target="_blank" rel="noreferrer noopener">Raku chatbooks</a>.</p> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Rock-Paper-Scissors and its Lizard-Spock extensions</h2> <p>Here is the graph of the standard RPS game and it “Lizard-Spock” extension:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html # Graph edges: LLM-generated and LLM-translates my @edges-emo = { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'crushes' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'cuts' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'covers' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f98e.png" alt="🦎" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'crushes' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f98e.png" alt="🦎" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'poisons' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'smashes' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f98e.png" alt="🦎" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'decapitates' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f98e.png" alt="🦎" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'eats' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'disproves' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'vaporizes' } ; # Edge-label rules my %edge-labels-emo; @edges-emo.map({ %edge-labels-emo{$_<from>}{$_<to>} = $_<label> }); # RPS-3 Lizard-Spock extension my $g-emo = Graph.new(@edges-emo, :directed); # Standard RPS-3 as a subgraph my $g-rps = $g-emo.subgraph(<<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" />>); # Plot the graphs together $g-rps.dot(|%opts, edge-labels => %edge-labels-emo, :svg) ~ $g-emo.dot(|%opts, edge-labels => %edge-labels-emo, :svg) </pre></div> <figure class="wp-block-image size-large"><a href="https://raw.githubusercontent.com/antononcube/RakuForPrediction-blog/refs/heads/main/Articles/Diagrams/Military-forces-interactions-graphs/07-RPS-and-RPS-Lizard-Spock.png" target="_blank" rel=" noreferrer noopener"><img data-attachment-id="1415" data-permalink="https://rakuforprediction.wordpress.com/2025/03/06/military-forces-interactions-graphs/07-rps-and-rps-lizard-spock/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-and-rps-lizard-spock.png" data-orig-size="1165,603" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="07-RPS-and-RPS-Lizard-Spock" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-and-rps-lizard-spock.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-and-rps-lizard-spock.png?w=1008" width="1008" height="521" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-and-rps-lizard-spock.png?w=1008" alt="" class="wp-image-1415" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-and-rps-lizard-spock.png?w=1008 1008w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-and-rps-lizard-spock.png?w=150 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-and-rps-lizard-spock.png?w=300 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-and-rps-lizard-spock.png?w=768 768w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-and-rps-lizard-spock.png?w=1024 1024w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-and-rps-lizard-spock.png 1165w" sizes="(max-width: 1008px) 100vw, 1008px" /></a></figure> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Simple analogy</h2> <p>We consider the following military analogy with RPS:</p> <ul class="wp-block-list"> <li>Tanks attack (and defeat) Infantry</li> <li>Guerillas defend against Tanks</li> <li>Infantry attacks Guerillas</li> </ul> <p>Here we obtain the corresponding graph edges using an LLM:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $war-game = rps-edge-dataset('tanks attack infantry, guerillas defend against tanks, infantry attacks querillas') </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # [{from => Tanks, label => attack, to => Infantry} {from => Guerillas, label => defend, to => Tanks} {from => Infantry, label => attack, to => Guerillas}] </pre></div> <p>Plotting the graphs together:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html my %edge-labels = Empty; for |$war-game -> %r { %edge-labels{%r<from>}{%r<to>} = %r<label> }; Graph.new($war-game, :directed).dot(|%opts-plain, :%edge-labels, :svg) ~ $g-rps.dot(|%opts, edge-labels => %edge-labels-emo, :svg) </pre></div> <figure class="wp-block-image size-large"><a href="https://raw.githubusercontent.com/antononcube/RakuForPrediction-blog/refs/heads/main/Articles/Diagrams/Military-forces-interactions-graphs/08-RPS-and-Tanks-Infantry-Guerillas.png" target="_blank" rel=" noreferrer noopener"><img data-attachment-id="1416" data-permalink="https://rakuforprediction.wordpress.com/2025/03/06/military-forces-interactions-graphs/08-rps-and-tanks-infantry-guerillas/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/08-rps-and-tanks-infantry-guerillas.png" data-orig-size="1161,550" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="08-RPS-and-Tanks-Infantry-Guerillas" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/08-rps-and-tanks-infantry-guerillas.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/08-rps-and-tanks-infantry-guerillas.png?w=1008" width="1008" height="477" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/08-rps-and-tanks-infantry-guerillas.png?w=1008" alt="" class="wp-image-1416" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/08-rps-and-tanks-infantry-guerillas.png?w=1008 1008w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/08-rps-and-tanks-infantry-guerillas.png?w=150 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/08-rps-and-tanks-infantry-guerillas.png?w=300 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/08-rps-and-tanks-infantry-guerillas.png?w=768 768w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/08-rps-and-tanks-infantry-guerillas.png?w=1024 1024w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/08-rps-and-tanks-infantry-guerillas.png 1161w" sizes="(max-width: 1008px) 100vw, 1008px" /></a></figure> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Military forces interaction</h2> <p>Here is a <a href="https://mermaid.js.org/">Mermaid-JS</a>-made graph of a more complicated military forces interactions diagram; see [NM1]:</p> <figure class="wp-block-image size-full"><img data-attachment-id="1413" data-permalink="https://rakuforprediction.wordpress.com/2025/03/06/military-forces-interactions-graphs/06-military-forces-interaction/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/06-military-forces-interaction.png" data-orig-size="331,482" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="06-military-forces-interaction" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/06-military-forces-interaction.png?w=206" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/06-military-forces-interaction.png?w=331" width="331" height="482" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/06-military-forces-interaction.png" alt="" class="wp-image-1413" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/06-military-forces-interaction.png 331w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/06-military-forces-interaction.png?w=103&h=150 103w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/06-military-forces-interaction.png?w=206&h=300 206w" sizes="(max-width: 331px) 100vw, 331px" /></figure> <p>Using diagram’s Mermaid code here the graph edges are LLM-generated:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html my $mmd-descr = q:to/END/; graph TD AT[Anti-tank weapons] --> |defend|Arm[Armor] Arm --> |attack|IA[Infantry and Artillery] Air[Air force] --> |attack|Arm Air --> |attack|IA M[Missiles] --> |defend|Air IA --> |attack|M IA --> |attack|AT END my $war-game2 = rps-edge-dataset($mmd-descr); $war-game2 ==> to-html(field-names => <from label to>) </pre></div> <p>Direct assignment (instead of using LLMs):</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $war-game2 = $[ {:from("Anti-tank weapons"), :label("defend"), :to("Armor")}, {:from("Armor"), :label("attack"), :to("Infantry and Artillery")}, {:from("Air force"), :label("attack"), :to("Armor")}, {:from("Air force"), :label("attack"), :to("Infantry and Artillery")}, {:from("Missiles"), :label("defend"), :to("Air force")}, {:from("Infantry and Artillery"), :label("attack"), :to("Missiles")}, {:from("Infantry and Artillery"), :label("attack"), :to("Anti-tank weapons")} ]; </pre></div> <p>The diagram does not correspond to modern warfare — it is taken from a doctoral thesis, [NM1], discussing reconstruction of historical military data. The corresponding graph can be upgraded with drones in a similar way as the Chuck-Norris-defeats-all upgrade in [<a href="https://rakuforprediction.wordpress.com/2025/03/03/rock-paper-scissors-extensions/" target="_blank" rel="noreferrer noopener">AA1</a>].</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $war-forces = Graph.new($war-game2, :directed); my $drone = "Air drones"; my $war-game-d = $war-game2.clone.append( $war-forces.vertex-list.map({ %( from => $drone, to => $_, label => 'attack' ) }) ); $war-game-d .= append( ['Missiles', 'Air force'].map({ %(from => $_, to => $drone, label => 'defend') }) ); my $war-forces-d = Graph.new($war-game-d, :directed); </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Graph(vertexes => 6, edges => 14, directed => True) </pre></div> <p>Here is the corresponding table:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html game-table($war-forces-d, link-value => '⊙', missing-value => '') </pre></div> <figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th></th><th>Air drones</th><th>Air force</th><th>Anti-tank weapons</th><th>Armor</th><th>Infantry and Artillery</th><th>Missiles</th></tr></thead><tbody><tr><td>Air drones</td><td></td><td>⊙</td><td>⊙</td><td>⊙</td><td>⊙</td><td>⊙</td></tr><tr><td>Air force</td><td>⊙</td><td></td><td></td><td>⊙</td><td>⊙</td><td></td></tr><tr><td>Anti-tank weapons</td><td></td><td></td><td></td><td>⊙</td><td></td><td></td></tr><tr><td>Armor</td><td></td><td></td><td></td><td></td><td>⊙</td><td></td></tr><tr><td>Infantry and Artillery</td><td></td><td></td><td>⊙</td><td></td><td></td><td>⊙</td></tr><tr><td>Missiles</td><td>⊙</td><td>⊙</td><td></td><td></td><td></td><td></td></tr></tbody></table></figure> <p>Here is the graph with different coloring for “attack” edges (gray) and “defend” edges (blue):</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html $war-forces-d.vertex-coordinates = ($war-forces-d.vertex-list Z=> Graph::Cycle($war-forces-d.vertex-count).vertex-coordinates{^$war-forces-d.vertex-count}.values).Hash; my %edge-labels; $war-game-d.map({ %edge-labels{$_<from>}{$_<to>} = $_<label> }); my %highlight = 'SlateBlue' => Graph.new( $war-game-d.grep(*<label> eq 'defend'), :directed).edges; $war-forces-d.dot( :%highlight, |merge-hash(%opts-plain, {:9graph-size, node-width => 0.7}), :%edge-labels, :svg ) </pre></div> <figure class="wp-block-image size-full"><a href="https://raw.githubusercontent.com/antononcube/RakuForPrediction-blog/refs/heads/main/Articles/Diagrams/Military-forces-interactions-graphs/09-Drones-beat-all.png" target="_blank" rel=" noreferrer noopener"><img data-attachment-id="1417" data-permalink="https://rakuforprediction.wordpress.com/2025/03/06/military-forces-interactions-graphs/09-drones-beat-all/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/09-drones-beat-all.png" data-orig-size="898,652" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="09-Drones-beat-all" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/09-drones-beat-all.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/09-drones-beat-all.png?w=898" loading="lazy" width="898" height="652" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/09-drones-beat-all.png" alt="" class="wp-image-1417" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/09-drones-beat-all.png 898w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/09-drones-beat-all.png?w=150&h=109 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/09-drones-beat-all.png?w=300&h=218 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/09-drones-beat-all.png?w=768&h=558 768w" sizes="(max-width: 898px) 100vw, 898px" /></a></figure> <p><strong>Remark:</strong> The graph above is just an example — real-life military forces interactions are more complicated.</p> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Generalized antagonism</h2> <p>Following the article <a href="https://www.researchgate.net/publication/356424971_Obobsennaa_model_Lancestera_formalizuusaa_konflikt_neskolkih_storon_The_General_Lanchester_Model_Defining_Multilateral_Conflicts">“The General Lanchester Model Defining Multilateral Conflicts”</a>, [SM1], we can make a graph for multiple, simultaneous conflicts (narrated exposition is given in the presentation <a href="https://www.youtube.com/watch?v=852vMS_6Qaw">“Upgrading Epidemiological Models into War Models”</a>, [AAv1]):</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html # Graph edges my @multi-conflict-edges = %(from=>1, to=>5, label=>'Neutrality', :!directed), %(from=>1, to=>3, label=>'Commensalism', :directed), %(from=>1, to=>4, label=>'Commensalism', :directed), %(from=>2, to=>1, label=>'Coercion', :directed), %(from=>2, to=>3, label=>'Alliance', :!directed), %(from=>2, to=>4, label=>'Guerilla war', :directed), %(from=>3, to=>4, label=>'Conflict', :!directed), %(from=>5, to=>3, label=>'Avoidance', :directed), %(from=>5, to=>4, label=>'Alliance', :!directed), %(from=>5, to=>2, label=>'Adaptation', :directed); @multi-conflict-edges .= deepmap({ $_ ~~ Bool:D ?? $_ !! $_.Str }); # Edg-label rules my %edge-labels; @multi-conflict-edges.map({ %edge-labels{$_<from>}{$_<to>} = $_<label> }); # Make an empty graph my $mc = Graph.new; # Add edge depending of its direction specification my @dir-edges; for @multi-conflict-edges -> %e { $mc.edge-add(%e<from>, %e<to>, :directed); if !%e<directed> { $mc.edge-add(%e<to>, %e<from>, :directed) } } # Vertex coordinates via Cycle graph $mc.vertex-coordinates = ($mc.vertex-list Z=> Graph::Cycle($mc.vertex-count).vertex-coordinates{^$mc.vertex-count}.values).Hash; # Graph plot $mc.dot(|merge-hash(%opts, {node-shape => 'square', :4edge-font-size }), :%edge-labels, highlight => { RosyBrown => <1 3 4>, SlateBlue => <2 5> }, :mixed, :svg) </pre></div> <figure class="wp-block-image size-full"><a href="https://raw.githubusercontent.com/antononcube/RakuForPrediction-blog/refs/heads/main/Articles/Diagrams/Military-forces-interactions-graphs/10-Multi-conflict.png" target="_blank" rel=" noreferrer noopener"><img data-attachment-id="1418" data-permalink="https://rakuforprediction.wordpress.com/2025/03/06/military-forces-interactions-graphs/10-multi-conflict/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/10-multi-conflict.png" data-orig-size="604,596" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="10-Multi-conflict" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/10-multi-conflict.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/10-multi-conflict.png?w=604" loading="lazy" width="604" height="596" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/10-multi-conflict.png" alt="" class="wp-image-1418" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/10-multi-conflict.png 604w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/10-multi-conflict.png?w=150&h=148 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/10-multi-conflict.png?w=300&h=296 300w" sizes="(max-width: 604px) 100vw, 604px" /></a></figure> <p><strong>Remark:</strong> The graph above is just for illustration. In order to do mathematical modeling additional interaction data is required; see [AAv1].</p> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>References</h2> <h3 class="wp-block-heading"><a></a>Articles, books, these</h3> <p>[AA1] Anton Antonov, <a href="https://rakuforprediction.wordpress.com/2025/03/03/rock-paper-scissors-extensions/">“Rock-Paper-Scissors extensions”</a>, (2025), <a href="https://rakuforprediction.wordpress.com/">RakuForPrediction at WordPress</a>.</p> <p>[AJ1] Archer Jones, <a href="https://www.press.uillinois.edu/books/?id=p069666">“The Art of War in Western World”</a>, (2000), <a href="https://www.press.uillinois.edu/">University of Illinois Press</a>. 768 pages, ISBN-10: 0252069668, ISBN-13: 978-0252069666.</p> <p>[SM1] Sergei Makarenko et al., <a href="https://www.researchgate.net/publication/356424971_Obobsennaa_model_Lancestera_formalizuusaa_konflikt_neskolkih_storon_The_General_Lanchester_Model_Defining_Multilateral_Conflicts">“Обобщенная модель Ланчестера, формализующая конфликт нескольких сторон”</a>, [Eng. “The General Lanchester Model Defining Multilateral Conflicts”], (2021), Automation of Control Processes № 2 (64), doi: 10.35752/1991-2927-2021-2-64-66-76.</p> <p>[NM1] Николай В. Митюков, <a href="https://www.dissercat.com/content/matematicheskie-modeli-i-programmnye-sredstva-dlya-rekonstruktsii-voenno-istoricheskikh-dann">“Математические модели и программные средства для реконструкции военно-исторических данных”</a>, (2009), <a href="https://www.dissercat.com/">disserCat</a>.</p> <h3 class="wp-block-heading"><a></a>Packages</h3> <p>[AAp1] Anton Antonov, <a href="https://github.com/antononcube/Raku-Graph">Graph Raku package</a>, (2024-2025), <a href="https://github.com/antononcube">GitHub/antononcube</a>.</p> <p>[AAp2] Anton Antonov, <a href="https://github.com/antononcube/Raku-LLM-Functions">LLM::Functions Raku package</a>, (2023-2024), <a href="https://github.com/antononcube">GitHub/antononcube</a>.</p> <p>[AAp3] Anton Antonov, <a href="https://github.com/antononcube/Raku-LLM-Prompts">LLM::Prompts Raku package</a>, (2023-2024), <a href="https://github.com/antononcube">GitHub/antononcube</a>.</p> <p>[AAp4] Anton Antonov, <a href="https://github.com/antononcube/Raku-Jupyter-Chatbook">Jupyter::Chatbook Raku package</a>, (2023-2024), <a href="https://github.com/antononcube">GitHub/antononcube</a>.</p> <p>[EMp1] Elizabeth Mattijsen, <a href="https://github.com/lizmat/Text-Emoji">Text::Emoji Raku package</a>, (2024-2025), <a href="https://github.com/lizmat">GitHub/lizmat</a>.</p> <h3 class="wp-block-heading"><a></a>Videos</h3> <p>[AAv1] Anton Antonov, <a href="https://www.youtube.com/watch?v=852vMS_6Qaw">“Upgrading Epidemiological Models into War Models”</a>, (2024), <a href="https://www.youtube.com/@WolframResearch">YouTube/@WolframResearch</a>.</p> <p>[Wv1] Wozamil, <a href="https://www.youtube.com/watch?v=x5Q6-wMx-K8">“Rock Paper Scissors Lizard Spock (Extended Cut) ~ The Big Bang Theory ~”</a>, (2012), <a href="https://www.youtube.com/@Wozamil">YouTube@Wozamil</a>.</p> <h2><a href="https://rakuforprediction.wordpress.com/2025/03/03/rock-paper-scissors-extensions/">Rock-Paper-Scissors extensions</h2></a> <p><h3>Published by Anton Antonov Antonov on 2025-03-03T21:09:20</h3></p> <h2 class="wp-block-heading"><a></a>Introduction</h2> <p>It is easy to make a simple <a href="https://en.wikipedia.org/wiki/Rock_paper_scissors">Rock-Paper-Scissors</a> (RPS) game graph using the Raku package <a href="https://raku.land/zef:antononcube/Graph">“Graph”</a>, [AAp1]. Here is such a graph in which the arrow directions indicate which item (vertex) wins:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #%html my $g0 = Graph.new(<<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" />>.Hash):d; $g0.dot(:3graph-size, engine => 'neato'):svg </pre></div> <figure class="wp-block-image size-full"><img data-attachment-id="1386" data-permalink="https://rakuforprediction.wordpress.com/2025/03/03/rock-paper-scissors-extensions/01-rps-quick/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/01-rps-quick.png" data-orig-size="434,398" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="01-RPS-quick" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/01-rps-quick.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/01-rps-quick.png?w=434" loading="lazy" width="434" height="398" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/01-rps-quick.png" alt="" class="wp-image-1386" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/01-rps-quick.png 434w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/01-rps-quick.png?w=150&h=138 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/01-rps-quick.png?w=300&h=275 300w" sizes="(max-width: 434px) 100vw, 434px" /></figure> <p>Easy, but now we want to:</p> <ul class="wp-block-list"> <li>Add labels to the edges <ul class="wp-block-list"> <li>Like “<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" /> crushes <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" />”</li> </ul> </li> <li>Change the items/weapons <ul class="wp-block-list"> <li><a href="https://en.wikipedia.org/wiki/Sansukumi-ken">Snake-Centipede-Snail</a></li> </ul> </li> <li>Make graphs for game <em>upgrades</em> <ul class="wp-block-list"> <li><a href="https://www.youtube.com/watch?v=x5Q6-wMx-K8">Rock-Paper-Scissors-Lizard-Spock</a></li> <li><a href="https://en.wikipedia.org/wiki/Rock_paper_scissors#Additional_weapons">Rock-Paper-Scissors-Fire-Water</a></li> </ul> </li> <li>Utilize Large Language Models (LLMs) <ul class="wp-block-list"> <li>In order to simplify the graph making</li> </ul> </li> </ul> <p>In this post (<a href="https://github.com/antononcube/RakuForPrediction-blog/blob/main/Notebooks/Jupyter/Rock-Paper-Scissors-extensions.ipynb" target="_blank" rel="noreferrer noopener">notebook</a>) we show how to do all of the above points.</p> <p><strong>Remark:</strong> Interesting analogies of the presented graphs can be made with warfare graphs, [AAv1]. For example, the graph tanks-infantry-guerillas is analogous to RPS.</p> <h3 class="wp-block-heading"><a></a>TL;DR</h3> <ul class="wp-block-list"> <li>LLMs “know” the RPS game and its upgrades.</li> <li>LLMs know how to (mostly, reliably) translate to emojis.</li> <li>The package “Graph” (via <a href="https://graphviz.org/doc/info/lang.html">Graphviz DOT</a>) can produce SVG plots that are readily rendered in different environments. <ul class="wp-block-list"> <li>And the graphs of hand-games like RPS look good.</li> </ul> </li> <li>The class <code>Graph</code> have handy methods and attributes that make the creation and modification of graphs smooth(er).</li> </ul> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Setup</h2> <p>This notebook is a Raku-chatbook, hence, its Jupyter session preloads certain packages and LLM-personas.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Preloaded in any chatbook # use LLM::Functions; # use LLM::Prompts; # Preloaded in a user init file # use Graph; # For this concrete session use Text::Emoji; </pre></div> <p>LLM configurations:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $conf4o = llm-configuration('chat-gpt', model => 'gpt-4o', :4096max-tokens, temperature => 0.4); my $conf4o-mini = llm-configuration('chat-gpt', model => 'gpt-4o-mini', :4096max-tokens, temperature => 0.4); ($conf4o, $conf4o-mini)».Hash».elems </pre></div> <p>Default options of Graph.dot:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $background = '#1F1F1F'; my $engine = 'neato'; my %opts = :$background, :6graph-size, :1edge-width, :3edge-font-size, edge-color => 'LightSlateGray', node-width => 0.2, node-height => 0.2, node-shape => 'circle', :node-labels, :8node-font-size, node-fill-color => '#1F1F1F', node-color => 'LightSlateGray', node-stroke-width => 0.6, arrow-size => 0.25, :$engine; my %opts-plain = merge-hash(%opts, {:5node-font-size, node-shape => 'ellipse', node-width => 0.27, node-height => 0.15}); (%opts, %opts-plain)».elems </pre></div> <h3 class="wp-block-heading"><a></a>Additional</h3> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> sub game-table(Graph:D $g, Str:D :$link-value = '+', Str:D :$missing-value = '-') { cross-tabulate($g.edges(:dataset), <from>, <to>) ==> -> %h { %h.map({ $_.key => ($g.vertex-list Z=> $_.value{$g.vertex-list}).Hash }).Hash }() ==> to-dataset(:$missing-value) ==> -> %h { for $g.vertex-list { %h{$_}{$_} = ''}; %h }() ==> -> %h { $g.vertex-list.map({ [|%h{$_}, "" => $_].Hash }) }() ==> to-html(field-names => ["", |$g.vertex-list]) ==> { .Str.subst('1', $link-value, :g).subst('(Any)', $missing-value, :g) }() } </pre></div> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>LLM request</h2> <p>Raku-chatbooks, [AAp4], can have initialization Raku code and specified preloaded LLM-personas. One such LLM-persona is <a href="https://github.com/antononcube/Raku-Jupyter-Chatbook/blob/master/resources/llm-personas.json">“raku”</a>. Here we use the “raku” chat object to get Raku code for the edges of the RPS extension <a href="https://bigbangtheory.fandom.com/wiki/Sam_Kass">Rock-Paper-Scissors-Lizard-Spock</a>, [Wv1].</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% chat raku Make an array the edges of a graph for the game Rock-Paper-Scissors-Lizard-Spock. Each edges is represented with a hash with the keys "from", "to", "label". The label corresponds to the action taken with the edge, like, "Paper covers Rock", "Paper disproves Spock". </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my @edges = ( { from => 'Rock', to => 'Scissors', label => 'Rock crushes Scissors' }, { from => 'Rock', to => 'Lizard', label => 'Rock crushes Lizard' }, { from => 'Paper', to => 'Rock', label => 'Paper covers Rock' }, { from => 'Paper', to => 'Spock', label => 'Paper disproves Spock' }, { from => 'Scissors', to => 'Paper', label => 'Scissors cuts Paper' }, { from => 'Scissors', to => 'Lizard', label => 'Scissors decapitates Lizard' }, { from => 'Lizard', to => 'Spock', label => 'Lizard poisons Spock' }, { from => 'Lizard', to => 'Paper', label => 'Lizard eats Paper' }, { from => 'Spock', to => 'Scissors', label => 'Spock smashes Scissors' }, { from => 'Spock', to => 'Rock', label => 'Spock vaporizes Rock' }, ); </pre></div> <p>We use the generated code in the next section.</p> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Plain text graph</h2> <p>Here we create the Rock-Paper-Scissors-Lizard-Spock graph generated with the LLM-magic cell above:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my @edges = { from => 'Rock', to => 'Scissors', label => 'Rock crushes Scissors' }, { from => 'Scissors', to => 'Paper', label => 'Scissors cuts Paper' }, { from => 'Paper', to => 'Rock', label => 'Paper covers Rock' }, { from => 'Rock', to => 'Lizard', label => 'Rock crushes Lizard' }, { from => 'Lizard', to => 'Spock', label => 'Lizard poisons Spock' }, { from => 'Spock', to => 'Scissors', label => 'Spock smashes Scissors' }, { from => 'Scissors', to => 'Lizard', label => 'Scissors decapitates Lizard' }, { from => 'Lizard', to => 'Paper', label => 'Lizard eats Paper' }, { from => 'Paper', to => 'Spock', label => 'Paper disproves Spock' }, { from => 'Spock', to => 'Rock', label => 'Spock vaporizes Rock' } ; my $g = Graph.new(@edges, :directed); </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Graph(vertexes => 5, edges => 10, directed => True) </pre></div> <p>Here we make the edge labels:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my %edge-labels; @edges.map({ %edge-labels{$_<from>}{$_<to>} = $_<label>.words[1] }); deduce-type(%edge-labels) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Assoc(Atom((Str)), Assoc(Atom((Str)), Atom((Str)), 2), 5) </pre></div> <p>Here we plot the graph:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html $g.dot(|%opts-plain, :%edge-labels):svg </pre></div> <figure class="wp-block-image size-full"><img data-attachment-id="1387" data-permalink="https://rakuforprediction.wordpress.com/2025/03/03/rock-paper-scissors-extensions/02-rps-lizard-spock/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/02-rps-lizard-spock.png" data-orig-size="855,775" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="02-RPS-Lizard-Spock" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/02-rps-lizard-spock.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/02-rps-lizard-spock.png?w=855" loading="lazy" width="855" height="775" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/02-rps-lizard-spock.png" alt="" class="wp-image-1387" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/02-rps-lizard-spock.png 855w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/02-rps-lizard-spock.png?w=150&h=136 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/02-rps-lizard-spock.png?w=300&h=272 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/02-rps-lizard-spock.png?w=768&h=696 768w" sizes="(max-width: 855px) 100vw, 855px" /></figure> <p><strong>Remark:</strong> Currently the class <code>Graph</code> does not “deal” with edge labels, but some of its methods (like, <code>dot</code>) do.</p> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Convenient LLM functions</h2> <h3 class="wp-block-heading"><a></a>Graph edges</h3> <p>Instead of using chat-cells, we can define an LLM function that provides the graph edges dataset for different RPS variants. Here is such an LLM function using “LLM::Functions”, [AAp1], and “LLM::Prompts”, [AAv2]:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my sub rps-edge-dataset($description, Str:D $game-name = 'Rock-Paper-Scissors', *%args) { llm-synthesize([ "Give the edges the graph for this $game-name variant description", 'Give the edges as an array of dictionaries. Each dictionary with keys "from", "to", "label",', 'where "label" has the action of "from" over "to".', $description, llm-prompt('NothingElse')('JSON') ], e => %args<llm-evaluator> // %args<e> // %args<conf> // $conf4o-mini, form => sub-parser('JSON'):drop ) } </pre></div> <p><strong>Remark::</strong> Both “LLM::Functions” and “LLM::Prompts” are pre-loaded in Raku chatbooks.</p> <h3 class="wp-block-heading"><a></a>Emoji translations</h3> <p>We can translate to emojis the plain-text vertex labels of RPS graphs in several ways:</p> <ol class="wp-block-list"> <li>Manually</li> <li>Using <code>to-emoji</code> of <a href="https://raku.land/zef:lizmat/Text::Emoji">“Text::Emoji”</a>, [EMp1]</li> <li>Via LLMs</li> </ol> <p>Here we take option 2:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my %additional = spock => to-emoji(':vulcan-salute:'), paper => to-emoji(":page-with-curl:"); say (:%additional); @edges.map(*<from>).map({ $_ => to-emoji(":$_:", %additional) }) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # additional => {paper => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c3.png" alt="📃" class="wp-smiley" style="height: 1em; max-height: 1em;" />, spock => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" />} # (Rock => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" /> Scissors => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" /> Paper => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c3.png" alt="📃" class="wp-smiley" style="height: 1em; max-height: 1em;" /> Rock => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" /> Lizard => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f98e.png" alt="🦎" class="wp-smiley" style="height: 1em; max-height: 1em;" /> Spock => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" /> Scissors => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" /> Lizard => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f98e.png" alt="🦎" class="wp-smiley" style="height: 1em; max-height: 1em;" /> Paper => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c3.png" alt="📃" class="wp-smiley" style="height: 1em; max-height: 1em;" /> Spock => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" />) </pre></div> <p>Again, let us define an LLM function for that does emojification. (I.e. for option 3.)</p> <p>One way is to do a simple application of the prompt “Emojify” and process its result into a dictionary:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $res = llm-synthesize( llm-prompt("Emojify")($g.vertex-list), e => $conf4o-mini ); $res.split(/\s+/, :skip-empty)».trim.Hash </pre></div> <p>It is better to have a function that provides a more “immediate” result:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my sub emoji-rules($words, *%args) { llm-synthesize( [ llm-prompt("Emojify")($words), 'Make a JSON dictionary of the original words as keys and the emojis as values', llm-prompt('NothingElse')('JSON') ], e => %args<llm-evaluator> // %args<e> // %args<conf> // $conf4o-mini, form => sub-parser('JSON'):drop ) } </pre></div> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Emoji graph</h2> <p>Let us remake the game graph using suitable emojis. Here are the corresponding egdes:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my @edges-emo = { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'crushes' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'cuts' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'covers' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f98e.png" alt="🦎" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'crushes' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f98e.png" alt="🦎" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'poisons' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'smashes' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f98e.png" alt="🦎" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'decapitates' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f98e.png" alt="🦎" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'eats' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'disproves' }, { from => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" />', to => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" />', label => 'vaporizes' } ; my $g-emo = Graph.new(@edges-emo, :directed); </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Graph(vertexes => 5, edges => 10, directed => True) </pre></div> <p>Here is a table of the upgraded game that shows the interaction between the different roles (hand plays):</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html game-table($g-emo) </pre></div> <figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f98e.png" alt="🦎" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th></tr></thead><tbody><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td></td><td>+</td><td>–</td><td>+</td><td>–</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>–</td><td></td><td>+</td><td>–</td><td>+</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>+</td><td>–</td><td></td><td>–</td><td>+</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f98e.png" alt="🦎" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>–</td><td>+</td><td>+</td><td></td><td>–</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>+</td><td>–</td><td>–</td><td>+</td><td></td></tr></tbody></table></figure> <p>Here we make the edge labels:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my %edge-labels-emo; @edges-emo.map({ %edge-labels-emo{$_<from>}{$_<to>} = $_<label> }); deduce-type(%edge-labels-emo) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Assoc(Atom((Str)), Assoc(Atom((Str)), Atom((Str)), 2), 5) </pre></div> <p>Here we plot the graph (using a variety of setup options):</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html $g-emo.dot(|%opts, edge-labels => %edge-labels-emo):svg </pre></div> <figure class="wp-block-image size-full"><img data-attachment-id="1388" data-permalink="https://rakuforprediction.wordpress.com/2025/03/03/rock-paper-scissors-extensions/03-rps-lizard-spock/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/03-rps-lizard-spock.png" data-orig-size="872,837" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="03-RPS-Lizard-Spock" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/03-rps-lizard-spock.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/03-rps-lizard-spock.png?w=872" loading="lazy" width="872" height="837" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/03-rps-lizard-spock.png" alt="" class="wp-image-1388" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/03-rps-lizard-spock.png 872w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/03-rps-lizard-spock.png?w=150&h=144 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/03-rps-lizard-spock.png?w=300&h=288 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/03-rps-lizard-spock.png?w=768&h=737 768w" sizes="(max-width: 872px) 100vw, 872px" /></figure> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Chuck Norris defeats them all!</h2> <p>Consider the <a href="https://www.merchandisingplaza.us/40488/2/T-shirts-Chuck-Norris-Chuck-Norris-Rock-Paper-Scissors-Lizard-Spock-TShirt-l.jpg">image</a> (from <a href="https://www.merchandisingplaza.us/Chuck-Norris/Chuck-Norris-Rock-Paper-Scissors-Lizard-Spock-TShirt-40488">www.merchandisingplaza.us</a>):</p> <figure class="wp-block-image"><img data-attachment-id="1399" data-permalink="https://rakuforprediction.wordpress.com/2025/03/03/rock-paper-scissors-extensions/t-shirts-chuck-norris-chuck-norris-rock-paper-scissors-lizard-spock-tshirt-l/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/t-shirts-chuck-norris-chuck-norris-rock-paper-scissors-lizard-spock-tshirt-l.jpg" data-orig-size="500,516" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="T-shirts-Chuck-Norris-Chuck-Norris-Rock-Paper-Scissors-Lizard-Spock-TShirt-l" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/t-shirts-chuck-norris-chuck-norris-rock-paper-scissors-lizard-spock-tshirt-l.jpg?w=291" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/t-shirts-chuck-norris-chuck-norris-rock-paper-scissors-lizard-spock-tshirt-l.jpg?w=500" loading="lazy" width="500" height="516" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/t-shirts-chuck-norris-chuck-norris-rock-paper-scissors-lizard-spock-tshirt-l.jpg" alt="" class="wp-image-1399" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/t-shirts-chuck-norris-chuck-norris-rock-paper-scissors-lizard-spock-tshirt-l.jpg 500w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/t-shirts-chuck-norris-chuck-norris-rock-paper-scissors-lizard-spock-tshirt-l.jpg?w=145&h=150 145w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/t-shirts-chuck-norris-chuck-norris-rock-paper-scissors-lizard-spock-tshirt-l.jpg?w=291&h=300 291w" sizes="(max-width: 500px) 100vw, 500px" /></figure> <p>Let us try to remake it with a graph plot. At this point we simply add a “foot connection” to all five vertices in the graph(s) above:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $chuck = "<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f9b6-1f3fb.png" alt="🦶🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" />"; my $g-chuck = $g.clone.edge-add( ($chuck X=> $g.vertex-list).Array, :directed); </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Graph(vertexes => 6, edges => 15, directed => True) </pre></div> <p>But we also have to rename the vertices to be hand-gestures:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> $g-chuck .= vertex-replace( { Scissors => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270c-1f3fb.png" alt="✌🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" />', Rock => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270a-1f3fb.png" alt="✊🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" />', Lizard => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f90f-1f3fb.png" alt="🤏🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" />', Spock => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596-1f3fb.png" alt="🖖🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" />', 'Paper' => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270b-1f3fb.png" alt="✋🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" />' } ) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Graph(vertexes => 6, edges => 15, directed => True) </pre></div> <p>Here is the interactions table of the upgraded game:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html game-table($g-chuck) </pre></div> <figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270a-1f3fb.png" alt="✊🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270b-1f3fb.png" alt="✋🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270c-1f3fb.png" alt="✌🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596-1f3fb.png" alt="🖖🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f90f-1f3fb.png" alt="🤏🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f9b6-1f3fb.png" alt="🦶🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th></tr></thead><tbody><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270a-1f3fb.png" alt="✊🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td></td><td>–</td><td>+</td><td>–</td><td>+</td><td>–</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270b-1f3fb.png" alt="✋🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>+</td><td></td><td>–</td><td>+</td><td>–</td><td>–</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270c-1f3fb.png" alt="✌🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>–</td><td>+</td><td></td><td>–</td><td>+</td><td>–</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596-1f3fb.png" alt="🖖🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>+</td><td>–</td><td>+</td><td></td><td>–</td><td>–</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f90f-1f3fb.png" alt="🤏🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>–</td><td>+</td><td>–</td><td>+</td><td></td><td>–</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f9b6-1f3fb.png" alt="🦶🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td><td></td></tr></tbody></table></figure> <p>In order to ensure that we get an “expected” graph plot, we can take the vertex coordinates of a <a href="https://en.wikipedia.org/wiki/Wheel_graph">wheel graph</a> or compute them by hand. Here we do the latter:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my @vs = <<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270a-1f3fb.png" alt="✊🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596-1f3fb.png" alt="🖖🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f90f-1f3fb.png" alt="🤏🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270c-1f3fb.png" alt="✌🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270b-1f3fb.png" alt="✋🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" />>; my %vertex-coordinates = @vs.kv.map( -> $i, $v { $v => [cos(π/2 + $i * 2 * π / 5), sin(π/2 + $i * 2 * π / 5)] }); %vertex-coordinates<<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f9b6-1f3fb.png" alt="🦶🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" />> = [0, 0]; $g-chuck.vertex-coordinates = %vertex-coordinates; deduce-type(%vertex-coordinates) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Struct([<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270a-1f3fb.png" alt="✊🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" />, <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270b-1f3fb.png" alt="✋🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" />, <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270c-1f3fb.png" alt="✌🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" />, <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596-1f3fb.png" alt="🖖🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" />, <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f90f-1f3fb.png" alt="🤏🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" />, <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f9b6-1f3fb.png" alt="🦶🏻" class="wp-smiley" style="height: 1em; max-height: 1em;" />], [Array, Array, Array, Array, Array, Array]) </pre></div> <p>Here we plot the graph:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html $g-chuck.dot( background => '#5f5b4f', graph-label => 'Chuck Norris Defeats All'.uc, font-color => '#b8aa79', :6graph-size, :2edge-width, :4edge-font-size, edge-color => 'AntiqueWhite', node-width => 0.56, node-height => 0.56, node-shape => 'circle', :node-labels, :38node-font-size, node-fill-color => '#b8aa79', node-color => 'Gray', node-stroke-width => 0.6, arrow-size => 0.26, engine => 'neato', :svg ) </pre></div> <figure class="wp-block-image size-full"><img data-attachment-id="1389" data-permalink="https://rakuforprediction.wordpress.com/2025/03/03/rock-paper-scissors-extensions/04-rps-lizard-spock-chucknorris/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/04-rps-lizard-spock-chucknorris.png" data-orig-size="825,684" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="04-RPS-Lizard-Spock-ChuckNorris" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/04-rps-lizard-spock-chucknorris.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/04-rps-lizard-spock-chucknorris.png?w=825" loading="lazy" width="825" height="684" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/04-rps-lizard-spock-chucknorris.png" alt="" class="wp-image-1389" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/04-rps-lizard-spock-chucknorris.png 825w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/04-rps-lizard-spock-chucknorris.png?w=150&h=124 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/04-rps-lizard-spock-chucknorris.png?w=300&h=249 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/04-rps-lizard-spock-chucknorris.png?w=768&h=637 768w" sizes="(max-width: 825px) 100vw, 825px" /></figure> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Using LLMs</h2> <h3 class="wp-block-heading"><a></a>Matching the colors</h3> <p>We can use “LLM vision” to get the colors of the original image:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $url = 'https://www.merchandisingplaza.us/40488/2/T-shirts-Chuck-Norris-Chuck-Norris-Rock-Paper-Scissors-Lizard-Spock-TShirt-l.jpg'; llm-vision-synthesize('What are the dominant colors in this image? Give them in hex code.', $url) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> The dominant colors in the image are: - Olive Green: #5B5D4A - Beige: #D0C28A - White: #FFFFFF - Black: #000000 </pre></div> <h3 class="wp-block-heading"><a></a>Graph generating with LLMs</h3> <p>Instead of specifying the graph edges by hand, we can use LLM-vision and suitable prompting. The results are not that good, but YMMV.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $res2 = llm-vision-synthesize([ 'Give the edges the graph for this image of Rock-Paper-Scissors-Lizard-Spock-Chuck -- use relevant emojis.', 'Give the edges as an array of dictionaries. Each dictionary with keys "from" and "to".', llm-prompt('NothingElse')('JSON') ], $url, e => $conf4o, form => sub-parser('JSON'):drop ) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # [{from => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270b.png" alt="✋" class="wp-smiley" style="height: 1em; max-height: 1em;" />, to => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270c.png" alt="✌" class="wp-smiley" style="height: 1em; max-height: 1em;" />} {from => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270c.png" alt="✌" class="wp-smiley" style="height: 1em; max-height: 1em;" />, to => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270a.png" alt="✊" class="wp-smiley" style="height: 1em; max-height: 1em;" />} {from => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270a.png" alt="✊" class="wp-smiley" style="height: 1em; max-height: 1em;" />, to => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f98e.png" alt="🦎" class="wp-smiley" style="height: 1em; max-height: 1em;" />} {from => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f98e.png" alt="🦎" class="wp-smiley" style="height: 1em; max-height: 1em;" />, to => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" />} {from => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" />, to => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270b.png" alt="✋" class="wp-smiley" style="height: 1em; max-height: 1em;" />} {from => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270b.png" alt="✋" class="wp-smiley" style="height: 1em; max-height: 1em;" />, to => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270a.png" alt="✊" class="wp-smiley" style="height: 1em; max-height: 1em;" />} {from => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270a.png" alt="✊" class="wp-smiley" style="height: 1em; max-height: 1em;" />, to => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270b.png" alt="✋" class="wp-smiley" style="height: 1em; max-height: 1em;" />} {from => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270c.png" alt="✌" class="wp-smiley" style="height: 1em; max-height: 1em;" />, to => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f98e.png" alt="🦎" class="wp-smiley" style="height: 1em; max-height: 1em;" />} {from => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f98e.png" alt="🦎" class="wp-smiley" style="height: 1em; max-height: 1em;" />, to => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270b.png" alt="✋" class="wp-smiley" style="height: 1em; max-height: 1em;" />} {from => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" />, to => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270c.png" alt="✌" class="wp-smiley" style="height: 1em; max-height: 1em;" />} {from => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270c.png" alt="✌" class="wp-smiley" style="height: 1em; max-height: 1em;" />, to => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" />} {from => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f596.png" alt="🖖" class="wp-smiley" style="height: 1em; max-height: 1em;" />, to => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/270a.png" alt="✊" class="wp-smiley" style="height: 1em; max-height: 1em;" />}] </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html Graph.new($res2, :directed).dot(:5graph-size, engine => 'neato', arrow-size => 0.5):svg </pre></div> <figure class="wp-block-image size-full"><img data-attachment-id="1390" data-permalink="https://rakuforprediction.wordpress.com/2025/03/03/rock-paper-scissors-extensions/05-rps-lizard-spock-chucknorris-llm-generated/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/05-rps-lizard-spock-chucknorris-llm-generated.png" data-orig-size="658,710" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="05-RPS-Lizard-Spock-ChuckNorris-LLM-generated" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/05-rps-lizard-spock-chucknorris-llm-generated.png?w=278" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/05-rps-lizard-spock-chucknorris-llm-generated.png?w=658" loading="lazy" width="658" height="710" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/05-rps-lizard-spock-chucknorris-llm-generated.png" alt="" class="wp-image-1390" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/05-rps-lizard-spock-chucknorris-llm-generated.png 658w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/05-rps-lizard-spock-chucknorris-llm-generated.png?w=139&h=150 139w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/05-rps-lizard-spock-chucknorris-llm-generated.png?w=278&h=300 278w" sizes="(max-width: 658px) 100vw, 658px" /></figure> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Rock-Paper-Scissors-Fire-Water</h2> <p>One notable variant is Rock-Paper-Scissors-Fire-Water. Here is its game table:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html my @edges = |('<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f525.png" alt="🔥" class="wp-smiley" style="height: 1em; max-height: 1em;" />' X=> $g0.vertex-list), |($g0.vertex-list X=> '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4a6.png" alt="💦" class="wp-smiley" style="height: 1em; max-height: 1em;" />'), '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4a6.png" alt="💦" class="wp-smiley" style="height: 1em; max-height: 1em;" />' => '<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f525.png" alt="🔥" class="wp-smiley" style="height: 1em; max-height: 1em;" />'; my $g-fire-water = $g0.clone.edge-add(@edges, :directed); game-table($g-fire-water) </pre></div> <figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4a6.png" alt="💦" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f525.png" alt="🔥" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th></tr></thead><tbody><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td></td><td>+</td><td>+</td><td>–</td><td>–</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4a6.png" alt="💦" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>–</td><td></td><td>–</td><td>+</td><td>–</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>–</td><td>+</td><td></td><td>–</td><td>+</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f525.png" alt="🔥" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>+</td><td>–</td><td>+</td><td></td><td>+</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>+</td><td>+</td><td>–</td><td>–</td><td></td></tr></tbody></table></figure> <p>Here is the graph:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html $g-fire-water.dot(|%opts, engine => 'neato'):svg </pre></div> <figure class="wp-block-image size-full"><img data-attachment-id="1391" data-permalink="https://rakuforprediction.wordpress.com/2025/03/03/rock-paper-scissors-extensions/06-rps-fire-water/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/06-rps-fire-water.png" data-orig-size="855,824" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="06-RPS-Fire-Water" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/06-rps-fire-water.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/06-rps-fire-water.png?w=855" loading="lazy" width="855" height="824" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/06-rps-fire-water.png" alt="" class="wp-image-1391" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/06-rps-fire-water.png 855w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/06-rps-fire-water.png?w=150&h=145 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/06-rps-fire-water.png?w=300&h=289 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/06-rps-fire-water.png?w=768&h=740 768w" sizes="(max-width: 855px) 100vw, 855px" /></figure> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Complete RPS upgrade via LLMs</h2> <p>Consider the <a href="https://www.umop.com/rps9.htm">game RPS-9</a>:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $txt = data-import('https://www.umop.com/rps9.htm', 'plaintext'); text-stats($txt) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # (chars => 2143 words => 355 lines => 46) </pre></div> <p>Extract the game description:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my ($start, $end) = 'relationships in RPS-9:', 'Each gesture beats out'; my $txt-rps9 = $txt.substr( $txt.index($start) + $start.chars .. $txt.index($end) - 1 ) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> ROCK POUNDS OUT FIRE, CRUSHES SCISSORS, HUMAN & SPONGE. FIRE MELTS SCISSORS, BURNS PAPER, HUMAN & SPONGE. SCISSORS SWISH THROUGH AIR, CUT PAPER, HUMAN & SPONGE. HUMAN CLEANS WITH SPONGE, WRITES PAPER, BREATHES AIR, DRINKS WATER. SPONGE SOAKS PAPER, USES AIR POCKETS, ABSORBS WATER, CLEANS GUN. PAPER FANS AIR, COVERS ROCK, FLOATS ON WATER, OUTLAWS GUN. AIR BLOWS OUT FIRE, ERODES ROCK, EVAPORATES WATER, TARNISHES GUN. WATER ERODES ROCK, PUTS OUT FIRE, RUSTS SCISSORS & GUN. GUN TARGETS ROCK, FIRES, OUTCLASSES SCISSORS, SHOOTS HUMAN. </pre></div> <p>Here we invoke the defined LLM function to get the edges of the corresponding graph:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my @rps-edges = |rps-edge-dataset($txt-rps9) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # [{from => ROCK, label => POUNDS OUT, to => FIRE} {from => ROCK, label => CRUSHES, to => SCISSORS} {from => ROCK, label => CRUSHES, to => HUMAN}, ..., {from => GUN, label => FIRES, to => FIRE}] </pre></div> <p>Here we translate the plaintext vertices into emojis:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my %emojied = emoji-rules(@rps-edges.map(*<from to>).flat.unique.sort) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> {AIR => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f32c.png" alt="🌬" class="wp-smiley" style="height: 1em; max-height: 1em;" />, FIRE => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f525.png" alt="🔥" class="wp-smiley" style="height: 1em; max-height: 1em;" />, GUN => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f52b.png" alt="🔫" class="wp-smiley" style="height: 1em; max-height: 1em;" />, HUMAN => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f464.png" alt="👤" class="wp-smiley" style="height: 1em; max-height: 1em;" />, PAPER => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" />, ROCK => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" />, SCISSORS => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" />, SPONGE => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f9fd.png" alt="🧽" class="wp-smiley" style="height: 1em; max-height: 1em;" />, WATER => <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4a7.png" alt="💧" class="wp-smiley" style="height: 1em; max-height: 1em;" />} </pre></div> <p>Here is the graph plot:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html my $g-rps9 = Graph.new(@rps-edges, :directed).vertex-replace(%emojied); $g-rps9.vertex-coordinates = $g-rps9.vertex-list Z=> Graph::Cycle(9).vertex-coordinates.values; my %edge-labels = Empty; $res3.map({ %edge-labels{%emojied{$_<from>}}{%emojied{$_<to>}} = "\"$_<label>\"" }); my %opts2 = %opts , %(:14node-font-size, node-shape => 'circle', node-width => 0.3, edge-width => 0.4); $g-rps9.dot(|%opts2, :!edge-labels, engine => 'neato', :svg) </pre></div> <figure class="wp-block-image size-full"><img data-attachment-id="1392" data-permalink="https://rakuforprediction.wordpress.com/2025/03/03/rock-paper-scissors-extensions/07-rps-9/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-9.png" data-orig-size="845,848" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="07-RPS-9" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-9.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-9.png?w=845" loading="lazy" width="845" height="848" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-9.png" alt="" class="wp-image-1392" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-9.png 845w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-9.png?w=150&h=150 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-9.png?w=300&h=300 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/03/07-rps-9.png?w=768&h=771 768w" sizes="(max-width: 845px) 100vw, 845px" /></figure> <p>Here is the game table:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html game-table($g-rps9) </pre></div> <figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f32c.png" alt="🌬" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f464.png" alt="👤" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4a7.png" alt="💧" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f525.png" alt="🔥" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f52b.png" alt="🔫" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f9fd.png" alt="🧽" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th><th><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" /></th></tr></thead><tbody><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2702.png" alt="✂" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td></td><td>–</td><td>+</td><td>–</td><td>+</td><td>–</td><td>–</td><td>+</td><td>–</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f32c.png" alt="🌬" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>–</td><td></td><td>–</td><td>+</td><td>–</td><td>+</td><td>+</td><td>–</td><td>+</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f464.png" alt="👤" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>–</td><td>+</td><td></td><td>+</td><td>+</td><td>–</td><td>–</td><td>+</td><td>–</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4a7.png" alt="💧" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>+</td><td>–</td><td>–</td><td></td><td>–</td><td>+</td><td>+</td><td>–</td><td>+</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f4c4.png" alt="📄" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>–</td><td>+</td><td>–</td><td>+</td><td></td><td>–</td><td>+</td><td>–</td><td>+</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f525.png" alt="🔥" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>+</td><td>–</td><td>+</td><td>–</td><td>+</td><td></td><td>–</td><td>+</td><td>–</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f52b.png" alt="🔫" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>+</td><td>–</td><td>+</td><td>–</td><td>–</td><td>–</td><td></td><td>–</td><td>+</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f9fd.png" alt="🧽" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>–</td><td>+</td><td>–</td><td>+</td><td>+</td><td>–</td><td>+</td><td></td><td>–</td></tr><tr><td><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1faa8.png" alt="🪨" class="wp-smiley" style="height: 1em; max-height: 1em;" /></td><td>+</td><td>–</td><td>+</td><td>–</td><td>–</td><td>+</td><td>–</td><td>+</td><td></td></tr></tbody></table></figure> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Future plans</h2> <p>In the (very near) future I plan to use the built-up RPS graph making know-how to make <a href="https://rakuforprediction.wordpress.com/2025/03/06/military-forces-interactions-graphs/" target="_blank" rel="noreferrer noopener">military forces interaction graphs</a>. (Discussed in [AJ1, SM1, NM1, AAv1].)</p> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>References</h2> <h3 class="wp-block-heading"><a></a>Articles, books, theses</h3> <p>[AJ1] Archer Jones, <a href="https://www.press.uillinois.edu/books/?id=p069666">“The Art of War in Western World”</a>, (2000), <a href="https://www.press.uillinois.edu/">University of Illinois Press</a>. 768 pages, ISBN-10: 0252069668, ISBN-13: 978-0252069666.</p> <p>[SM1] Sergei Makarenko et al., <a href="https://www.researchgate.net/publication/356424971_Obobsennaa_model_Lancestera_formalizuusaa_konflikt_neskolkih_storon_The_General_Lanchester_Model_Defining_Multilateral_Conflicts">“Обобщенная модель Ланчестера, формализующая конфликт нескольких сторон”</a>, [Eng. “The General Lanchester Model Defining Multilateral Conflicts”], (2021), Automation of Control Processes № 2 (64), doi: 10.35752/1991-2927-2021-2-64-66-76.</p> <p>[NM1] Николай В. Митюков, <a href="https://www.dissercat.com/content/matematicheskie-modeli-i-programmnye-sredstva-dlya-rekonstruktsii-voenno-istoricheskikh-dann">“Математические модели и программные средства для реконструкции военно-исторических данных”</a>, (2009), <a href="https://www.dissercat.com/">disserCat</a>.</p> <h3 class="wp-block-heading"><a></a>Packages</h3> <p>[AAp1] Anton Antonov, <a href="https://github.com/antononcube/Raku-Graph">Graph Raku package</a>, (2024-2025), <a href="https://github.com/antononcube">GitHub/antononcube</a>.</p> <p>[AAp2] Anton Antonov, <a href="https://github.com/antononcube/Raku-LLM-Functions">LLM::Functions Raku package</a>, (2023-2024), <a href="https://github.com/antononcube">GitHub/antononcube</a>.</p> <p>[AAp3] Anton Antonov, <a href="https://github.com/antononcube/Raku-LLM-Prompts">LLM::Prompts Raku package</a>, (2023-2024), <a href="https://github.com/antononcube">GitHub/antononcube</a>.</p> <p>[AAp4] Anton Antonov, <a href="https://github.com/antononcube/Raku-Jupyter-Chatbook">Jupyter::Chatbook Raku package</a>, (2023-2024), <a href="https://github.com/antononcube">GitHub/antononcube</a>.</p> <p>[EMp1] Elizabeth Mattijsen, <a href="https://github.com/lizmat/Text-Emoji">Text::Emoji Raku package</a>, (2024-2025), <a href="https://github.com/lizmat">GitHub/lizmat</a>.</p> <h3 class="wp-block-heading"><a></a>Videos</h3> <p>[AAv1] Anton Antonov, <a href="https://www.youtube.com/watch?v=852vMS_6Qaw">“Upgrading Epidemiological Models into War Models”</a>, (2024), <a href="https://www.youtube.com/@WolframResearch">YouTube/@WolframResearch</a>.</p> <p>[Wv1] Wozamil, <a href="https://www.youtube.com/watch?v=x5Q6-wMx-K8">“Rock Paper Scissors Lizard Spock (Extended Cut) ~ The Big Bang Theory ~”</a>, (2012), <a href="https://www.youtube.com/@Wozamil">YouTube@Wozamil</a>.</p> <h2><a href="https://rakudoweekly.blog/2025/03/03/2025-09-counting-down/">2025.09 Counting Down</h2></a> <p><h3>Published by Elizabeth Mattijsen on 2025-03-03T12:56:09</h3></p> <p>In the past 3 weeks, <em>Stefan Seifert</em> has worked on RakuAST development again, and the results are pretty amazing! Since then, <strong>86</strong> more spectest files are passing completely using the new Raku grammar, which brings completion from 86.8% to 92.5%, with only 101 spectest files (1244 -> 1345) to go. Kudos!</p> <h3 class="wp-block-heading">Alexey’s Corner</h3> <p><em>Alexey Melezhik</em> explains how you can set up your own Continuous Integration testing on Sparky without needing to code anything, in: <a href="https://sparrowdo.wordpress.com/2025/02/26/no-code-ci-for-raku-modules/">No code CI for Raku modules</a>.</p> <h3 class="wp-block-heading">Fernando’s Corner</h3> <p><em>Fernando Correa de Oliveira</em> introduces a new way of looking at event handling and persistence in part 1 of a blog series: <a href="https://dev.to/fco/sourcing-part-1-exploring-event-sourcing-in-raku-5e3">Exploring Event Sourcing in Raku</a>.</p> <h3 class="wp-block-heading">Weeklies</h3> <p><a href="https://theweeklychallenge.org/blog/perl-weekly-challenge-311/">Weekly Challenge #311</a> is available for your perusal.</p> <h3 class="wp-block-heading">New Pull Requests</h3> <ul class="wp-block-list"> <li><a href="https://github.com/Raku/doc/pull/4562">Update terms.rakudoc</a></li> </ul> <h3 class="wp-block-heading">Core Developments</h3> <p>The “master” branch in the <a href="https://github.com/rakudo/rakudo">rakudo repository</a> has been <strong>deleted</strong>. If you observe any error related to a missing “master” branch, you are warmly recommended to substitute “master” by “main”. Note that the “master” branch has been frozen for more than two years by now, and its existence only creates confusion at this time.</p> <p>Furthermore:</p> <ul class="wp-block-list"> <li><em>Jimmy Zhuo</em> and <em>Timo Paulssen</em> together got at the bottom of excessive CPU usage of an idle REPL on Windows, which actually was a bug for any event processing on Windows.</li> <li><em>Timo Paulssen</em> made sure that precomp files are correctly aligned to 8-byte boundaries, so that the underlying bytecode is also correctly aligned in memory.</li> <li><em>Elizabeth Mattijsen</em> fixed sub <code>min</code> / <code>max</code> with a <code>Range</code> argument.</li> </ul> <p>In RakuAST developments this week:</p> <ul class="wp-block-list"> <li><em>Stefan Seifert</em> continued working on RakuAST. The most significant fixes: <ul class="wp-block-list"> <li>operators / terms defined as variables</li> <li><code>return</code> with pair syntax</li> <li>several variable visibility issues at <code>BEGIN</code> time</li> <li>fixes to <code>ss///</code> and <code>S//</code></li> <li>several (sub-)signature and generics issues</li> <li>binding attributes in method arguments</li> <li>several issues related to categoricals</li> <li>support <code><|c></code> and <code><|w></code> assertions in regexes</li> <li>several <code>return</code> issues / return value issues</li> <li>progress in making <code>require</code> work</li> <li>and many, many, <strong>many</strong> more smaller fixes!</li> </ul> </li> <li><em>Elizabeth Mattijsen</em> fixed an issue with literal values.</li> <li>The number of passing test-files with the new Raku grammar are now 141/153 (<code>make test</code> +0) and 1244/1345 (<code>make spectest</code> <strong>+57</strong>).</li> </ul> <h3 class="wp-block-heading"> Meanwhile on Mastodon</h3> <ul class="wp-block-list"> <li><a href="https://fosstodon.org/@melezhik/114063021992802153">Template6 added</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://fosstodon.org/@manwar/114076955754327846">Same, but with different arguments</a> by <em>Mohammad Sajid Anwar</em>.</li> </ul> <h3 class="wp-block-heading">Questions about Raku</h3> <ul class="wp-block-list"> <li><a href="https://bitbang.social/@profoundlynerdy/114060452695309473">How to create a custom encoding?</a> by <em>Profoundly Nerdly</em>.</li> <li><a href="https://www.reddit.com/r/rakulang/comments/1ixtx2g/can_i_pass_a_hash_directly_into_a_constructor_ie/">Can I pass a hash directly into a constructor (i.e. instead of named arguments)?</a> by <em>ralfmuschall</em>.</li> </ul> <h3 class="wp-block-heading">Comments about Raku</h3> <ul class="wp-block-list"> <li><a href="https://news.ycombinator.com/item?id=43184599">Easy and powerful</a> by <em>Steve Roe</em>.</li> <li><a href="https://www.reddit.com/r/ProgrammingLanguages/comments/1iz4yl5/comment/mf0auxi/">On <code>Q</code> lang</a> by <em>Ralph Mellor</em>.</li> <li><a href="https://news.ycombinator.com/item?id=43203494">Planning to mess with <code>LLM::Functions</code></a> by <em>Steve Roe</em>.</li> </ul> <h3 class="wp-block-heading">New Raku Modules</h3> <ul class="wp-block-list"> <li><a href="https://raku.land/zef:librasteve/Air">Air</a> “A way to create web components without cro templates” by <em>Steve Roe</em>.</li> <li><a href="https://raku.land/zef:librasteve/Air::Play">Air::Play</a> “Playing with the HARC stack” by <em>Steve Roe</em>.</li> </ul> <h3 class="wp-block-heading">Updated Raku Modules</h3> <ul class="wp-block-list"> <li><a href="https://raku.land/zef:lizmat/CodeUnit">CodeUnit</a>, <a href="https://raku.land/zef:lizmat/Ecosystem::Cache">Ecosystem::Cache</a>, <a href="https://raku.land/zef:lizmat/JSON::Fast::Hyper">JSON::Fast::Hyper</a> by <em>Elizabeth Mattijsen</em>.</li> <li><a href="https://raku.land/zef:melezhik/Sparrow6">Sparrow6</a>, <a href="https://raku.land/zef:melezhik/Sparrowdo">Sparrowdo</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://raku.land/zef:ingy/YAMLScript">YAMLScript</a> by <em>Ingy döt Net</em>.</li> <li><a href="https://raku.land/zef:tbrowder/Number::More">Number::More</a> by <em>Tom Browder</em>.</li> <li><a href="https://raku.land/zef:finanalyst/Rakuast::RakuDoc::Render">Rakuast::RakuDoc::Render</a> by <em>Richard Hainsworth</em>.</li> <li><a href="https://raku.land/zef:antononcube/Math::NumberTheory">Math::NumberTheory</a> by <em>Anton Antonov</em>.</li> <li><a href="https://raku.land/github:ugexe/zef">zef</a> by <em>Nick Logan</em>.</li> <li><a href="https://raku.land/zef:bduggan/Geo::Basic">Geo::Basic</a> by <em>Brian Duggan</em>.</li> <li><a href="https://raku.land/zef:demayl/SSH::LibSSH">SSH::LibSSH</a> by <em>Denis Kanchev</em>.</li> </ul> <h3 class="wp-block-heading"><a href="https://github.com/lizmat/articles/blob/main/weekly-2022.51.md#winding-down"></a>Winding down</h3> <p>Please keep staying safe and healthy, and keep up the good work! Even after this even crazier week 6 of hopefully only 209.</p> <p> Meanwhile, still: Слава Україні!  Героям слава!</p> <p class="has-small-font-size">If you like what I’m doing, committing to a <a href="https://github.com/sponsors/lizmat/">small sponsorship</a> would mean a great deal!</p> <h2><a href="https://dev.to/fco/sourcing-part-1-exploring-event-sourcing-in-raku-5e3">Sourcing: Part 1 – Exploring Event Sourcing in Raku</h2></a> <p><h3>Published by Fernando Correa de Oliveira on 2025-03-02T17:11:16</h3></p> <p>I’ve been studying a very interesting approach to building systems called Event Sourcing. In event sourcing, every change to an application’s state is captured as an immutable event. Instead of storing only the current state, you persist a sequence of events that record all the actions that have taken place. This approach not only allows for an accurate reconstruction of the system state by replaying events but also provides a robust audit log and improves debugging and analysis capabilities.</p> <p>For my studies, I started writing a framework to easily implement event sourcing in Raku. It’s called Sourcing.</p> <p>Sourcing has a few key concepts. In this first part, we’ll discuss three of them:</p> <ol> <li>Events</li> <li>Event Store</li> <li>Projections</li> </ol> <h2> Events </h2> <p>Events are the core building blocks in an event-sourced system. Each event represents a fact that has occurred in the past, such as an order being created or a payment being initiated.</p> <p>Here are some important points about events:</p> <ul> <li>Immutability:</li> </ul> <p>Once an event is recorded, it should never change. This immutability guarantees that the historical record remains accurate and reliable.</p> <ul> <li>Past Facts:</li> </ul> <p>Events capture facts that have already happened, meaning they are a definitive record of the state transitions within your system.</p> <ul> <li>Audit Trail:</li> </ul> <p>By recording every event, you create a full history of all actions, making it easier to trace issues or understand how a certain state was reached.</p> <h2> Event Store </h2> <p>The Event Store is a specialized storage mechanism for events. It acts as a single source of truth for all the state changes in your system by providing these features:</p> <ul> <li>Append-Only Log:</li> </ul> <p>Events are stored in an append-only manner, ensuring that once an event is recorded, it is never modified.</p> <ul> <li>Order Preservation:</li> </ul> <p>The sequence of events is maintained, which is critical for reconstructing the state by replaying the events in the order they occurred.</p> <ul> <li>Durability and Auditability:</li> </ul> <p>The event store keeps a permanent record of every event, which can be used for debugging, analytics, or even recreating the state at any given point in time.</p> <h2> Projections </h2> <p>Projections are responsible for transforming the raw stream of events into a read model that is easier to query and work with.</p> <p>Many projections can/should use the same set of events.<br> They serve several important functions:</p> <ul> <li>State Aggregation:</li> </ul> <p>Projections listen to events from the Event Store and update a view or model that represents the current state of the system.</p> <ul> <li>Multiple Views:</li> </ul> <p>You can create multiple projections for the same set of events, each tailored to different client needs. For example, one projection might track the status of a food order while another aggregates sales data.</p> <ul> <li>Decoupling:</li> </ul> <p>By separating the write model (events) from the read model (projections), you can optimize the performance and scalability of your application.</p> <p>The following code shows how a projection, DeliveryStatus,<br> is implemented in our framework. Each projection must define an apply method for every event it wants to handle.<br> When an event is emitted, the projection’s corresponding apply method is called to update its state.</p> <h2> Food Delivery Example </h2> <p>To demonstrate the framework, let’s build a simple Food Delivery system. This system uses a few types of events to represent different stages in the order lifecycle, an example of one of the events class wold be:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>use Sourcing; use UUID::V4; unit event Sourcing::FoodDelivery::Event::OrderCreated; has Str $.order-id = uuid-v4; has Str $.delivery-code = (1 ..^ 100)>> .fmt("%02d") .pick; has UInt $.user-id is required; has UInt $.restaurant-id is required; has Str %.item{Str}; </code></pre> </div> <div class="table-wrapper-paragraph"><table> <thead> <tr> <th>Event</th> <th>description</th> </tr> </thead> <tbody> <tr> <td>OrderCreated</td> <td>Represents when a new order is created.</td> </tr> <tr> <td>PaymentInitiated</td> <td>Indicates that the payment process has started.</td> </tr> <tr> <td>PaymentConfirmed</td> <td>Signals that the payment was successfully completed.</td> </tr> <tr> <td>PaymentFailed</td> <td>Represents a failure or error during the payment process.</td> </tr> <tr> <td>OrderAccepted</td> <td>Indicates that the restaurant has accepted the order and begun preparation.</td> </tr> <tr> <td>DeliveryStarted</td> <td>Marks the moment when the deliverer collects the order.</td> </tr> <tr> <td>DeliveryCompleted</td> <td>Represents the event when the deliverer confirms that the delivery is complete.</td> </tr> <tr> <td>OrderDelivered</td> <td>Marks the final stage when the order is successfully delivered.</td> </tr> </tbody> </table></div> <h2> Projection: DeliveryStatus </h2> <p>Projections consume events to maintain a read model. In this example, the DeliveryStatus projection tracks the status of an order and stores the delivery code required for the deliverer.</p> <p>When a projection object is created, it is registered with a manager (to be described in a future post) and remains in memory until it is unloaded (a feature still in development).</p> <p>Projections don't need to use all events and events don't need to be used by projections. On this example, I'm defining some Payment events that are not being used by the projection. That's completely ok. if in the future, a new projection is created, it will consume those events (if defined to) and generate new objects using those historical data.<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>use Sourcing; use OrderCreated; use OrderAccepted; use DeliveryStarted; use DeliveryCompleted; use OrderDelivered; unit projection Sourcing::FoodDelivery::DeliveryStatus; has Str $.order-id is aggregation-id; has Str $.status = ""; has DateTime $.last-status .= now; has Str $.delivery-code; method summary is query{ :sync } { %( :$!order-id, :$!status, :$!last-status, :$!delivery-code, ) } multi method apply(OrderCreated $_) { $!status = "placed"; $!last-status = .timestamp; $!delivery-code = .delivery-code; } multi method apply(OrderAccepted $_) { $!status = "preparing"; $!last-status = .timestamp; } multi method apply(DeliveryStarted $_) { $!status = "collected"; $!last-status = .timestamp; } multi method apply(DeliveryCompleted $_) { $!status = "delivered"; $!last-status = .timestamp; } multi method apply(OrderDelivered $_) { $!status = "done"; $!last-status = .timestamp; } </code></pre> </div> <h3> The apply Method </h3> <p>Every projection needs an apply method candidate for each event it intends to handle. When an event is emitted, the projection automatically calls the corresponding apply method to update its state based on the event’s data.</p> <h3> The is query Trait </h3> <p>Methods marked with the is query trait are exposed to clients. When marked with the <code>:sync</code> parameter, these methods first process all pending events, ensuring that the projection’s state is up-to-date before returning the result.</p> <h2> Usage Example </h2> <p>Note: Although events should ideally be emitted via dedicated commands, for this example we emit them directly.<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>... my Sourcing::Client $s = Sourcing::Client.new; $s.register-class: "DeliveryStatus"; my RedEventStore::Client $store = RedEventStore::Client.new, my \DeliveryStatus = $s.get-class-client("DeliveryStatus"); sub emit-event(Sourcing::Event $event, :$order) { say "{ gray $event.timestamp.hh-mm-ss }: emitting the event: { yellow $event.^shortname }"; $store.add-event: $event; given $order.summary { say "Current state: STATUS: { green .<status> }; CODE: { green .<delivery-code> }" } sleep 1 } sub MAIN(Str :$order-id = uuid-v4) { say "Using order-id: { red $order-id }"; my $order = DeliveryStatus.new: :$order-id; my $user-id = ^10 .pick; my $restaurant-id = ^10 .pick; my $prepare-minutes = 10; my $deliverer-id = ^10 .pick; emit-event :$order, OrderCreated.new: :$order-id, :$user-id, :$restaurant-id; emit-event :$order, PaymentInitiated.new: :$order-id, :payment-data<blablabla>; emit-event :$order, PaymentConfirmed.new: :$order-id; emit-event :$order, OrderAccepted.new: :$order-id, :$restaurant-id, :$prepare-minutes; emit-event :$order, DeliveryStarted.new: :$order-id, :$deliverer-id; emit-event :$order, DeliveryCompleted.new: :$order-id, :delivery-code("10"); emit-event :$order, OrderDelivered.new: :$order-id; } </code></pre> </div> <p>Running this, it will print out something like this:<br> <a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fqzu6h6vg899e5gckhc6b.png" class="article-body-image-wrapper"><img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fqzu6h6vg899e5gckhc6b.png" alt="output" width="800" height="529"></a></p> <h2> Projection: UserOrders </h2> <p>Another simple projection we could use here would be a projection to show all orders of a user. It would be simply consume a single type of event.</p> <p>Different from the first projection that aggregates the events using the order-id, the UserOrders uses the user-id as aggregation-id to group all event "from" the same user together. (I plan to, at some point, make it possible to map event attributes with different named to a aggregation-id on projections).<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>use Sourcing; use OrderCreated; unit projection Sourcing::FoodDelivery::UserOrders; has Str $.user-id is aggregation-id; has Str @.orders; method users-orders is query { @!orders } multi method apply(OrderCreated $_) { @!orders.push: .order-id } </code></pre> </div> <h2> Conclusion </h2> <p>In this post, we explored the fundamentals of event sourcing and how to implement a basic framework in Raku using the Sourcing library.</p> <p>We covered the core concepts of events, the event store, and projections, and demonstrated these ideas with a practical Food Delivery example.</p> <p>Event sourcing provides a robust foundation for building systems that are scalable, auditable, and easier to debug. In future posts, we’ll delve deeper into advanced topics such as command handling, event replay, and integrating with production-grade data stores. Stay tuned for more insights into building reliable and resilient systems with event sourcing!</p> <h2><a href="https://rakudoweekly.blog/2025/02/24/2025-08-starting-an-avalanche/">2025.08 Starting An Avalanche</h2></a> <p><h3>Published by Elizabeth Mattijsen on 2025-02-24T13:21:18</h3></p> <p><em>Elizabeth Mattijsen</em> started a series of blog posts about their <a href="https://raku.land/zef:lizmat/REPL">REPL distribution</a>, which in turn formed the impetus for six other distributions, each with their own unique features. The first episode is called: <a href="https://dev.to/lizmat/repl-avalanche-45hh">REPL Avalanche</a>!</p> <h1 class="wp-block-heading">Rakudo Compiler Release</h1> <p><em>Justin DeVuyst</em> (with help by <em>Will Coleda</em>) has produced the second release of the Rakudo compiler for the Raku Programming Language in 2025: <a href="https://rakudo.org/post/announce-rakudo-release-2025.02">2025.02</a>.</p> <p>This release includes fixes such as allowing heredocs in the <code>REPL</code>, Hashes in which the order of keys is fixed, make <code>foo.item</code> and <code>$(foo)</code> <strong>always</strong> return a container, and several hyper operator related fixes.</p> <p><a href="https://rakudo.org/downloads">Binary packages</a> will become available shortly, as well as updates to <a href="https://rakudo.org/star">Rakudo Star</a>, if they are not already. Kudos to all involved!</p> <h3 class="wp-block-heading">Alexey’s Corner</h3> <p><em>Alexey Melezhik</em> explained how to <a href="https://sparrowdo.wordpress.com/2025/02/19/managing-multiple-ssh-hosts-using-inventory-files-in-sparrowdo/">manage multiple ssh hosts with Sparrowdo</a>. Furthermore, they are working on getting the <a href="https://rockylinux.org">Rocky Linux</a> distribution to <a href="https://github.com/melezhik/rocky-linux-lamp-check">use Sparrowdo to check its LAMP installation</a>.</p> <h3 class="wp-block-heading">André’s Corner</h3> <p><em>André Machado</em> has written a nice introductory blog post about the Raku Programming Language: <a href="https://machaddr.substack.com/p/raku-a-journey-of-innovation-and">A Journey of Innovation and Community-Driven Expressiveness</a>. Although yours truly can’t help but feel the blog post got a little AI help.</p> <h3 class="wp-block-heading">Weeklies</h3> <p><a href="https://theweeklychallenge.org/blog/perl-weekly-challenge-310/">Weekly Challenge #310</a> is available for your perusal.</p> <h3 class="wp-block-heading">New Problem Solving Issues</h3> <ul class="wp-block-list"> <li><a href="https://github.com/raku/problem-solving/issues/465">Add a Rakupod directive for a page break</a></li> <li><a href="https://github.com/raku/problem-solving/issues/466">There is no easy way to use an <code>EXPORT</code> sub to export a long named spaces class</a></li> </ul> <h3 class="wp-block-heading">New Pull Requests</h3> <ul class="wp-block-list"> <li><a href="https://github.com/Raku/doc/pull/4559">Split 5to6 section into 3 paragraphs</a></li> </ul> <h3 class="wp-block-heading">Core Developments</h3> <ul class="wp-block-list"> <li><em>Daniel Green</em> provided some micro-optimizations in MoarVM, NQP and Rakudo.</li> <li><em>Elizabeth Mattijsen</em> was also busy: <ul class="wp-block-list"> <li>fixed multi-dimensional array slices with lazy indices in 6.e</li> <li><code>Blob ~ Blobx</code> takes the type of left-hand side in 6.e</li> <li>fixed <code>Associative.deepmap</code> with regards to handling <code>Empty</code></li> <li>fixed <code>Range.Numeric</code> with regards to handling <code>Inf</code> and <code>-Inf</code> endpoints</li> <li>added support for <code>0</code> in <code>rotor</code></li> <li>make grammar changes persist in the REPL (e.g. with custom operators)</li> <li>allow for multi-line comments in the REPL</li> </ul> </li> <li>And quite a few smaller tweaks and fixes.</li> </ul> <p>In RakuAST developments this week:</p> <ul class="wp-block-list"> <li><em>Stefan Seifert</em> continued working on RakuAST. The most significant fixes: <ul class="wp-block-list"> <li>non-trivial lazy loops</li> <li>allow declaration of <code>$_</code> in loops and other loop related fixes</li> <li>handling labels with loop structures</li> <li>a large number of regex related features, such as fixing LTM (Longest Token Match) matching and interpolation of attributes in regexes</li> <li>exceptions thrown in <code>CHECK</code> phasers</li> <li>support added for <code>tr///</code> and <code>TR///</code></li> <li>better handling of subroutine stubs</li> <li>and many, many more smaller fixes!</li> </ul> </li> <li>The number of passing test-files with the new Raku grammar are now 141/153 (<code>make test</code> -1) and 1187/1355 (<code>make spectest</code> +17).</li> </ul> <h3 class="wp-block-heading"> Meanwhile on Mastodon</h3> <ul class="wp-block-list"> <li><a href="https://fosstodon.org/@melezhik/114021067737517274">Best of both worlds</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://fosstodon.org/@melezhik/114021085421562761">Grouping of hosts</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://fosstodon.org/@melezhik/114024786223580118">An APKBUILD file checker</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://fosstodon.org/@melezhik/114035695448405050">Closer to Fabric</a> by <em>Alexey Melezhik</em>.</li> </ul> <h3 class="wp-block-heading">Meanwhile on BlueSky</h3> <ul class="wp-block-list"> <li><a href="https://bsky.app/profile/melezhik.bsky.social/post/3lim3hb6iv22m">Managing multiple ssh hosts</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://bsky.app/profile/richardkadrey.bsky.social/post/3lihlcqzvz222">Brackets, and what they mean</a> by <em>Richard Kadrey</em>.</li> </ul> <h3 class="wp-block-heading">Questions about Raku</h3> <ul class="wp-block-list"> <li><a href="https://stackoverflow.com/questions/79449714/how-to-explicitly-use-the-default-value-of-a-parameter">How to explicitly use the default value of a parameter?</a> by <em>fingolfin</em>.</li> </ul> <h3 class="wp-block-heading">Comments about Raku</h3> <ul class="wp-block-list"> <li><a href="https://lobste.rs/s/oxkl4d/open_letter_browser_os_makers#c_3bbapz">So masochistic</a> by <em>Summer</em>.</li> <li><a href="https://www.reddit.com/r/ProgrammingLanguages/comments/1itzcn1/comment/me03hj8/">25 years of steady design and implementation work</a> by <em>Ralph Mellor</em>.</li> </ul> <h3 class="wp-block-heading">New Raku Modules</h3> <ul class="wp-block-list"> <li><a href="https://raku.land/zef:lizmat/CodeUnit">CodeUnit</a> “Provide a unit for execution of code” by <em>Elizabeth Mattijsen</em>.</li> </ul> <h3 class="wp-block-heading">Updated Raku Modules</h3> <ul class="wp-block-list"> <li><a href="https://raku.land/zef:dwarring/PDF::Font::Loader::HarfBuzz">PDF::Font::Loader::HarfBuzz</a>, <a href="https://raku.land/zef:dwarring/PDF::Content">PDF::Content</a>, <a href="https://raku.land/zef:dwarring/Font::FreeType">Font::FreeType</a> by <em>David Warring</em>.</li> <li><a href="https://raku.land/zef:raku-community-modules/XML">XML</a> by <em>Tim Nelson</em>.</li> <li><a href="https://raku.land/zef:melezhik/Sparky">Sparky</a>, <a href="https://raku.land/zef:melezhik/Sparrowdo">Sparrowdo</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://raku.land/zef:lizmat/REPL">REPL</a>, <a href="https://raku.land/zef:raku-community-modules/Data::MessagePack">Data::MessagePack</a>, <a href="https://raku.land/zef:lizmat/Commands">Commands</a>, <a href="https://raku.land/zef:lizmat/Text::Emoji">Text::Emoji</a>, <a href="https://raku.land/zef:lizmat/Ecosystem::Cache">Ecosystem::Cache</a> by <em>Elizabeth Mattijsen</em>.</li> <li><a href="https://raku.land/zef:antononcube/Math::NumberTheory">Math::NumberTheory</a> by <em>Anton Antonov</em>.</li> <li><a href="https://raku.land/zef:2colours/Ddt">Ddt</a> by <em>Márton Polgár</em>.</li> </ul> <h3 class="wp-block-heading"><a href="https://github.com/lizmat/articles/blob/main/weekly-2022.51.md#winding-down"></a>Winding down</h3> <p>Again, quite a lot of core work this week, both on old and new stuff!</p> <p>Please keep staying safe and healthy, and keep up the good work! Even after week 5 of hopefully only 209.</p> <p> Meanwhile, still: Слава Україні!  Героям слава!</p> <p class="has-small-font-size">If you like what I’m doing, committing to a <a href="https://github.com/sponsors/lizmat/">small sponsorship</a> would mean a great deal!</p> <h2><a href="https://dev.to/lizmat/repl-avalanche-45hh">REPL Avalanche</h2></a> <p><h3>Published by Elizabeth Mattijsen on 2025-02-22T13:37:25</h3></p> <p>Sometime in early November, I decided to have a look at re-imagining the <a href="https://docs.raku.org/language/REPL" rel="noopener noreferrer">REPL (Read, Eval, Print, Loop)</a> that is provided by default by the <a href="https://raku.org" rel="noopener noreferrer">Raku Programming Language</a>. Little did I know that that work would in the end result in <em>six</em> new distributions. Each providing some useful sub-functionality of the REPL, but also providing useful functionality outside of the REPL context.</p> <blockquote> <p>Parallel to this effort, a <a href="https://github.com/raku/problem-solving/issues/459" rel="noopener noreferrer">Problem Solving Issue</a> was created about the lack of configurability of the standard REPL, and a proof of concept <a href="https://github.com/Raku/problem-solving/pull/460" rel="noopener noreferrer">Pull Request</a> was made by <em>Will Coleda</em>, which proved to be inspirational.</p> </blockquote> <h2> The standard REPL </h2> <p>But first, for the uninitiated, a small introduction. The Raku standard REPL can be invoked by calling <code>raku</code> from the command line without any arguments:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>$ raku Welcome to Rakudo™ v2025.01. Implementing the Raku® Programming Language v6.d. Built on MoarVM version 2025.01. To exit type 'exit' or '^D' [0] > </code></pre> </div> <p>And then you can enter Raku code and have it immediately executed. For example:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>[0] > my $a = 42 42 [1] > say "a = $a" a = 42 [1] > say $*0 42 [1] > exit $ </code></pre> </div> <p>The number between square brackets indicates the number of expression values that have been saved. It is also the index at which the next expression value will be saved.</p> <p>Note that that number went from <code>0</code> to <code>1</code> after the first line entered. But <strong>not</strong> after the second line entered. This is because the first line did <strong>not</strong> cause any output. In that case, the REPL will show the value of the expression and keep it for future reference.</p> <p>The second line entered <em>did</em> cause some output, so the value was <strong>not</strong> saved, and the index was <strong>not</strong> incremented.</p> <p>Finally, the third line shows how you can refer to the originally saved value, by accessing the dynamic variable <code>$*0</code> (with the number corresponding to the index at which the value was saved).</p> <blockquote> <p>There is no help available in the standard REPL: it doesn't even have any commands! The way to exit the REPL is to type "exit". Which is in fact calling the Raku <a href="https://docs.raku.org/routine/exit" rel="noopener noreferrer"><code>exit</code></a> function that exits the current process.</p> </blockquote> <h2> Three months on </h2> <p>Three months on since November 2024, the <a href="https://raku.land/zef:lizmat/REPL" rel="noopener noreferrer"><code>REPL</code></a> distribution provides a REPL that is ready for production (so to speak, as the use of this tool in production would be limited).</p> <blockquote> <p><code>zef install REPL</code> is enough to install this distribution and all of its dependencies.</p> </blockquote> <p>The <code>REPL</code> distribution provides the same features as the standard Raku REPL, but also provides:</p> <ul> <li>configurable prompt through command line arguments and environment variables</li> <li>REPL specific commands that can be entered by starting the line with "=" to distinguish them from Raku code</li> <li>command shortcuts ("=q" being short for "=quit" to exit the REPL)</li> <li>help sections for beginners (=introduction, =completions)</li> <li>context-sensitive TAB completions</li> <li>special purpose TAB completions (\123 → ¹²³ → ₁₂₃, foo! → FOO → Foo, \heart → 🫀 → 💓 → ...)</li> <li>show stack trace (=stack), only makes sense if called when in the <code>repl</code> sub</li> <li>save code entered so far (=write)</li> <li>reload code that was saved before (=read)</li> <li>edit file inside the repl or code saved (=edit)</li> <li>allow creation / switching between contexts (=context)</li> <li>installs a command-line script <code>repl</code> for direct access</li> <li>provides a <code>REPL</code> role to facilitate further customization</li> </ul> <p>Over the coming months, a number of additional features will probably be added as well, depending on demand. But first, let's have a look at one of the features.</p> <h2> Configuring the prompt </h2> <p>When using the <code>repl</code> command line script, one can customize the prompt with the <code>--the-prompt</code> and <code>--symbols</code> named arguments. For example:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>$ repl --the-prompt='[:index:] :HHMM: :symbol: ' --symbols=🦋,🔥 Welcome to Rakudo™ v2025.01. Implementing the Raku® Programming Language v6.d. Built on MoarVM version 2025.01. To exit type '=quit' or '^D'. Type '=help' for help. [0] 20:51 🦋 if 42 { [0] 20:51 🔥 say "foo" [0] 20:51 🔥 } foo [0] 20:52 🦋 </code></pre> </div> <p>The <code>--the-prompt</code> part of the customization exists of a string that can hold any Unicode codepoints, as well as:</p> <ul> <li>ANSI color / formatting codes (e.g. <code>\e[33m</code>)</li> <li> <code>strftime</code> escape codes (e.g. <code>%R</code>)</li> <li>ANSI color placeholder (.e.g. :yellow:)</li> <li> <code>strftime</code> placeholder (.e.g. :HHMM:)</li> <li>emoji placeholder specification (.e.g. 🦋)</li> <li>special placeholder (.e.g. :index:, :symbol:)</li> </ul> <blockquote> <p>That's quite a lot of possibilities: the <a href="https://raku.land/zef:lizmat/Prompt::Expand" rel="noopener noreferrer"><code>Prompt::Expand</code></a> distribution provides an overview, as well as the <a href="https://raku.land/zef:lizmat/Text::Emoji" rel="noopener noreferrer"><code>Text::Emoji</code></a> distribution for the emojis.</p> </blockquote> <p>The same applies for the <code>--symbols</code> argument. This argument specifies what the <code>:symbol:</code> placeholder should be converted to. This symbol indicates the state of the REPL with regards to code compilation. Two states are currently recognized:</p> <ul> <li>ready to start new expression (in this example: 🦋, default ">")</li> <li>in the middle of a multi-line expression (in this example: 🔥, default "*").</li> </ul> <p>Note that the different states are separated by a comma in the specification.</p> <p>If the <code>--the-prompt</code> part does not contain a <code>:symbol:</code> placeholder, it will be automatically added at the end. So the above example could also be written as:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>$ repl --the-prompt='[:index'] %R' --symbols=':butterfly:,:fire:' </code></pre> </div> <p>"That's all nice, but I really don't want to enter all of those arguments every time", you might think, or even say out loud. Fear not, there are also environment variables that you can set to achieve the same result. The above example could also be written as:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>$ export RAKUDO_REPL_PROMPT='[:index:] :HHMM:' $ export RAKUDO_REPL_SYMBOLS=':butterfly:,:fire:' $ repl [0] 20:57 🦋 </code></pre> </div> <p>If you want, you can put those environment variables in your startup script, so that these settings always apply by default.</p> <blockquote> <p>A word of caution: yours truly has spent a lot of time playing with the prompt settings. It can be a serious time-sink.</p> </blockquote> <p>And of course, if you're only interested in removing the <code>[0]</code> part from the prompt (as was the original reason for the Problem Solving Issue), you can just export an empty <code>RAKUDO_REPL_PROMPT</code>: <code>export RAKUDO_REPL_PROMPT=</code>.</p> <h2> Still in core? </h2> <p>Although it was my original intent that the <code>REPL</code> module would be incorporated into the core, it now feels like it has gotten so many features (and dependencies) that it has probably become too big to be incorporated in the Raku core. Which leads to the question whether a subset of the functionality should be incorporated in core, or that maybe the REPL should be removed from core altogether. And include the <code>REPL</code> distribution and its dependencies in all derived packaging, such as Rakudo Star.</p> <h2> Conclusion </h2> <p>This is the first part of a series of blog posts about the <code>REPL</code> distribution. Future installments will look at the available commands, the completions logic, and how the <code>repl</code> subroutine can be used in debugging your code.</p> <p>If you're curious as to which new distributions that were created to supporte the <code>REPL</code> distribution, and can't wait, here they are (in alphabetical order):</p> <ul> <li> <a href="https://raku.land/zef:lizmat/CodeUnit" rel="noopener noreferrer"><code>CodeUnit</code></a> - provide a unit for execution of code</li> <li> <a href="https://raku.land/zef:lizmat/Commands" rel="noopener noreferrer"><code>Commands</code></a> - handle interactive user commands</li> <li> <a href="https://raku.land/zef:lizmat/DateTime::strftime" rel="noopener noreferrer"><code>DateTime::strftime</code></a> - provide strftime() formatting for <code>DateTime</code> objects</li> <li> <a href="https://raku.land/zef:lizmat/Prompt" rel="noopener noreferrer"><code>Prompt</code></a> - a smarter prompt for user interaction</li> <li> <a href="https://raku.land/zef:lizmat/Prompt::Expand" rel="noopener noreferrer"><code>Prompt::Expand</code></a> - provide prompt expansion logic</li> <li> <a href="https://raku.land/zef:lizmat/Text::Emoji" rel="noopener noreferrer"><code>Text::Emoji</code></a> - provide :text: to emoji translation</li> </ul> <p>Image courtesy of <em>Salve J. Nilsen</em></p> <p><em>If you like what I'm doing, committing to a <a href="https://github.com/sponsors/lizmat/" rel="noopener noreferrer">small sponsorship</a> would mean a great deal to me!</em></p> <h2><a href="https://rakudo.org/post/announce-rakudo-release-2025.02"> Rakudo compiler, Release #180 (2025.02)</h2></a> <p><h3>Published on 2025-02-22T00:00:00</h3></p> <h2><a href="https://rakudoweekly.blog/2025/02/17/2025-07-unexpected-quora/">2025.07 Unexpected Quora</h2></a> <p><h3>Published by Elizabeth Mattijsen on 2025-02-17T15:15:05</h3></p> <p><em>Jan M Savage</em> has answered several questions on Quora the past week:</p> <ul class="wp-block-list"> <li><a href="https://www.quora.com/What-are-some-experiences-using-GitHub-Copilot-agent-mode/answer/Jan-M-Savage">Some AI experiences?</a></li> <li><a href="https://www.quora.com/Is-Perl-or-Raku-worth-it-in-2022/answer/Jan-M-Savage">Is it worth it?</a></li> <li><a href="https://www.quora.com/How-do-I-write-a-program-in-C-to-accept-three-sides-of-a-triangle-and-determine-whether-they-form-a-valid-triangle/answer/Jan-M-Savage">A valid triangle?</a></li> <li><a href="https://www.quora.com/What-is-a-program-to-print-an-odd-number-from-1-to-9-using-a-for-loop/answer/Jan-M-Savage">An odd number?</a></li> </ul> <p>Although a nitpicker could find fault with some of the answers and examples, it’s really nice to see people liking the Raku Programming Language from unexpected corners of the world!</p> <h3 class="wp-block-heading">Alexey’s Corner</h3> <p><em>Alexey Melezhik</em> <a href="https://sparrowdo.wordpress.com/2025/02/04/harden-syslog-conf-by-sparrow-checks/">hardened their <code>sysctl.conf</code></a>.</p> <h3 class="wp-block-heading">Weeklies</h3> <p><a href="https://theweeklychallenge.org/blog/perl-weekly-challenge-309/">Weekly Challenge #309</a> is available for your perusal.</p> <h3 class="wp-block-heading">New Pull Requests</h3> <ul class="wp-block-list"> <li><a href="https://github.com/Raku/nqp/pull/836">Use locals instead of class variables</a></li> </ul> <h3 class="wp-block-heading">Core Developments</h3> <ul class="wp-block-list"> <li><em>Daniel Green</em> simplified decoding error reporting function on MoarVM.</li> <li><em>Timo Paulssen</em> and <em>Daniel Green</em> fixed several recently introduced decoding issues.</li> <li><em>Stefan Seifert</em> properly fixed a long standing NQP issue with register initialization when writing bytecode for repeat loops.</li> <li><em>Elizabeth Mattijsen</em> was also busy: <ul class="wp-block-list"> <li>made <code>foo.item</code> (and thus <code>$(foo)</code>) <strong>always</strong> return a container</li> <li>allowed here-docs in the REPL</li> <li>fixed hyper operators and <code>.deepmap</code> producing unexpected types / errors</li> <li>introduced ordered hashes with <code>Hash::Ordered</code></li> <li>fixed a <code>MAIN</code> dispatch issue with named array arguments</li> </ul> </li> </ul> <p>In RakuAST developments this week:</p> <ul class="wp-block-list"> <li><em>Stefan Seifert</em> started working on RakuAST again: <ul class="wp-block-list"> <li>pointy blocks with loop modifiers</li> <li>quite a lot of (nested) thunk / block related issues</li> <li>post constraints on non-subset type parameters</li> <li>exceptions in <code>.ACCEPTS</code> at compile time</li> <li>fixed implicits such <code>&?ROUTINE</code> in methods</li> <li>fixed labels on lazy loops</li> <li>several regex related issues</li> <li>and many more smaller fixes!</li> </ul> </li> <li>The number of passing test-files with the new Raku grammar are now 142/153 (<code>make test</code> +0) and 1170/1355 (<code>make spectest</code> +12).</li> </ul> <h3 class="wp-block-heading"> Meanwhile on Mastodon</h3> <ul class="wp-block-list"> <li><a href="https://fosstodon.org/@melezhik/113986747915961426">Like Ansible</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://fosstodon.org/@melezhik/113986794463740717">Simple, super flexible</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://fosstodon.org/@melezhik/113986878564509926">If you don’t mind a bit of Raku</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://fantastic.earth/@samebchase/113989468465919642">Solved an exercise</a> by <em>Samuel Chase</em>.</li> <li><a href="https://fosstodon.org/@melezhik/113990913327838025">Doesn’t scale well</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://fosstodon.org/@melezhik/113991096740784071">With an Ansible like DSL</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://fosstodon.org/@melezhik/114002916014540393">More isolated plugins</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://fosstodon.org/@melezhik/114003062381135798">Minimalistic CICD</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://chaos.social/@sjn/114008648429337137">Are you a bylaws geek?</a> by <em>Salve J. Nilsen</em>.</li> <li><a href="https://peoplemaking.games/@timotimo/114009266069130673">The <code>!</code> meta-operator</a> by <em>Timo Paulsen</em>.</li> </ul> <h3 class="wp-block-heading">Questions about Raku</h3> <ul class="wp-block-list"> <li><a href="https://stackoverflow.com/questions/79434774/using-recursive-regexes-in-raku-how-to-limit-recursion-levels">Using Recursive Regexes in Raku: how to limit recursion-levels?</a> by <em>jubilatious1</em>.</li> </ul> <h3 class="wp-block-heading">Comments about Raku</h3> <ul class="wp-block-list"> <li><a href="https://news.ycombinator.com/item?id=43069380">Rats by default</a> by <em>Steve Roe</em>.</li> </ul> <h3 class="wp-block-heading">New Raku Modules</h3> <ul class="wp-block-list"> <li><a href="https://raku.land/zef:demayl/Cro::HTTP::Session::SQLite">Cro::HTTP::Session::SQLite</a> “An implementation of Cro persistent sessions using SQLite” by <em>Denis Kanchev</em>.</li> </ul> <h3 class="wp-block-heading">Updated Raku Modules</h3> <ul class="wp-block-list"> <li><a href="https://raku.land/zef:lizmat/App::Rak::Markdown">App::Rak::Markdown</a>, <a href="https://raku.land/zef:lizmat/rak">rak</a>, <a href="https://raku.land/zef:lizmat/Git::Files">Git::Files</a>, <a href="https://raku.land/zef:raku-community-modules/Chemistry::Elements">Chemistry::Elements</a>, <a href="https://raku.land/zef:lizmat/Rakudo::Cache">Rakudo::Cache</a>, <a href="https://raku.land/zef:lizmat/App::Rak">App::Rak</a>, <a href="https://raku.land/zef:raku-community-modules/OO::Monitors">OO::Monitors</a>, <a href="https://raku.land/zef:raku-community-modules/OO::Actors">OO::Actors</a>, <a href="https://raku.land/zef:lizmat/Hash2Class">Hash2Class</a>, <a href="https://raku.land/zef:lizmat/hyperize">hyperize</a>, <a href="https://raku.land/zef:lizmat/IRC::Client">IRC::Client</a>, <a href="https://raku.land/zef:lizmat/IRC::Client::Plugin::Rakkable">IRC::Client::Plugin::Rakkable</a>, <a href="https://raku.land/zef:raku-community-modules/Data::TextOrBinary">Data::TextOrBinary</a>, <a href="https://raku.land/zef:lizmat/REPL">REPL</a> by <em>Elizabeth Mattijsen</em>.</li> <li><a href="https://raku.land/zef:martimm/Gnome::N">Gnome::N</a>, <a href="https://raku.land/zef:martimm/Gnome::GObject">Gnome::GObject</a>, <a href="https://raku.land/zef:martimm/Gnome::Gio">Gnome::Gio</a>, <a href="https://raku.land/zef:martimm/Gnome::Gdk4">Gnome::Gdk4</a>, <a href="https://raku.land/zef:martimm/Gnome::GdkPixbuf">Gnome::GdkPixbuf</a>, <a href="https://raku.land/zef:martimm/Gnome::Pango">Gnome::Pango</a>, <a href="https://raku.land/zef:martimm/Gnome::Gsk4">Gnome::Gsk4</a>, <a href="https://raku.land/zef:martimm/Gnome::Gtk4">Gnome::Gtk4</a>, <a href="https://raku.land/zef:martimm/Gnome::Graphene">Gnome::Graphene</a> by <em>Marcel Timmerman</em>.</li> <li><a href="https://raku.land/zef:coke/App::SudokuHelper">App::SudokuHelper</a> by <em>Will Coleda</em>.</li> <li><a href="https://raku.land/zef:bduggan/Map::Leaflet">Map::Leaflet</a>, <a href="https://raku.land/zef:bduggan/Geo::Polyline">Geo::Polyline</a> by <em>Brian Duggan</em>.</li> <li><a href="https://raku.land/zef:melezhik/Sparrow6">Sparrow6</a>, <a href="https://raku.land/zef:melezhik/Tomtit">Tomtit</a>, <a href="https://raku.land/zef:melezhik/Tomty">Tomty</a>, <a href="https://raku.land/zef:melezhik/Sparrowdo">Sparrowdo</a> by <em>Alexey Melezhik</em>.</li> <li><a href="https://raku.land/zef:antononcube/FunctionalParsers">FunctionalParsers</a>, <a href="https://raku.land/zef:antononcube/Math::SpecialFunctions">Math::SpecialFunctions</a>, <a href="https://raku.land/zef:antononcube/Math::NumberTheory">Math::NumberTheory</a> by <em>Anton Antonov</em>.</li> <li><a href="https://raku.land/zef:samy/App::Stouch">App::Stouch</a> by <em>Samuel Young</em>.</li> <li><a href="https://raku.land/zef:demanuel/HTML::EscapeUtils">HTML::EscapeUtils</a>, <a href="https://raku.land/zef:demanuel/App::JSONDiff">App::JSONDiff</a> by <em>demanuel</em>.</li> <li><a href="https://raku.land/zef:dwarring/FontConfig">FontConfig</a> by <em>David Warring</em>.</li> </ul> <h3 class="wp-block-heading"><a href="https://github.com/lizmat/articles/blob/main/weekly-2022.51.md#winding-down"></a>Winding down</h3> <p>Quite a lot of core work this week! And quite a few module updates. It almost feels like spring is already here!</p> <p>Please keep staying safe and healthy, and keep up the good work! Even after week 4 of hopefully only 209.</p> <p> Meanwhile, still: Слава Україні!  Героям слава!</p> <p class="has-small-font-size">If you like what I’m doing, committing to a <a href="https://github.com/sponsors/lizmat/">small sponsorship</a> would mean a great deal!</p> <h2><a href="https://dev.to/lizmat/test-coverage-in-practice-n10">Test coverage in practice</h2></a> <p><h3>Published by Elizabeth Mattijsen on 2025-02-01T12:43:01</h3></p> <blockquote> <p>This is part 4 in the <a href="https://dev.to/lizmat/series/30086">"Towards more coverage"</a> blog series.</p> </blockquote> <p>The <a href="https://dev.to/lizmat/towards-more-coverage-fne">first blog post</a> showed the basic usage of the <a href="https://raku.land/zef:lizmat/Test::Coverage" rel="noopener noreferrer"><code>Test::Coverage</code></a> distribution. But it can do more!</p> <h2> All code covered? </h2> <p>If you have a situation where all of the test in a distribution cover all of the executable code, you can simplify the test file to:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="k">use</span> <span class="nn">Test::</span><span class="nv">Coverage</span><span class="p">;</span> <span class="nv">must</span><span class="o">-</span><span class="nv">be</span><span class="o">-</span><span class="nv">complete</span><span class="p">;</span> </code></pre> </div> <p>This will set the <code>plan</code> to 1 test, and mark it ok if indeed all code is covered.</p> <p>Should that fail for some reason, then the <a href="https://raku.land/zef:lizmat/Test::Coverage#report" rel="noopener noreferrer"><code>report</code></a> and <a href="https://raku.land/zef:lizmat/Test::Coverage#source-with-coverage" rel="noopener noreferrer"><code>source-with-coverage</code></a> subroutines will be called, providing the developer with feedback on which lines were not covered.</p> <p>If you want to be clear to yourself what the goal is for coverage testing, you could mark this subroutine as "todo":<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="k">use</span> <span class="nn">Test::</span><span class="nv">Coverage</span><span class="p">;</span> <span class="nv">todo</span> <span class="p">"</span><span class="s2">needs more tests!</span><span class="p">";</span> <span class="nv">must</span><span class="o">-</span><span class="nv">be</span><span class="o">-</span><span class="nv">complete</span><span class="p">;</span> </code></pre> </div> <p>This will make sure that on the one hand the failure of coverage testing will <strong>not</strong> stop you from uploading a new release with e.g. <a href="https://raku.land/zef:skaji/App::Mi6" rel="noopener noreferrer"><code>App::Mi6</code></a>, but will make it clear to any current or future maintainer that the intent was to cover all code in tests.</p> <p>Personally, I wouldn't do this though. Coverage testing feels too much like bondage to me then, and programming really should be fun!</p> <h2> Are all code paths really covered? </h2> <p>Even if the tests say that all lines of code are covered, does that guarantee that all code paths are covered? Probably not. Why is that?</p> <p>Well, that's because the granularity of code coverage is <strong>source line</strong> based. And a lot can happen in one line of Raku source code! For example, take this simple ternary:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>my $a = <foo frobnicate>.pick; # either "foo" or "frobnicate" my $b = $a eq "foo" ?? "bar" !! "baz"; </code></pre> </div> <p>From a code coverage point of view, it doesn't matter whether <code>$a</code> had the value <code>"foo"</code> or <code>"frobnicate"</code> in the assignment to <code>$b</code>. But for the execution of your code, that might really have different results!</p> <p>One way around that, might be to split that ternary over multiple lines:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="k">my</span> <span class="nv">$a</span> <span class="o">=</span> <span class="o"><</span><span class="nv">foo</span> <span class="nv">frobnicate</span><span class="o">>.</span><span class="nv">pick</span><span class="p">;</span> <span class="c1"># either "foo" or "frobnicate"</span> <span class="k">my</span> <span class="nv">$b</span> <span class="o">=</span> <span class="nv">$a</span> <span class="ow">eq</span> <span class="p">"</span><span class="s2">foo</span><span class="p">"</span> <span class="p">??</span> <span class="p">"</span><span class="s2">bar</span><span class="p">"</span> <span class="o">!!</span> <span class="p">"</span><span class="s2">baz</span><span class="p">"</span> <span class="p">;</span> </code></pre> </div> <p>But that also feels a bit icky: adapting your coding style just to satisfy any coverage testing.</p> <p>It's probably better to realize that 100% coverage testing is still <strong>no</strong> guarantee that all possible code paths are tested. And that you should always remain vigilant with regards to testing your code: find new ways to break it, and make tests to spot such breakage!</p> <h2> Status update </h2> <p>Of the <a href="https://raku.land/zef:lizmat" rel="noopener noreferrer">225 distributions</a> that I am currently personally maintaining, 108 of them now have coverage testing enabled. 32 of them have complete coverage testing. So that's still a lot of modules not being tested, and definitely a lot not fully covered. After the initial push to test the coverage testing itself, I did about 100 of them. That gave me a good idea of all of the corner cases.</p> <p>Now I add coverage testing only if a distribution needs updating. And any new distribution gets coverage testing, of course!</p> <h2> What about RakuAST? </h2> <p>The new <a href="https://raku.land/zef:lizmat/Slang::Lambda" rel="noopener noreferrer"><code>Slang::Lambda</code></a> is a <strong>very</strong> small distribution that allows one to use <code>λ</code> as a synonym for the pointy block starter <code>-></code> in Raku source code. This is <a href="https://github.com/lizmat/Slang-Lambda/blob/main/lib/Slang/Lambda.rakumod" rel="noopener noreferrer">a <em>really</em> small module</a>, so coverage testing should be easy, one would think.</p> <p>The problem is that one piece of code is executed when the legacy grammar is used, and the other piece of code is executed if the new Raku grammar (RakuAST) is used. And one can only be executing code in one or the other grammar!</p> <p>Fortunately, the <code>Test::Coverage</code> distribution allows one to handle that case as well, by being a little more verbose. The test file looks like:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="k">use</span> <span class="nn">Test::</span><span class="nv">Coverage</span><span class="p">;</span> <span class="nv">default</span><span class="o">-</span><span class="nv">coverage</span><span class="o">-</span><span class="nv">setup</span><span class="p">;</span> <span class="nv">run</span><span class="p">;</span> <span class="nv">%*ENV</span><span class="o"><</span><span class="nv">RAKUDO_RAKUAST</span><span class="o">>=</span><span class="mi">1</span><span class="p">;</span> <span class="nv">run</span><span class="p">;</span> <span class="nv">must</span><span class="o">-</span><span class="nv">be</span><span class="o">-</span><span class="nv">complete</span><span class="p">;</span> </code></pre> </div> <p>Note that now we explicitely run the <a href="https://raku.land/zef:lizmat/Test::Coverage#default-coverage-setup" rel="noopener noreferrer"><code>default-coverage-setup</code></a> and <a href="https://raku.land/zef:lizmat/Test::Coverage#run" rel="noopener noreferrer"><code>run</code></a> subroutines (which would normally be run under the hood by the test routines).</p> <p>Just before the second time the <code>run</code> subroutine is called, the <code>RAKUDO_RAKUAST</code> environment variable is set: this will cause the <code>raku</code> executable to use the new Raku grammar when parsing code. In this case, it will mark the RakuAST specific code in the Lambda slang as executed.</p> <p>The call to <code>must-be-complete</code> will then see that all is ok, and mark the test is ok.</p> <h2> Conclusion </h2> <p>Coverage testing will help you determine which parts of the code in a distribution are not tested yet. But is <strong>no</strong> guarantee that really all possible code path have been tested.</p> <p>Adding coverage testing at a later stage is a lot of work. Make sure that you add it during development of a distribution, as you would with any other tests.</p> <p>The <code>Test::Coverage</code> distribution allows for further flexibility in a lot of corner cases as well, not just for the simple, straightforward testing cases.</p> <p>This concludes this series of blog posts for now. New episodes may be added at a later stage: never say never!</p> <p><em>If you like what I'm doing, committing to a <a href="https://github.com/sponsors/lizmat/" rel="noopener noreferrer">small sponsorship</a> would mean a great deal to me!</em></p> <h2><a href="https://dev.to/melezhik/validating-config-files-with-raku-and-sparrow-taskcheck-dsl-1ekp">Validating configuration files with Raku and Sparrow Task::Check DSL</h2></a> <p><h3>Published by Alexey Melezhik on 2025-01-30T08:53:50</h3></p> <p>In real world we have a lot of pure structured or structured in hard to parse formats configuration files:</p> <p><code>/etc/default/nginx</code>, <code>/etc/default/grub</code>, <code>logrotate.conf</code>, <code>sysctl.conf</code> - to name a few. </p> <p>In configuration management it's not only important to maintain specific configurations but also validate those changes to make it sure we don't break things.</p> <p>Sparrow - is an automation tool written on Raku that allow to make such a validation by leveraging Task::Check DSL rules which underneath are just Raku regular expressions.</p> <p>In the rest of this post we take a look at real life example.</p> <h1> Grub </h1> <p>Consider linux GRUB boot loader file <code>/etc/default/grub</code> with following configuration:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>GRUB_DEFAULT=0 #GRUB_HIDDEN_TIMEOUT=0 GRUB_HIDDEN_TIMEOUT_QUIET=true GRUB_TIMEOUT=2 GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian` GRUB_CMDLINE_LINUX_DEFAULT="rootfstype=ext4 quiet splash acpi_osi=" GRUB_CMDLINE_LINUX="" GRUB_ENABLE_CRYPTODISK=true # Uncomment to enable BadRAM filtering, modify to suit your needs # This works with Linux (no patch required) and with any kernel that obtains # the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...) #GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef" # Uncomment to disable graphical terminal (grub-pc only) #GRUB_TERMINAL=console # The resolution used on graphical terminal # note that you can use only modes which your graphic card supports via VBE # you can see them in real GRUB with the command `vbeinfo' #GRUB_GFXMODE=640x480 # Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux #GRUB_DISABLE_LINUX_UUID=true # Uncomment to disable generation of recovery mode menu entries #GRUB_DISABLE_RECOVERY="true" # Uncomment to get a beep at grub start #GRUB_INIT_TUNE="480 440 1" GRUB_DISABLE_OS_PROBER="true" </code></pre> </div> <p>Though configuration format is quite simple, based on VAR=value semantics, it still allows some freedom in specification. </p> <p>For example to enable variables we could use one of many forms - <code>yes/"yes"</code>, <code>true/"true"</code> and just <code>1</code>. Another challenge is to check that some variable are disabled - they might be just commented or “switched off” via negation form, e.g. VAR=0,false,""</p> <p>As rigorous operations we want to validate a couple of things:</p> <ul> <li><p>Boot on crypted disks is supported by setting GRUB_ENABLE_CRYPTODISK to one of the values: <code>1,yes,true</code> and optional quoted form - <code>"yes","true"</code></p></li> <li><p>OS prober capability is disabled or in other words GRUB_DISABLE_OS_PROBER is not set by using any of forms - <code>1,yes,true,"yes","true"</code> </p></li> </ul> <p>Before go parsing let's create simple Raku program (Sparrow task), that dumps out configuration removing empty and commented lines, the whole output will be placed inside <code>>>></code>, <code><<<</code> markers for visibility:</p> <p>task.raku<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="c1">#!raku</span> <span class="k">my</span> <span class="nv">$path</span> <span class="o">=</span> <span class="nv">config</span><span class="p">()</span><span class="o"><</span><span class="nv">path</span><span class="o">></span><span class="p">;</span> <span class="nv">say</span> <span class="p">"</span><span class="s2">validate config {</span><span class="si">$path</span><span class="s2">} ...</span><span class="p">";</span> <span class="nv">say</span> <span class="p">"</span><span class="s2">>>></span><span class="p">";</span> <span class="k">my</span> <span class="nv">@out</span><span class="p">;</span> <span class="k">for</span> <span class="nv">$path</span><span class="o">.</span><span class="nv">IO</span><span class="o">.</span><span class="nv">lines</span> <span class="o">-></span> <span class="nv">$i</span> <span class="p">{</span> <span class="nv">$i</span><span class="o">.</span><span class="nb">chomp</span><span class="p">;</span> <span class="k">next</span> <span class="k">unless</span> <span class="nv">$i</span> <span class="o">~~</span> <span class="sr">/\S+/</span><span class="p">;</span> <span class="nv">@out</span><span class="o">.</span><span class="nb">push</span><span class="p">(</span><span class="nv">$i</span><span class="p">)</span> <span class="k">unless</span> <span class="nv">$i</span> <span class="o">~~</span> <span class="sr">/^ \s* '#'/</span><span class="p">;</span> <span class="p">};</span> <span class="nv">say</span> <span class="nb">sort</span><span class="p">(</span><span class="nv">@out</span><span class="p">)</span><span class="o">.</span><span class="nb">join</span><span class="p">("</span><span class="se">\n</span><span class="p">");</span> <span class="nv">say</span> <span class="p">"</span><span class="s2"><<<</span><span class="p">";</span> </code></pre> </div> <p>Now let's write verification scenario by creating rules in Sparrow::Task::Check DSL format:</p> <p>task.check<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>note: validate config between: {'>>>'} {'<<<'} note: allow to boot on crypted disks regexp: ^^ \s* "GRUB_ENABLE_CRYPTODISK=" (<[y 1 true]> | '"true"' | '"yes"' ) note: never allow to enable os prober !regexp: ^^ \s* "GRUB_DISABLE_OS_PROBER=" (<[y 1 true]> | '"true"' | '"yes"' ) end: </code></pre> </div> <p>The DSL code has there major parts:</p> <ul> <li><code>between: {'>>>'} {'<<<'}</code></li> </ul> <p>Setting a context for a search in between <code>>>></code> ... <code><<<</code> delimiter, this is an example of so called <em>search context modifiers</em>, in this case - Range modifier. </p> <p>In other words this is precaution that we only search within specific boundaries of text output</p> <ul> <li> <code>regexp: ^^ \s* "GRUB_ENABLE_CRYPTODISK=" (<[y 1 true]> | '"true"' | '"yes"' )</code> </li> </ul> <p>Verifying that we have <code>GRUB_ENABLE_CRYPTODISK</code> set to one of values (y,1,true,"true","yes"). </p> <p><code>regexp:</code> marker means Raku regular expression is used for validation.</p> <ul> <li><code>note: never allow to enable os prober !regexp: ^^ \s* "GRUB_DISABLE_OS_PROBER=" (<[y 1 true]> | '"true"' | '"yes"' )</code></li> </ul> <p>Verifying we DON'T(*) have <code>GRUB_ENABLE_CRYPTODISK</code> set to one of values (y,1,true,"true","yes"), in other words that GRUB_ENABLE_CRYPTODISK is disabled. </p> <p>(*) <code>!</code> sign before <code>regexp:</code> marker means negation logic.</p> <p><code>Note:</code> expressions are human readable comments that are showed up in a report to help us correlate every check to business logic</p> <h1> Test report </h1> <p>Now let's run our script and get a result of test. As this is Sparrow scenario, we use <code>s6</code> - Sparrow CLI runner:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code><span class="c">#!bash</span> s6 <span class="nt">--task-run</span> .@path<span class="o">=</span>/etc/default/grub </code></pre> </div> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>14:09:20 :: [sparrowtask] - run sparrow task .@path=/etc/default/grub 14:09:20 :: [sparrowtask] - run thing . [task run: task.raku - .] [task stdout] 14:09:21 :: validate config examples/grub ... 14:09:21 :: >>> 14:09:21 :: GRUB_CMDLINE_LINUX="" 14:09:21 :: GRUB_CMDLINE_LINUX_DEFAULT="rootfstype=ext4 quiet splash acpi_osi=" 14:09:21 :: GRUB_DEFAULT=0 14:09:21 :: GRUB_DISABLE_OS_PROBER="true" 14:09:21 :: GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian` 14:09:21 :: GRUB_ENABLE_CRYPTODISK=true 14:09:21 :: GRUB_HIDDEN_TIMEOUT_QUIET=true 14:09:21 :: GRUB_TIMEOUT=2 14:09:21 :: <<< [task check] # validate config # allow to boot on crypted disks stdout match (r) <^^ \s* "GRUB_ENABLE_CRYPTODISK=" (<[y 1 true]> | '"true"' | '"yes"' )> True # never allow to enable os prober stdout match (r) <!^^ \s* "GRUB_DISABLE_OS_PROBER=" (<[y 1 true]> | '"true"' | '"yes"' )> False ================= TASK CHECK FAIL </code></pre> </div> <ul> <li><em>PS for colorful report please click <a href="https://pasteboard.co/grlUKxSkrGYf.jpg" rel="noopener noreferrer">here</a></em></li> </ul> <p>Expectedly we have following results:</p> <ul> <li>GRUB_ENABLE_CRYPTODISK check passed</li> <li>GRUB_DISABLE_OS_PROBER fails </li> </ul> <p>Pretty impressive results considering such a little amount of code has been written!</p> <p>Sparrow <a href="https://github.com/melezhik/Sparrow6/blob/master/documentation/taskchecks.md" rel="noopener noreferrer">Task::Check</a>s is comprehensive and super flexible tool allowing one to write validation scenarios for arbitrary data formats and complexity, check out documentation pages to know more.</p> <p>That is it. Thanks for reading </p> <h2><a href="https://rakudo.org/post/announce-rakudo-release-2025.01"> Rakudo compiler, Release #179 (2025.01)</h2></a> <p><h3>Published on 2025-01-25T00:00:00</h3></p> <h2><a href="https://dev.to/melezhik/useful-terminal-plugins-to-build-golang-code-gj0">Useful terminal plugins to build golang code</h2></a> <p><h3>Published by Alexey Melezhik on 2025-01-24T09:16:01</h3></p> <p>Recently, I've released 2 new Sparrow plugins to help build Golang code:</p> <ul> <li><a href="https://sparrowhub.io/plugin/go-build/0.000001" rel="noopener noreferrer">go-build</a></li> </ul> <p><em>Simple golang code builder</em></p> <ul> <li><a href="https://sparrowhub.io/plugin/go-format/0.000001" rel="noopener noreferrer">go-format</a></li> </ul> <p><em>Golang code formatter</em></p> <p>They are simple yet useful, providing two essentials tools to build Golang code in terminal:</p> <ul> <li><p>Build code</p></li> <li><p>Format code</p></li> </ul> <p>The rest of the post shows how to use the plugins</p> <h1> Installation </h1> <p>To install plugins we need to use <a href="https://github.com/melezhik/Tomtit" rel="noopener noreferrer">Tomtit</a> task runner with profiles - preset of Raku scenarios that grouped by topics:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code><span class="c">#!bash</span> <span class="nb">cd</span> /to/you/source/code tom <span class="nt">--init</span> <span class="c"># initialize Tomtit for the first time</span> tom <span class="nt">--profile</span> go <span class="c"># install go profile scenarios</span> </code></pre> </div> <p>Now once we've installed <code>go</code> profile, we have an access to <code>go-build</code> and <code>go-format</code> scenarios which are just a Raku wrappers to run mentioned plugins. Code is simple enough and editable to adjust your project specific needs</p> <p>Go-build scenario:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code><span class="c">#!bash</span> tom <span class="nt">--cat</span> go-build </code></pre> </div> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="c1">#!raku</span> <span class="k">my</span> <span class="nv">$path</span> <span class="o">=</span> <span class="p">[</span> <span class="p">"</span><span class="s2">cmd/app/main.go</span><span class="p">",</span> <span class="p">"</span><span class="s2">cmd/cli/cli.go</span><span class="p">"</span> <span class="p">];</span> <span class="nv">task</span><span class="o">-</span><span class="nv">run</span> <span class="p">"</span><span class="s2">build</span><span class="p">",</span> <span class="p">"</span><span class="s2">go-build</span><span class="p">",</span> <span class="o">%</span><span class="p">(</span> <span class="p">:</span><span class="nv">$path</span><span class="p">,</span> <span class="p">);</span> </code></pre> </div> <p>Go-format scenario:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code><span class="c">#!bash</span> tom <span class="nt">--cat</span> go-format </code></pre> </div> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code><span class="c">#!bash</span> task-run <span class="s2">"go format"</span>, <span class="s2">"go-format"</span><span class="p">;</span> </code></pre> </div> <h1> Customization </h1> <p>In case of <code>go-build</code> scenario - adjust <code>path</code> array with files to build for your project specific:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code><span class="c">#!bash</span> <span class="nb">export </span><span class="nv">EDITOR</span><span class="o">=</span>nano tom <span class="nt">--edit</span> go-build </code></pre> </div> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="c1">#!raku</span> <span class="k">my</span> <span class="nv">$path</span> <span class="o">=</span> <span class="p">[</span> <span class="p">"</span><span class="s2">cmd/app/app.go</span><span class="p">",</span> <span class="p">"</span><span class="s2">cmd/app2/app2.go</span><span class="p">",</span> <span class="p">];</span> <span class="c1"># the rest of the code stays the same </span> </code></pre> </div> <h1> Run </h1> <p>To run plugins - use <code>tom</code> cli runner:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>#!bash tom go-build 11:56:26 :: go build cmd/main.go 11:56:27 :: cmd/main.go </code></pre> </div> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>#!bash tom go-format 11:57:18 :: cmd/main.go </code></pre> </div> <p>That is it. </p> <p>Follow plugins <a href="https://sparrowhub.io/search?q=%27go-%27" rel="noopener noreferrer">documentation</a> to get familiar with plugins parameters.</p> <p>Thanks for reading </p> <h2><a href="https://dev.to/melezhik/simple-find-in-source-code-with-tomtit-and-sparrow-312">Simple search in source code with Tomtit and Sparrow</h2></a> <p><h3>Published by Alexey Melezhik on 2025-01-23T20:21:22</h3></p> <p>Recently, I've released a simple Sparrow plugin called <a href="https://sparrowhub.io/plugin/find/0.000007" rel="noopener noreferrer">find</a> to search text in source code, which is based on find and grep core linux utilities.</p> <p>The rest of the document shows how to perform search in terminal by using <a href="https://github.com/melezhik/Tomtit" rel="noopener noreferrer">Tomtit</a> task runner.</p> <h1> Installation </h1> <p>To install the plugin, we need to use Tomtit task runner with profiles - preset of Raku scenarios that grouped by topics:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code><span class="c">#!bash</span> <span class="nb">cd</span> /to/you/source/code tom <span class="nt">--init</span> <span class="c"># initialize Tomtit for the first time</span> tom <span class="nt">--profile</span> code <span class="c"># install code profile scenarios</span> </code></pre> </div> <p>Now once we've installed <code>code</code> profile scenarios (for now it's just a single one), we have an access to <code>search</code> scenario which is just a Raku wrapper to do search with <code>find</code> plugin:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code><span class="c">#!bash</span> tom <span class="nt">--cat</span> search </code></pre> </div> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="c1">#!raku</span> <span class="k">my</span> <span class="nv">$ext</span> <span class="o">=</span> <span class="nv">prompt</span><span class="p">("</span><span class="s2">ext (go): </span><span class="p">");</span> <span class="nv">$ext</span> <span class="o">=</span> <span class="p">"</span><span class="s2">go</span><span class="p">"</span> <span class="k">unless</span> <span class="nv">$ext</span><span class="p">;</span> <span class="k">my</span> <span class="nv">$search1</span> <span class="o">=</span> <span class="nv">prompt</span><span class="p">("</span><span class="s2">search1: </span><span class="p">");</span> <span class="k">my</span> <span class="nv">$search2</span> <span class="o">=</span> <span class="nv">prompt</span><span class="p">("</span><span class="s2">search2: </span><span class="p">");</span> <span class="k">my</span> <span class="nv">$exclude</span> <span class="o">=</span> <span class="nv">prompt</span><span class="p">("</span><span class="s2">exclude: </span><span class="p">");</span> <span class="nv">say</span> <span class="p">"</span><span class="s2">find [</span><span class="si">$search1</span><span class="s2">] [</span><span class="si">$search2</span><span class="s2">] !</span><span class="si">$exclude</span><span class="s2"> in </span><span class="si">$ext</span><span class="p">";</span> <span class="nv">task</span><span class="o">-</span><span class="nv">run</span> <span class="p">"</span><span class="s2">find </span><span class="si">$search1</span><span class="s2"> </span><span class="si">$search2</span><span class="s2"> in </span><span class="si">$ext</span><span class="p">",</span> <span class="p">"</span><span class="s2">find</span><span class="p">",</span> <span class="o">%</span><span class="p">(</span> <span class="p">:</span><span class="nv">$ext</span><span class="p">,</span> <span class="p">:</span><span class="nv">$search1</span><span class="p">,</span> <span class="s">search2</span> <span class="o">=></span> <span class="nv">$search2</span> <span class="o">||</span> <span class="p">"",</span> <span class="s">exclude</span> <span class="o">=></span> <span class="nv">$exclude</span> <span class="o">||</span> <span class="p">"",</span> <span class="p">);</span> </code></pre> </div> <p>The logic is following:</p> <p>‘search1’ filters results out by first string and then ‘search2’ filter is applied for final results, ‘ext’ defines files extension to search.</p> <h1> Customization </h1> <p>Stub code shipped with Tomtit profile is good enough, we only need to change it a bit, to make default choice for file extenstion suitable for Raku project:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code><span class="c">#!bash</span> <span class="nb">export </span><span class="nv">EDITOR</span><span class="o">=</span>nano tom <span class="nt">--edit</span> search </code></pre> </div> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="c1">#!raku</span> <span class="k">my</span> <span class="nv">$ext</span> <span class="o">=</span> <span class="nv">prompt</span><span class="p">("</span><span class="s2">ext (rakumod): </span><span class="p">");</span> <span class="nv">$ext</span> <span class="o">=</span> <span class="p">"</span><span class="s2">rakumod</span><span class="p">"</span> <span class="k">unless</span> <span class="nv">$ext</span><span class="p">;</span> <span class="c1"># ... the rest of the code is the same</span> </code></pre> </div> <h1> Search </h1> <p>Now let's search files in my <a href="https://github.com/melezhik/Tomtit" rel="noopener noreferrer">Tomtit</a> repository. Say, I want to find all exported functions.<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code><span class="c">#!bash</span> tom search </code></pre> </div> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>load configuration from /home/melezhik/projects/Tomtit/.tom/env/config.raku ext (rakumod): search1: sub search2: is export exclude: find [sub] [is export] ! in rakumod 23:09:06 :: find [sub] [is export] [!] in *.rakumod 23:09:06 :: === 23:09:06 :: find . -name *.rakumod -exec grep --color -H 'sub' {} \; | grep 'is export' 23:09:06 :: case2 23:09:06 :: ./lib/Tomtit/Completion.rakumod:sub complete () is export { 23:09:06 :: ./lib/Tomtit.rakumod:our sub check-if-init ( $dir ) is export { 23:09:06 :: ./lib/Tomtit.rakumod:our sub init ($dir) is export { 23:09:06 :: ./lib/Tomtit.rakumod:our sub load-conf () is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub tomtit-usage () is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub tomtit-help () is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub tomtit-clean ($dir) is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub scenario-last ($dir) is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub scenario-run ($dir,$scenario,%args?) is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub scenario-remove ($dir,$scenario) is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub scenario-cat ($dir,$scenario,%args?) is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub scenario-edit ($dir,$scenario) is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub environment-edit ($dir,$env) is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub environment-list ($dir) is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub environment-set ($dir,$env) is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub environment-show ($dir) is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub environment-cat ($dir,$env,%args?) is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub scenario-doc ($dir,$scenario) is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub scenario-list ($dir) is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub scenario-list-print ($dir) is export { 23:09:06 :: ./lib/Tomtit.rakumod:multi sub profile-list is export { 23:09:06 :: ./lib/Tomtit.rakumod:multi sub profile-list($dir,$profile is copy) is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub profile-install ($dir, $profile is copy, %args?) is export { 23:09:06 :: ./lib/Tomtit.rakumod:sub completion-install () is export { </code></pre> </div> <p>That is it. Follow plugin <a href="https://sparrowhub.io/plugin/find/0.000007" rel="noopener noreferrer">documentation</a> to get familiar with plugin parameters.</p> <p>Thanks for reading</p> <h2><a href="https://dev.to/lizmat/creating-coverage-information-4ke6">Creating coverage information</h2></a> <p><h3>Published by Elizabeth Mattijsen on 2025-01-22T17:52:41</h3></p> <blockquote> <p>This is part 3 in the <a href="https://dev.to/lizmat/series/30086">"Towards more coverage"</a> blog series.</p> </blockquote> <p>The first blog introduced the new <a href="https://raku.land/zef:lizmat/Test::Coverage" rel="noopener noreferrer"><code>Test::Coverage</code></a> module that allows a developer to test how well the tests of a distribution actually check the code of a distribtion. The second blog delved into the development process of making the <a href="https://raku.land/zef:lizmat/Code::Coverable" rel="noopener noreferrer"><code>Code::Coverable</code></a> distribution that provides the logic to the determine the lines in a source-file that <em>can</em> be covered.</p> <h2> Code::Coverage </h2> <p>With the <code>Code::Coverable</code> module now being operational, it was time to package this up in a module that would actually create coverage information.</p> <p>And that became <a href="https://raku.land/zef:lizmat/Code::Coverage" rel="noopener noreferrer"><code>Code::Coverage</code></a> module. And in its simplest use, it looks like this:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="k">use</span> <span class="nn">Code::</span><span class="nv">Coverage</span><span class="p">;</span> <span class="k">my</span> <span class="nv">$coverage</span> <span class="o">=</span> <span class="nn">Code::</span><span class="nv">Coverage</span><span class="o">.</span><span class="k">new</span><span class="p">(</span> <span class="s">targets</span> <span class="o">=></span> <span class="nv">@targets</span><span class="p">,</span> <span class="s">runners</span> <span class="o">=></span> <span class="nv">@test</span><span class="o">-</span><span class="nv">scripts</span> <span class="p">);</span> <span class="nv">$coverage</span><span class="o">.</span><span class="nv">run</span><span class="p">;</span> <span class="nv">say</span> <span class="o">.</span><span class="nv">key</span> <span class="o">~</span> <span class="p">"</span><span class="s2">: </span><span class="p">"</span> <span class="o">~</span> <span class="o">.</span><span class="nv">value</span> <span class="k">for</span> <span class="nv">$coverage</span><span class="o">.</span><span class="nv">missed</span><span class="p">;</span> </code></pre> </div> <p>At object instantiation, provide two named arguments.</p> <p>The first one is called <code>targets</code> and should contain the <code>use</code> targets for which to provide coverage information. These use targets are usually the <code>key</code> of the <code>Code::Coverable</code> object returned by the <code>coverables</code> subroutine of the <code>Code::Coverable</code> distribution.</p> <blockquote> <p>In the <code>Test::Coverage</code> case, this would be obtained from the "provides" section from the <code>META6.json</code> file of a distribution.</p> </blockquote> <p>The second one is called <code>runners</code>, and it should contain the path(s) of the scripts that should be executed to create coverage information.</p> <blockquote> <p>In the <code>Test::Coverage</code> that would be set up with all the <code>.t</code> and <code>.rakutest</code> files in the <code>t</code> and <code>xt</code> directories of the distribution.</p> </blockquote> <p>Then, call the <code>run</code> method on the <code>Code::Coverage</code> object to execute the scripts and create the coverage information, scan and process it.</p> <p>After that, you can call one of several report methods, such as <code>missed</code>, which will then report the line numbers of the code that was <strong>not</strong> covered.</p> <h2> Multiple runs </h2> <p>Sometimes you may want to run the same script multiple times, but with different arguments, to get complete coverage information. And you can! Just call the <code>run</code> method once again, and specify any command line arguments as arguments to <code>run</code> method:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="nv">$coverage</span><span class="o">.</span><span class="nv">run</span><span class="p">("</span><span class="s2">foo</span><span class="p">");</span> <span class="nv">$coverage</span><span class="o">.</span><span class="nv">run</span><span class="p">("</span><span class="s2">bar</span><span class="p">");</span> </code></pre> </div> <p>After each call, the coverage information is updated and you can call any of the report methods again.</p> <p>If the different code paths depend on environment variables, you should just set those in the <code>%*ENV</code> variable:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="nv">%*ENV</span><span class="o"><</span><span class="nv">FROBNICATE</span><span class="o">>=</span><span class="mi">1</span> <span class="nv">$coverage</span><span class="o">.</span><span class="nv">run</span><span class="p">;</span> <span class="c1"># run with frobnication</span> </code></pre> </div> <h2> Annotations </h2> <p>It is always nice to know the line numbers of code that didn't get covered. But that would still be a lot of looking up and down the code to find out what the contents was of the lines that didn't get covered.</p> <p>To make that process easier, the <code>Code::Coverage</code> object has an <code>annotated</code> method that will produce the source code of a <code>use</code> target, and prefix each line of source with one of these four possibilities:</p> <ul> <li>"<code>*</code>" - line was coverable, and covered</li> <li>"<code>✱</code>" - line was <strong>not</strong> coverable, but covered anyway</li> <li>"<code>x</code>" - line was coverable and <strong>not</strong> covered</li> <li>"<code></code>" - line was not coverable</li> </ul> <p>Looks familiar? It should be if you've read the first post of this blog series! Yes, that's indeed the logic that <code>Test::Coverage</code> uses to create the <code>.rakucov</code> files.</p> <h2> More control </h2> <p>The <a href="https://raku.land/zef:lizmat/Code::Coverage#new" rel="noopener noreferrer"><code>new</code></a> method allows for more customization / configuration of the coverage process, such as an option to specify where (temporary) coverage files are to be stored, and whether they should be removed or not after processing.</p> <p>It also allows you to inspect the output that the scripts produced in each call to the <code>run</code> method (both STDOUT and STDERR), if you would like to find out what the scripts actually did.</p> <h2> Conclusion </h2> <p>As you may have gathered: the <code>Code::Coverable</code> and <code>Code::Coverage</code> modules are the workhorses of the <code>Test::Coverage</code> module, which is basically a thin layer around these modules. I find that to be a good design principle: have a "backend" (computer) and a "frontend" (human interaction) functionality. Or in "git" terms: separated in "plumbing" and "porcelain".</p> <p><em>If you like what I'm doing, committing to a <a href="https://github.com/sponsors/lizmat/" rel="noopener noreferrer">small sponsorship</a> would mean a great deal to me!</em></p> <h2><a href="https://dev.to/melezhik/developing-a-simple-sparrow-plugin-with-raku-and-bash-51g0">Developing a simple Sparrow plugin with Raku and Bash</h2></a> <p><h3>Published by Alexey Melezhik on 2025-01-20T14:27:08</h3></p> <p>Sparrow is very efficient in writing scripts to automate system management using Raku and Bash, in today's post I am going to show an example ...</p> <h2> Deploy configuration file and restart service upon changes </h2> <p>Let's create a simple plugin that enable / disable some configuration flags in service configuration file and if any changes occur reload a service. Format of flags in configuration file is following:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>var=true|false </code></pre> </div> <p>Here is an example of configuration file:</p> <p><code>config.txt</code><br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>debug=true sentry=false tls=false </code></pre> </div> <p>Usually services are managed as systemd units, so <code>service reload</code> could be used to reload service configuration.</p> <p>——</p> <p>Let’s get started and create root folder for the plugin:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code><span class="c">#!bash</span> <span class="nb">mkdir</span> <span class="nt">-p</span> service-config-reload <span class="nb">cd </span>service-config-reload </code></pre> </div> <p>Now inside plugin root directory:</p> <p><code>hook.raku</code>:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="c1">#!raku</span> <span class="k">my</span> <span class="nv">$cfg</span><span class="o">-</span><span class="nv">orig</span> <span class="o">=</span> <span class="nv">config</span><span class="p">()</span><span class="o"><</span><span class="nv">path</span><span class="o">>.</span><span class="nv">IO</span><span class="o">.</span><span class="nv">slurp</span><span class="p">();</span> <span class="k">my</span> <span class="nv">$cfg</span> <span class="o">=</span> <span class="nv">$cfg</span><span class="o">-</span><span class="nv">orig</span><span class="p">;</span> <span class="k">for</span> <span class="nv">config</span><span class="p">()</span><span class="o"><</span><span class="nv">enable</span><span class="o">><></span> <span class="o">-></span> <span class="nv">$f</span> <span class="p">{</span> <span class="nv">$cfg</span> <span class="o">~~</span> <span class="sr">s/<?wb> "$f=" \S+ /$f=true/</span><span class="p">;</span> <span class="p">}</span> <span class="k">for</span> <span class="nv">config</span><span class="p">()</span><span class="o"><</span><span class="nv">disable</span><span class="o">><></span> <span class="o">-></span> <span class="nv">$f</span> <span class="p">{</span> <span class="nv">$cfg</span> <span class="o">~~</span> <span class="sr">s/<?wb> "$f=" \S+ /$f=false/</span><span class="p">;</span> <span class="p">}</span> <span class="nv">set_stdout</span><span class="p">(</span><span class="nv">$cfg</span><span class="p">);</span> <span class="k">if</span> <span class="nv">$cfg</span> <span class="ow">ne</span> <span class="nv">$cfg</span><span class="o">-</span><span class="nv">orig</span> <span class="p">{</span> <span class="nv">set_stdout</span><span class="p">("</span><span class="s2">config changed</span><span class="p">");</span> <span class="nv">config</span><span class="p">()</span><span class="o"><</span><span class="nv">path</span><span class="o">>.</span><span class="nv">IO</span><span class="o">.</span><span class="nv">spurt</span><span class="p">(</span><span class="nv">$cfg</span><span class="p">);</span> <span class="nv">set_stdout</span><span class="p">("</span><span class="s2">config updated</span><span class="p">");</span> <span class="nv">run_task</span> <span class="p">"</span><span class="s2">service_restart</span><span class="p">",</span> <span class="o">%</span><span class="p">(</span> <span class="s">service</span> <span class="o">=></span> <span class="nv">config</span><span class="p">()</span><span class="o"><</span><span class="nv">service</span><span class="o">></span><span class="p">,</span> <span class="s">path</span> <span class="o">=></span> <span class="nv">config</span><span class="p">()</span><span class="o"><</span><span class="nv">path</span><span class="o">></span><span class="p">,</span> <span class="p">);</span> <span class="p">}</span> </code></pre> </div> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>mkdir -p tasks/service_restart/ </code></pre> </div> <p><code>tasks/service_restart/task.bash</code><br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code><span class="c">#!bash</span> <span class="nb">sudo </span>service <span class="nv">$service</span> reload </code></pre> </div> <p>In real life applications services might have a linter capability where before configuration is applied it's checked and if any error occur application of changes is terminated:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code><span class="c">#!bash</span> <span class="nb">set</span> <span class="nt">-e</span> <span class="nb">sudo</span> /usr/bin/<span class="nv">$service</span> <span class="nt">--check</span> <span class="nv">$path</span> <span class="nb">sudo </span>service <span class="nv">$service</span> reload </code></pre> </div> <h1> Publish plugin </h1> <p>Packaging scenario as a plugin will allow users to reuse it as a library.</p> <p><code>sparrow.json</code><br> </p> <div class="highlight js-code-highlight"> <pre class="highlight json"><code><span class="p">{</span><span class="w"> </span><span class="nl">"name"</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="s2">"service-config-reload"</span><span class="p">,</span><span class="w"> </span><span class="nl">"description"</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="s2">"Deploy configuration file and restart service upon changes"</span><span class="p">,</span><span class="w"> </span><span class="nl">"version"</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="s2">"0.0.1"</span><span class="p">,</span><span class="w"> </span><span class="nl">"category"</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="s2">"linux"</span><span class="p">,</span><span class="w"> </span><span class="nl">"url"</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="s2">"https://github.com/melezhik/sparrow-plugins/tree/master/service-config-reload"</span><span class="w"> </span><span class="p">}</span><span class="w"> </span></code></pre> </div> <p>This last command will upload plugin to local Sparrow repository (see this <a href="https://github.com/melezhik/Sparrow6/blob/master/documentation/repository.md#create-repository" rel="noopener noreferrer">doc</a> for more details)<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code><span class="c">#!bash</span> s6 <span class="nt">--upload</span> </code></pre> </div> <h2> Use of plugins </h2> <p>Anywhere in Raku scenario, just do this:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight perl"><code><span class="c1">#!raku</span> <span class="k">use</span> <span class="nn">Sparrow6::</span><span class="nv">DSL</span><span class="p">;</span> <span class="nv">task</span><span class="o">-</span><span class="nv">run</span> <span class="p">"</span><span class="s2">apply and reload</span><span class="p">",</span> <span class="p">"</span><span class="s2">service-config-reload</span><span class="p">",</span> <span class="o">%</span><span class="p">(</span> <span class="p">:</span><span class="nv">path</span><span class="o"><</span><span class="nv">config</span><span class="o">.</span><span class="nv">txt</span><span class="o">></span><span class="p">,</span> <span class="p">:</span><span class="nv">service</span><span class="o"><</span><span class="nv">app</span><span class="o">></span><span class="p">,</span> <span class="s">enable</span> <span class="o">=></span> <span class="o"><</span><span class="nv">tls</span> <span class="nv">sentry</span><span class="o">></span><span class="p">,</span> <span class="c1"># enable TLS and use of Sentry for production</span> <span class="s">disable</span> <span class="o">=></span> <span class="o"><</span><span class="nv">debug</span><span class="o">></span><span class="p">,</span> <span class="c1"># disable debug mode for production </span> <span class="p">)</span> </code></pre> </div> <p>Thanks for reading!</p> <h1> PS </h1> <p>Source code of the plugin could be found here - <a href="https://github.com/melezhik/sparrow-plugins/tree/master/service-config-reload" rel="noopener noreferrer">https://github.com/melezhik/sparrow-plugins/tree/master/service-config-reload</a></p> <h2><a href="https://rakuforprediction.wordpress.com/2025/01/03/numeric-properties-of-2025/">Numeric properties of 2025</h2></a> <p><h3>Published by Anton Antonov Antonov on 2025-01-04T03:08:19</h3></p> <h2 class="wp-block-heading"><a></a>Introduction</h2> <p>This blog post demonstrates many of the mathematical properties for the number <code>2025</code> given in the Wolfram notebook <a href="https://community.wolfram.com/groups/-/m/t/3347182">“Happy 2025 == 1³+2³+3³+4³+5³+6³+7³+8³+9³ !”</a>, [EPn1], by <a href="https://en.wikipedia.org/wiki/Ed_Pegg_Jr.">Ed Pegg Jr</a>.</p> <p>We cannot computationally demonstrate <em>easily</em> all of the properties in Raku; for the full exposition of them see [EPn1].</p> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Most impressive</h2> <p>First, let us demonstrate the most impressive numeric property of <code>2025</code>:</p> <p>2025 = 1<sup>3</sup> + 2<sup>3</sup> + 3<sup>3</sup> + 4<sup>3</sup> + 5<sup>3</sup> + 6<sup>3</sup> + 7<sup>3</sup> + 8<sup>3</sup> + 9<sup>3</sup> = (1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9)<sup>2</sup></p> <p>This comes from the following identify, often referred to as <a href="https://mathworld.wolfram.com/FaulhabersFormula.html">Faulhaber’s Formula or Nicomachus’s Theorem</a>:</p> <p>∑<sub>n</sub><sup>k=1</sup> k<sup>3</sup> = (∑<sub>n</sub><sup>k=1</sup> k)<sup>2</sup></p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> say "Sum of cubes of 1..9 : ", [+] (1..9)>>³; say "Sum of 1..9 squared : ", ((1..9).sum)²; </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Sum of cubes of 1..9 : 2025 # Sum of 1..9 squared : 2025 </pre></div> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Simple</h2> <p>Next, let us demonstrate the simple numeric properties of <code>2025</code>.</p> <p><code>2025</code> is obtained by the square of the sum of (i) the number of the first two digits, <code>20</code>, with (ii) the number of the 3rd and 4th digits, <code>25</code>:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> (20+25) ** 2 </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # 2025 </pre></div> <p>An odd property of the square root, <code>45 == 2025.sqrt</code>, is that it is the smallest integer where the periodic part of the reciprocal is <code>2</code>:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> use Rat::Precise; (1/45).precise(40) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # 0.0222222222222222222222222222222222222222 </pre></div> <p><code>2025</code> is the denominator for this sum of squares:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> (1/9² + 1/5²).nude </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # (106 2025) </pre></div> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Divisors-based</h2> <p><code>2025</code> is the product of greatest common divisors of <code>15</code> for numbers less than <code>15</code>:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> [*] (1..14).map(15 gcd *) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # 2025 </pre></div> <p><code>2025</code> is the product of the proper divisors of <code>45</code>:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> use Math::NumberTheory; my @proper = divisors(45).head(*-1) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # [1 3 5 9 15] </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> [*] |@proper </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # 2025 </pre></div> <p>The package <a href="https://raku.land/zef:raku-community-modules/Math::Sequences">“Math::Sequences”</a> has the functions <a href="https://en.wikipedia.org/wiki/Divisor_function"><code>sigma</code></a> (aka σ) and <a href="https://en.wikipedia.org/wiki/Euler%27s_totient_function"><code>totient</code></a> (aka φ, not to be confused with the <a href="https://en.wikipedia.org/wiki/Golden_ratio">Golden ratio</a> φ) that — in principle — can be used to demonstrate the <em><strong>rare property</strong></em>:</p> <p>σ<sub>1</sub>(φ(2025<sup>3</sup>)) = φ(σ<sub>1</sub>(2025<sup>3</sup>))</p> <p>But those package functions are too slow. Instead, we are using a <a href="https://raku.land/zef:antononcube/Jupyter::Chatbook" target="_blank" rel="noreferrer noopener">Raku-chatbook</a> <a href="https://www.wolframalpha.com/">Wolfram|Alpha</a> cell, [<a href="https://rakuforprediction.wordpress.com/2024/05/18/chatbook-new-magic-cells/">AA1</a>, <a href="https://rakuforprediction.wordpress.com/2024/05/06/wwwwolframalpha/">AA2</a>], to verify that formula:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% wolfram-alpha DivisorSigma[1, EulerPhi[2025^3]] == EulerPhi[DivisorSigma[1, 2025^3]] </pre></div> <figure class="wp-block-image size-large is-resized"><img data-attachment-id="1360" data-permalink="https://rakuforprediction.wordpress.com/2025/01/03/numeric-properties-of-2025/wolfram-alpha-rare-property-for-2025/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/wolfram-alpha-rare-property-for-2025.png" data-orig-size="646,352" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Wolfram-Alpha-rare-property-for-2025" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/wolfram-alpha-rare-property-for-2025.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/wolfram-alpha-rare-property-for-2025.png?w=646" loading="lazy" width="646" height="352" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/wolfram-alpha-rare-property-for-2025.png?w=646" alt="" class="wp-image-1360" style="width:471px;height:auto" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/wolfram-alpha-rare-property-for-2025.png 646w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/wolfram-alpha-rare-property-for-2025.png?w=150 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/wolfram-alpha-rare-property-for-2025.png?w=300 300w" sizes="(max-width: 646px) 100vw, 646px" /></figure> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Phi number system</h2> <p>Digits of <code>2025</code> represented in the <a href="https://mathworld.wolfram.com/PhiNumberSystem.html">Phi number system</a>:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my @res = phi-number-system(2025); </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # [15 13 10 5 3 1 -6 -11 -16] </pre></div> <p>Verification:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> @res.map({ ϕ ** $_ }).sum.round(10e-11); </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # 2025 </pre></div> <p><strong>Remark:</strong> We have to round (using a small multiple of <code>10</code>) because of the approximation of the Golden ratio used in “Math::NumberTheory”.</p> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Good permutations</h2> <p>From [EPn1]:</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <p>There are also 2025 “good” permutations (<a href="https://oeis.org/A006717">A006717</a>), where all rotations have a single number in the correct place.</p> </blockquote> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> goodPermutations = Select[Permutations[Range[9]],Union[Table[Count[RotateRight[#,k]-Range[9],0],{k,0,8}]]=={1}&]; goodPermutations//Length&nbsp; </pre></div> <p>Here is the corresponding Raku code:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my @good-permutations = [1..9].permutations.race(:4degree).grep( -> @p { my @res = (^9).map( -> $k { (@p.rotate(-$k) <<->> (1..9)).grep(0).elems }).unique.sort; @res eqv [1] }); @good-permutations.elems </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # 2025 </pre></div> <p>Optimization of the code above <a href="https://gist.github.com/timo/f8480b00d88104afe93876c267840c80">suggested by @timo</a>:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my @good-permutations = [1..9].permutations.race(:4degree).grep( -> @p { my @res = (^9).map( -> $k { (@p.rotate(-$k) >>-<< (1..9)).grep(0).elems }).unique.head(2); @res.elems == 1 && @res[0] == 1 }); @good-permutations.elems </pre></div> <p><strong>Remark:</strong> This is an embarrassingly parallel computation for which the order of the results does not matter. Sequential-computation-wise, Wolfram Language is ≈12 times faster than Raku’s first “good permutations” finding code, and ≈2.5 times faster than the second. (On a few years old laptop.)</p> <p>Here are the first of the “good” permutations:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html @good-permutations.head ==> -> @p { (^9).map( -> $k { @p.rotate(-$k) }) }() ==> { $_[(1, 9, 2, 7, 6, 8, 4, 3, 5) <<->> 1]}() ==> { .map(1..9)Z=>*)».Hash.Array }() ==> to-dataset() ==> to-html(field-names => (1..9)».Str) </pre></div> <figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th></tr></thead><tbody><tr><td>1</td><td>3</td><td>2</td><td>7</td><td>9</td><td>8</td><td>4</td><td>6</td><td>5</td></tr><tr><td>3</td><td>2</td><td>7</td><td>9</td><td>8</td><td>4</td><td>6</td><td>5</td><td>1</td></tr><tr><td>5</td><td>1</td><td>3</td><td>2</td><td>7</td><td>9</td><td>8</td><td>4</td><td>6</td></tr><tr><td>7</td><td>9</td><td>8</td><td>4</td><td>6</td><td>5</td><td>1</td><td>3</td><td>2</td></tr><tr><td>9</td><td>8</td><td>4</td><td>6</td><td>5</td><td>1</td><td>3</td><td>2</td><td>7</td></tr><tr><td>2</td><td>7</td><td>9</td><td>8</td><td>4</td><td>6</td><td>5</td><td>1</td><td>3</td></tr><tr><td>4</td><td>6</td><td>5</td><td>1</td><td>3</td><td>2</td><td>7</td><td>9</td><td>8</td></tr><tr><td>6</td><td>5</td><td>1</td><td>3</td><td>2</td><td>7</td><td>9</td><td>8</td><td>4</td></tr><tr><td>8</td><td>4</td><td>6</td><td>5</td><td>1</td><td>3</td><td>2</td><td>7</td><td>9</td></tr></tbody></table></figure> <p><strong>Remark:</strong> The diagonal of the table is with the digits from 1 to 9. Also, observe the “shifts” between the consecutive rows above.</p> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>References</h2> <p>[AA1] Anton Antonov, <a href="https://rakuforprediction.wordpress.com/2024/05/18/chatbook-new-magic-cells/">“Chatbook New Magic Cells”</a>, (2024), <a href="https://rakuforprediction.wordpress.com/">RakuForPrediction at WordPress</a>.</p> <p>[AA2] Anton Antonov, <a href="https://rakuforprediction.wordpress.com/2024/05/06/wwwwolframalpha/">“WWW::WolframAlpha”</a>, (2024), <a href="https://rakuforprediction.wordpress.com/">RakuForPrediction at WordPress</a>.</p> <p>[EPn1] Ed Pegg, <a href="https://community.wolfram.com/groups/-/m/t/3347182">“Happy 2025 =1³+2³+3³+4³+5³+6³+7³+8³+9³!”</a>, Wolfram Community, STAFFPICKS, December 30, 2024.</p> <h2><a href="https://rakuforprediction.wordpress.com/2025/01/01/doomsday-clock-parsing-and-plotting/">Doomsday clock parsing and plotting</h2></a> <p><h3>Published by Anton Antonov Antonov on 2025-01-01T20:47:48</h3></p> <h2 class="wp-block-heading"><a></a>Introduction</h2> <p><a href="https://thebulletin.org/doomsday-clock/">The Doomsday Clock</a> is a symbolic timepiece maintained by the <a href="https://thebulletin.org/">Bulletin of the Atomic Scientists (BAS)</a> since 1947. It represents how close humanity is perceived to be to global catastrophe, primarily <a href="https://en.wikipedia.org/wiki/Nuclear_warfare">nuclear war</a> but also including climate change and biological threats. The clock’s hands are set annually to reflect the current state of global security; midnight signifies theoretical doomsday.</p> <p>In this post(<a href="https://github.com/antononcube/RakuForPrediction-blog/blob/main/Notebooks/Jupyter/Doomsday-clock-parsing-and-plotting.ipynb" target="_blank" rel="noreferrer noopener">notebook</a>) we consider two tasks:</p> <ul class="wp-block-list"> <li><strong>Parsing of Doomsday Clock reading statements</strong> <ul class="wp-block-list"> <li>Using both <a href="https://raku.land/zef:antononcube/FunctionalParsers" target="_blank" rel="noreferrer noopener">Functional Parsers (FP)</a> (aka <a href="https://en.wikipedia.org/wiki/Parser_combinator" target="_blank" rel="noreferrer noopener">“parser combinators”</a>), [AAp1], and Large Language Models (LLMs). <ul class="wp-block-list"> <li>We take text data from the past announcements, and extract the Doomsday Clock reading statements.</li> </ul> </li> </ul> </li> <li><strong>Evolution of Doomsday Clock times</strong> <ul class="wp-block-list"> <li>We extract relevant Doomsday Clock timeline data from the corresponding Wikipedia page. <ul class="wp-block-list"> <li>(Instead of using a page from BAS.)</li> </ul> </li> <li>We show how timeline data from that Wikipedia page can be processed with LLMs.</li> <li>The result plot shows the evolution of the minutes to midnight. <ul class="wp-block-list"> <li>The plot could show trends, highlighting significant global events that influenced the clock setting.</li> <li>Hence, we put in informative callouts and tooltips.</li> </ul> </li> </ul> </li> </ul> <p>The data extraction and visualization in the post (notebook) serve educational purposes or provide insights into historical trends of global threats as perceived by experts. We try to make the ingestion and processing code universal and robust, suitable for multiple evaluations now or in the (near) future.</p> <p><strong>Remark:</strong> Keep in mind that the Doomsday Clock is a metaphor and its settings are not just data points but reflections of complex global dynamics (by <a href="https://thebulletin.org/2024/01/press-release-doomsday-clock-remains-at-90-seconds-to-midnight/">certain experts</a> and a <a href="https://thebulletin.org/about-us/board-of-sponsors/">board of sponsors</a>.)</p> <p><strong>Remark:</strong> Currently (2024-12-30) Doomsday Clock is set at <a href="https://thebulletin.org/doomsday-clock/">90 seconds before midnight</a>.</p> <p><strong>Remark:</strong> This post (notebook) is the Raku-version of the Wolfram Language (WL) <a href="https://community.wolfram.com/groups/-/m/t/3347065">notebook with the same name</a>, [AAn1]. That is why the “standard” Raku-grammar approach is not used. (Although, in the preliminary versions of this work relevant Raku grammars were <em>generated</em> via both LLMs and Raku packages.)</p> <p>I was very impressed by the looks and tune-ability of WL’s <a href="https://reference.wolfram.com/language/ref/ClockGauge.html.en"><code>ClockGauge</code></a>, so, I programmed a similar clock gauge in Raku’s package <a href="https://raku.land/zef:antononcube/JavaScript::D3">“JavaScript::D3”</a> (which is based on <a href="https://d3js.org/">D3.js</a>.)</p> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Setup</h2> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> use LLM::Functions; use LLM::Prompts; use LLM::Configurations; use Text::SubParsers; use Data::Translators; use Data::TypeSystem; use Data::Importers; use Data::Reshapers; use Hash::Merge; use FunctionalParsers :ALL; use FunctionalParsers::EBNF; use Math::DistanceFunctions::Edit; use Lingua::NumericWordForms; </pre></div> <h3 class="wp-block-heading"><a></a>JavaScript::D3</h3> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $background = 'none'; my $stroke-color = 'Ivory'; my $fill-color = 'none'; </pre></div> <h3 class="wp-block-heading"><a></a>JavaScript::Google::Charts</h3> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $format = 'html'; my $titleTextStyle = { color => 'Ivory' }; my $backgroundColor = '#1F1F1F'; my $legendTextStyle = { color => 'Silver' }; my $legend = { position => "none", textStyle => {fontSize => 14, color => 'Silver'} }; my $hAxis = { title => 'x', titleTextStyle => { color => 'Silver' }, textStyle => { color => 'Gray'}, logScale => False, format => 'scientific'}; my $vAxis = { title => 'y', titleTextStyle => { color => 'Silver' }, textStyle => { color => 'Gray'}, logScale => False, format => 'scientific'}; my $annotations = {textStyle => {color => 'Silver', fontSize => 10}}; my $chartArea = {left => 50, right => 50, top => 50, bottom => 50, width => '90%', height => '90%'}; my $background = '1F1F1F'; </pre></div> <h3 class="wp-block-heading"><a></a>Functional parsers</h3> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my sub parsing-test-table(&parser, @phrases) { my @field-names = ['statement', 'parser output']; my @res = @phrases.map({ @field-names Z=> [$_, &parser($_.words).raku] })».Hash.Array; to-html(@res, :@field-names) } </pre></div> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Data ingestion</h2> <p>Here we ingest the Doomsday Clock timeline page and show corresponding statistics:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $url = "https://thebulletin.org/doomsday-clock/past-announcements/"; my $txtEN = data-import($url, "plaintext"); text-stats($txtEN) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # (chars => 73722 words => 11573 lines => 756) </pre></div> <p>By observing the (plain) text of that page we see the Doomsday Clock time setting can be extracted from the sentence(s) that begin with the following phrase:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $start-phrase = 'Bulletin of the Atomic Scientists'; my $sentence = $txtEN.lines.first({ / ^ $start-phrase /}) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Bulletin of the Atomic Scientists, with a clock reading 90 seconds to midnight </pre></div> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Grammar and parsers</h2> <p>Here is a grammar in <a href="https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form">Extended Backus-Naur Form (EBNF)</a> for parsing Doomsday Clock statements:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $ebnf = q:to/END/; <TOP> = <clock-reading> ; <clock-reading> = <opening> , ( <minutes> | [ <minutes> , [ 'and' | ',' ] ] , <seconds> ) , 'to' , 'midnight' ; <opening> = [ { <any> } ] , 'clock' , [ 'is' ] , 'reading' ; <any> = '_String' ; <minutes> = <integer> <& ( 'minute' | 'minutes' ) ; <seconds> = <integer> <& ( 'second' | 'seconds' ) ; <integer> = '_Integer' <@ &{ $_.Int } ; END text-stats($ebnf) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # (chars => 364 words => 76 lines => 6) </pre></div> <p><strong>Remark:</strong> The EBNF grammar above can be obtained with LLMs using a suitable prompt with example sentences. (We do not discuss that approach further in this notebook.)</p> <p>Here the parsing functions are generated from the EBNF string above:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my @defs = fp-ebnf-parse($ebnf, <CODE>, name => 'Doomed2', actions => 'Raku::Code').head.tail; .say for @defs.reverse </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # my &pINTEGER = apply(&{ $_.Int }, symbol('_Integer')); # my &pSECONDS = sequence-pick-left(&pINTEGER, (alternatives(symbol('second'), symbol('seconds')))); # my &pMINUTES = sequence-pick-left(&pINTEGER, (alternatives(symbol('minute'), symbol('minutes')))); # my &pANY = symbol('_String'); # my &pOPENING = sequence(option(many(&pANY)), sequence(symbol('clock'), sequence(option(symbol('is')), symbol('reading')))); # my &pCLOCK-READING = sequence(&pOPENING, sequence((alternatives(&pMINUTES, sequence(option(sequence(&pMINUTES, option(alternatives(symbol('and'), symbol(','))))), &pSECONDS))), sequence(symbol('to'), symbol('midnight')))); # my &pTOP = &pCLOCK-READING; </pre></div> <p><strong>Remark:</strong> The function <code>fb-ebnf-parse</code> has a <a href="https://github.com/antononcube/Raku-FunctionalParsers/blob/main/doc/Parser-code-generation.md">variety of actions</a> for generating code from EBNF strings. For example, with <code>actions => 'Raku::Class'</code> the generation above would produce a class, which might be more convenient to do further development with (via inheritance or direct changes.)</p> <p>Here the imperative code above — assigned to <code>@defs</code> — is re-written using the infix form of the parser combinators:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my &pINTEGER = satisfy({ $_ ~~ /\d+/ }) «o {.Int}; my &pMINUTES = &pINTEGER «& (symbol('minute') «|» symbol('minutes')) «o { [minute => $_,] }; my &pSECONDS = &pINTEGER «& (symbol('second') «|» symbol('seconds')) «o { [second => $_,] }; my &pANY = satisfy({ $_ ~~ /\w+/ }); my &pOPENING = option(many(&pANY)) «&» symbol('clock') «&» option(symbol('is')) «&» symbol('reading'); my &pCLOCK-READING = &pOPENING «&» (&pMINUTES «|» option(&pMINUTES «&» option(symbol('and') «|» symbol(','))) «&» &pSECONDS) «&» symbol('to') «&» symbol('midnight'); my &pTOP = &pCLOCK-READING; </pre></div> <p>We must redefine the parser <code>pANY</code> (corresponding to the EBNF rule “<any>”) in order to prevent <code>pANY</code> of gobbling the word “clock” and in that way making the parser <code>pOPENING</code> fail.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> &pANY = satisfy({ $_ ne 'clock' && $_ ~~ /\w+/}); </pre></div> <p>Here are random sentences generated with the grammar:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> .say for fp-random-sentence($ebnf, 12).sort; </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # clock reading 681 minutes to midnight # clock reading 788 minutes to midnight # clock is reading 584 seconds to midnight # clock is reading 721 second to midnight # clock is reading 229 minute and 631 second to midnight # clock is reading 458 minutes to midnight # clock is reading 727 minute to midnight # F3V; clock is reading 431 minute to midnight # FXK<GQ 3RJJJ clock is reading 369 seconds to midnight # NRP FNSEE K0EQO OPE clock is reading 101 minute to midnight # QJDV; R<K7S; JMQ>HD AA31 clock is reading 369 minute 871 second to midnight # QKQGK FZJ@BB M8C1BD BPI;C: clock reading 45 minute 925 second to midnight </pre></div> <p>Verifications of the (sub-)parsers:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> "90 seconds".words.&pSECONDS </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # ((() [second => 90])) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> "That doomsday clock is reading".words.&pOPENING </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # ((() (((((That doomsday)) clock) (is)) reading))) </pre></div> <p>Here the “top” parser is applied:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $str = "the doomsday clock is reading 90 seconds to midnight"; $str.words.&pTOP </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # ((() ((((((((the doomsday)) clock) (is)) reading) (() [second => 90])) to) midnight))) </pre></div> <p>Here the sentence extracted above is parsed and interpreted into an association with keys “Minutes” and “Seconds”:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> $sentence.words.&pTOP.tail.flat.grep(* ~~ Pair) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # (second => 90) </pre></div> <p>Let us redefine <code>pCLOCK-READING</code> to return a minutes-&-seconds dictionary, <code>pTOP</code> to return a corresponding date-time:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> &pCLOCK-READING = &pCLOCK-READING «o { $_.flat.grep(* ~~ Pair).Hash }; &pTOP = &pCLOCK-READING «o { Date.today.DateTime.earlier(seconds => ($_<minute> // 0) * 60 + ($_<second>// 0) ) }; </pre></div> <p>Here we assign and show the results of those two parsers:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $doom-reading = $sentence.words.&pCLOCK-READING.head.tail; my $doom-time = $sentence.words.&pTOP.head.tail; .say for (:$doom-reading, :$doom-time) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # doom-reading => {second => 90} # doom-time => 2024-12-31T23:58:30Z </pre></div> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Plotting the clock</h2> <p>Using the interpretation derived above plot the corresponding clock with <code>js-dr-clock-gauge</code>:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% js js-d3-clock-gauge($doom-time) </pre></div> <figure class="wp-block-image size-large"><img data-attachment-id="1334" data-permalink="https://rakuforprediction.wordpress.com/2025/01/01/doomsday-clock-parsing-and-plotting/doomsday-clock-as-clock-gauge-light/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-as-clock-gauge-light.png" data-orig-size="245,242" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Doomsday-clock-as-clock-gauge-light" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-as-clock-gauge-light.png?w=245" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-as-clock-gauge-light.png?w=245" loading="lazy" width="245" height="242" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-as-clock-gauge-light.png?w=245" alt="" class="wp-image-1334" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-as-clock-gauge-light.png 245w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-as-clock-gauge-light.png?w=150 150w" sizes="(max-width: 245px) 100vw, 245px" /></figure> <p>Let us define a map with clock-gauge plot options.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my @scale-ranges = (0, 0.01 ... 0.66).rotor(2=>-1).map({ ([0, 60], $_) }); my @scale-ranges2 = (0, 0.01 ... 0.82).rotor(2=>-1).map({ ([0, 60], $_) }); my %opts = background => 'none', stroke-color => 'Black', stroke-width => 0, title-color => 'Ivory', title-font-family => 'Helvetica', hour-hand-color => 'Orange', second-hand-color => 'Red', color-scheme => 'Magma', fill-color => 'AntiqueWhite', :@scale-ranges, color-scheme-interpolation-range => (0.11, 0.95), margins => {top => 60, left => 20, right => 20, bottom => 60}, height => 420, gauge-labels => {Doomsday => [0.5, 0.35], 'clock' => [0.5 ,0.28]}, gauge-labels-color => 'DarkSlateGray', gauge-labels-font-family => 'Krungthep', ; %opts.elems </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # 16 </pre></div> <p>Here are different “doomsday clock” examples:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% js [ {color-scheme => 'Plasma', fill-color => 'MistyRose', gauge-labels-color => 'Orchid'}, {color-scheme => 'Spectral', fill-color => '#4e65ac', stroke-color => 'DarkRed', stroke-width => 10, gauge-labels => %()}, {color-scheme => 'Cividis', fill-color => 'DarkSlateGray', gauge-labels => {Doomsday => [0.5, 0.6], 'clock' => [0.5 ,0.36]}, scale-ranges => @scale-ranges2}, ].map({ js-d3-clock-gauge(:23hour, :58minute, :30second, |merge-hash(%opts.clone, $_, :!deep)) }).join("\n") </pre></div> <figure class="wp-block-image size-large"><a href="https://raw.githubusercontent.com/antononcube/RakuForPrediction-blog/refs/heads/main/Articles/Diagrams/Doomsday-clock-parsing-and-plotting/Doomsday-clock-examples-dark-3.png" target="_blank" rel=" noreferrer noopener"><img data-attachment-id="1335" data-permalink="https://rakuforprediction.wordpress.com/2025/01/01/doomsday-clock-parsing-and-plotting/doomsday-clock-examples-dark-3/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-examples-dark-3.png" data-orig-size="1522,512" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Doomsday-clock-examples-dark-3" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-examples-dark-3.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-examples-dark-3.png?w=1008" loading="lazy" width="1024" height="344" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-examples-dark-3.png?w=1024" alt="" class="wp-image-1335" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-examples-dark-3.png?w=1024 1024w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-examples-dark-3.png?w=150 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-examples-dark-3.png?w=300 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-examples-dark-3.png?w=768 768w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-examples-dark-3.png 1522w" sizes="(max-width: 1024px) 100vw, 1024px" /></a></figure> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>More robust parsing</h2> <p>More robust parsing of Doomsday Clock statements can be obtained in these three ways:</p> <ul class="wp-block-list"> <li>“Fuzzy” match of words <ul class="wp-block-list"> <li>For misspellings like “doomsdat” instead of “doomsday.”</li> </ul> </li> <li>Parsing of numeric word forms. <ul class="wp-block-list"> <li>For statements, like, “two minutes and twenty five seconds.”</li> </ul> </li> <li>Delegating the parsing to LLMs when grammar parsing fails.</li> </ul> <h3 class="wp-block-heading"><a></a>Fuzzy matching</h3> <p>The parser <code>satisfy</code> can be used to handle misspellings (via, say, <code>edit-distance</code> from <a href="https://raku.land/zef:antononcube/Math::DistanceFunctions">“Math::DistanceFunctions”</a>):</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html my &pDD = satisfy({ edit-distance($_, "doomsday") ≤ 2 }) «o {"doomsday"}; my @phrases = "doomsdat", "doomsday", "dumzday"; parsing-test-table(&pDD, @phrases) </pre></div> <figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>statement</th><th>parser output</th></tr></thead><tbody><tr><td>doomsdat</td><td>(((), “doomsday”),).Seq</td></tr><tr><td>doomsday</td><td>(((), “doomsday”),).Seq</td></tr><tr><td>dumzday</td><td>().Seq</td></tr></tbody></table></figure> <p>But since “FunctionalParsers” provides the generic parser <code>fuzzy-symbol</code> (that takes a word and a distance as arguments) we use that parser below.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html my &pDD2 = fuzzy-symbol("doomsday", 2); my @phrases = "doomsdat", "doomsday", "dumzday"; parsing-test-table(&pDD2, @phrases) </pre></div> <figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>statement</th><th>parser output</th></tr></thead><tbody><tr><td>doomsdat</td><td>(((), “doomsday”),)</td></tr><tr><td>doomsday</td><td>(((), “doomsday”),)</td></tr><tr><td>dumzday</td><td>()</td></tr></tbody></table></figure> <p>In order to include the misspelling handling into the grammar we manually rewrite the grammar. (The grammar is small, so, it is not that hard to do.)</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my &pINTEGER = satisfy({ $_ ~~ /\d+/ }) «o {.Int}; my &pMINUTES = &pINTEGER «& (fuzzy-symbol('minute', 2) «|» fuzzy-symbol('minutes', 2)) «o { [minute => $_,] }; my &pSECONDS = &pINTEGER «& (fuzzy-symbol('second', 2) «|» fuzzy-symbol('seconds', 2)) «o { [second => $_,] }; my &pANY = satisfy({ edit-distance($_, 'clock') > 2 && $_ ~~ /\w+/ }); my &pOPENING = option(many(&pANY)) «&» fuzzy-symbol('clock', 1) «&» option(symbol('is')) «&» fuzzy-symbol('reading', 2); my &pCLOCK-READING = &pOPENING «&» (&pMINUTES «|» option(&pMINUTES «&» option(symbol('and') «|» symbol(','))) «&» &pSECONDS) «&» symbol('to') «&» fuzzy-symbol('midnight', 2); &pCLOCK-READING = &pCLOCK-READING «o { $_.flat.grep(* ~~ Pair).Hash }; &pTOP = &pCLOCK-READING «o { Date.today.DateTime.earlier(seconds => ($_<minute> // 0) * 60 + ($_<second>// 0) ) }; </pre></div> <p>Here is a verification table with correct- and incorrect spellings:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html my @phrases = "doomsday clock is reading 2 seconds to midnight", "dooms day cloc is readding 2 minute and 22 sekonds to mildnight"; parsing-test-table(shortest(&pCLOCK-READING), @phrases) </pre></div> <figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>statement</th><th>parser output</th></tr></thead><tbody><tr><td>doomsday clock is reading 2 seconds to midnight</td><td>(((), {:second(2)}),)</td></tr><tr><td>dooms day cloc is readding 2 minute and 22 sekonds to mildnight</td><td>(((), {:minute(2), :second(22)}),)</td></tr></tbody></table></figure> <h3 class="wp-block-heading"><a></a>Parsing of numeric word forms</h3> <p>One way to make the parsing more robust is to implement the ability to parse integer names (or numeric word forms) not just integers.</p> <p><strong>Remark:</strong> For a fuller discussion — and code — of numeric word forms parsing see the tech note <a href="https://resources.wolframcloud.com/PacletRepository/resources/AntonAntonov/FunctionalParsers/tutorial/Integernamesparsing.html">“Integer names parsing”</a> of the paclet <a href="https://resources.wolframcloud.com/PacletRepository/resources/AntonAntonov/FunctionalParsers/">“FunctionalParsers”</a>, [AAp1].</p> <p>First, we make an association that connects integer names with corresponding integer values:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my %worded-values = (^100).map({ to-numeric-word-form($_) => $_ }); %worded-values.elems </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # 100 </pre></div> <p><strong>Remark:</strong> The function <code>to-numeric-word-form</code> is provided by <a href="https://raku.land/zef:antononcube/Lingua::NumericWordForms">“Lingua::NumericWordForms”</a>, [AAp3].</p> <p>Here is how the rules look like:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> %worded-values.pick(6) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # (ninety four => 94 forty three => 43 ninety eight => 98 seventy three => 73 ninety two => 92 eleven => 11) </pre></div> <p>Here we program the integer names parser:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my &pUpTo10 = alternatives( |(^10)».&to-numeric-word-form.map({ symbol($_.trim) }) ); my &p10s = alternatives( |(10, 20 ... 90)».&to-numeric-word-form.map({ symbol($_.trim) }) ); my &pWordedInteger = (&p10s «&» &pUpTo10 «|» &p10s «|» &pUpTo10) «o { %worded-values{$_.flat.join(' ')} }; </pre></div> <p>Here is a verification table of that parser:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html my @phrases = "three", "fifty seven", "thirti one"; parsing-test-table(&pWordedInteger, @phrases) </pre></div> <figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>statement</th><th>parser output</th></tr></thead><tbody><tr><td>three</td><td>(((), 3),).Seq</td></tr><tr><td>fifty seven</td><td>(((), 57), ((“seven”,), 50)).Seq</td></tr><tr><td>thirti one</td><td>().Seq</td></tr></tbody></table></figure> <p>There are two parsing results for “fifty seven”, because <code>&pWordedInteger</code> is defined with:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> &p10s «|» &pUpTo10 «|» p10s ... </pre></div> <p>This can be remedied by using <code>just</code> or <code>shortest</code>:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html parsing-test-table( just(&pWordedInteger), @phrases) </pre></div> <figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>statement</th><th>parser output</th></tr></thead><tbody><tr><td>three</td><td>(“((), 3),).Seq</td></tr><tr><td>fifty seven</td><td>(“((), 57),).Seq</td></tr><tr><td>thirti one</td><td>().Seq</td></tr></tbody></table></figure> <p>Let us change <code>&pINTEGER</code> to parse both integers and integer names:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html &pINTEGER = &satisfy({ $_ ~~ /\d+/ }) «o {.Int} «|» &pWordedInteger; my @phrases = "12", "3", "three", "forty five"; parsing-test-table( just(&pINTEGER), @phrases) </pre></div> <figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>statement</th><th>parser output</th></tr></thead><tbody><tr><td>12</td><td>($((), 12),).Seq</td></tr><tr><td>3</td><td>($((), 3),).Seq</td></tr><tr><td>three</td><td>($((), 3),).Seq</td></tr><tr><td>forty five</td><td>($((), 45),).Seq</td></tr></tbody></table></figure> <p><strong>Remark:</strong> <code>&pINTEGER</code> has to be evaluated before the definitions of the rest of the parsers programmed in the previous subsection.</p> <p>Let us try the new parser using integer names for the clock time:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $str = "the doomsday clock is reading two minutes and forty five seconds to midnight"; $str.words ==> take-first(&pCLOCK-READING)() </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # ((() {minute => 2, second => 45})) </pre></div> <h3 class="wp-block-heading"><a></a>Enhance with LLM parsing</h3> <p>There are multiple ways to employ LLMs for extracting “clock readings” from arbitrary statements for Doomsday Clock readings, readouts, and measures. Here we use <a href="https://en.wikipedia.org/wiki/Prompt_engineering#Other_techniques">LLM few-shot training</a>:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my &flop = llm-example-function([ "the doomsday clock is reading two minutes and forty five seconds to midnight" => '{"minute":2, "second": 45}', "the clock of the doomsday gives 92 seconds to midnight" => '{"minute":0, "second": 92}', "The bulletin atomic scientist maybe is set to a minute an 3 seconds." => '{"minute":1, "second": 3}' ], e => $conf4o, form => sub-parser('JSON') ) </pre></div> <p>Here is an example invocation:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> &flop("Maybe the doomsday watch is at 23:58:03") </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # {minute => 1, second => 57} </pre></div> <p>The following function combines the parsing with the grammar and the LLM example function — the latter is used for fallback parsing:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my sub get-clock-reading(Str:D $st) { my $op = just(&pCLOCK-READING)($st.words); my %h = $op.elems > 0 && $op.head.head.elems == 0 ?? $op.head.tail !! &flop( $st ); return Date.today.DateTime.earlier(seconds => (%h<minute> // 0) * 60 + (%h<second> // 0) ) } </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # &get-clock-reading </pre></div> <h3 class="wp-block-heading"><a></a>Robust parser demo</h3> <p>Here is the application of the combine function above over a certain “random” Doomsday Clock statement:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $s = "You know, sort of, that dooms-day watch is 1 and half minute be... before the big boom. (Of doom...)"; $s.&get-clock-reading </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # 2024-12-31T23:58:30Z </pre></div> <p><strong>Remark:</strong> The same type of robust grammar-and-LLM combination is explained in more detail in the video <a href="https://www.youtube.com/watch?v=QOsVTCQZq_s&t=3700s">“Robust LLM pipelines (Mathematica, Python, Raku)”</a>, [AAv1]. (See, also, the corresponding notebook [AAn1].)</p> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Timeline</h2> <p>In this section we extract Doomsday Clock timeline data and make a corresponding plot.</p> <h3 class="wp-block-heading"><a></a>Parsing page data</h3> <p>Instead of using the <a href="https://thebulletin.org/doomsday-clock/timeline/">official Doomsday clock timeline page</a> we use <a href="https://en.wikipedia.org/wiki/Doomsday_Clock">Wikipedia</a>.<br>We can extract the Doomsday Clock timeline using LLMs. Here we get the plaintext of the Wikipedia page and show statistics:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $url = "https://en.wikipedia.org/wiki/Doomsday_Clock"; my $txtWk = data-import($url, "plaintext"); text-stats($txtWk) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # (chars => 42728 words => 6231 lines => 853) </pre></div> <p>Here we get the Doomsday Clock timeline table from that page in JSON format using an LLM (or ingest a <a href="https://github.com/antononcube/RakuForPrediction-blog/blob/main/Data/doomsday-clock-timeline-table.csv" target="_blank" rel="noreferrer noopener">previous extraction saved as a CSV file</a>):</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $res; if False { $res = llm-synthesize([ "Give the time table of the doomsday clock as a time series that is a JSON array.", "Each element of the array is a dictionary with keys 'Year', 'MinutesToMidnight', 'Time', 'Summary', 'Description'.", "Do not shorten or summarize the descriptions -- use their full texts.", "The column 'Summary' should have summaries of the descriptions, each summary no more than 10 words.", $txtWk, llm-prompt("NothingElse")("JSON") ], e => $conf4o, form => sub-parser('JSON'):drop ); } else { my @field-names = <Year MinutesToMidnight Time Summary Description>; my $url = 'https://raw.githubusercontent.com/antononcube/RakuForPrediction-blog/refs/heads/main/Data/doomsday-clock-timeline-table.csv'; $res = data-import($url, headers => 'auto'); $res = $res.map({ my %h = $_.clone; %h<Year> = %h<Year>.Int; %h<MinutesToMidnight> = %h<MinutesToMidnight>.Num; %h }).Array } deduce-type($res) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Vector(Struct([Description, MinutesToMidnight, Summary, Time, Year], [Str, Num, Str, Str, Int]), 26) </pre></div> <p>Here the LLM result is tabulated:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html my @field-names = <Year MinutesToMidnight Time Summary Description>; $res ==> to-html(:@field-names, align => 'left') </pre></div> <figure class="wp-block-image size-large"><img data-attachment-id="1336" data-permalink="https://rakuforprediction.wordpress.com/2025/01/01/doomsday-clock-parsing-and-plotting/doomsday-clock-timeline-table-light/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-timeline-table-light.png" data-orig-size="766,744" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Doomsday-clock-timeline-table-light" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-timeline-table-light.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-timeline-table-light.png?w=766" loading="lazy" width="766" height="744" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-timeline-table-light.png?w=766" alt="" class="wp-image-1336" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-timeline-table-light.png 766w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-timeline-table-light.png?w=150 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-timeline-table-light.png?w=300 300w" sizes="(max-width: 766px) 100vw, 766px" /></figure> <p><strong>Remark:</strong> The LLM derived summaries in the table above are based on the descriptions in the column “Reason” in the Wikipedia data table.<br>The tooltips of the plot below use the summaries.</p> <h3 class="wp-block-heading"><a></a>Timeline plot</h3> <p>In order to have informative Doomsday Clock evolution plot we obtain and partition dataset’s time series into step-function pairs:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my @dsDoomsdayTimes = |$res; my @ts0 = @dsDoomsdayTimes.map({ <Year MinutesToMidnight role:tooltip> Z=> $_<Year MinutesToMidnight Summary> })».Hash; my @ts1 = @dsDoomsdayTimes.rotor(2=>-1).map({[ %( <Year MinutesToMidnight mark role:tooltip> Z=> $_.head<Year MinutesToMidnight MinutesToMidnight Summary>), %( <Year MinutesToMidnight mark role:tooltip> Z=> [$_.tail<Year>, $_.head<MinutesToMidnight>, NaN, '']) ]}).map(*.Slip); @ts1 = @ts1.push( merge-hash(@ts0.tail, {mark => @ts0.tail<MinutesToMidnight>}) ); deduce-type(@ts1):tally </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Vector(Struct([MinutesToMidnight, Year, mark, role:tooltip], [Num, Int, Num, Str]), 51) </pre></div> <p>Here are added callout annotations indicating the year and the minutes before midnight:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my @ts2 = @ts1.map({ my %h = $_.clone; my $s = ($_<MinutesToMidnight> * 60) mod 60; $s = $s > 0 ?? " {$s}s" !! ''; if %h<mark> === NaN { %h<role:annotation> = ''; } else { %h<role:annotation> = "{%h<Year>}: {floor($_<MinutesToMidnight>)}m" ~ $s; } %h }); deduce-type(@ts2):tally </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Vector(Struct([MinutesToMidnight, Year, mark, role:annotation, role:tooltip], [Num, Int, Num, Str, Str]), 51) </pre></div> <p>Finally, here is the plot:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html js-google-charts('ComboChart', @ts2, column-names => <Year MinutesToMidnight mark role:annotation role:tooltip>, width => 1200, height => 500, title => "Doomsday clock: minutes to midnight, {@dsDoomsdayTimes.map(*<Year>).Array.&{ (.min, .max).join('-') }}", series => { 0 => {type => 'line', lineWidth => 4, color => 'DarkOrange'}, 1 => {type => 'scatter', pointSize => 10, opacity => 0.1, color => 'Blue'}, }, hAxis => { title => 'Year', format => '####', titleTextStyle => { color => 'Silver' }, textStyle => { color => 'Gray'}, viewWindow => { min => 1945, max => 2026} }, vAxes => { 0 => { title => 'Minutes to Midnight', titleTextStyle => { color => 'Silver' }, textStyle => { color => 'Gray'} }, 1 => { titleTextStyle => { color => 'Silver' }, textStyle => { color => 'Gray'}, ticks => (^18).map({ [ v => $_, f => ($_ ?? "23::{60-$_}" !! '00:00' ) ] }).Array } }, :$annotations, :$titleTextStyle, :$backgroundColor, :$legend, :$chartArea, :$format, div-id => 'DoomsdayClock', :!png-button ) </pre></div> <figure class="wp-block-image size-large"><img data-attachment-id="1337" data-permalink="https://rakuforprediction.wordpress.com/2025/01/01/doomsday-clock-parsing-and-plotting/doomsday-clock-timeline-plot-light-tooltip/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-timeline-plot-light-tooltip.png" data-orig-size="1414,600" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Doomsday-clock-timeline-plot-light-tooltip" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-timeline-plot-light-tooltip.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-timeline-plot-light-tooltip.png?w=1008" loading="lazy" width="1024" height="434" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-timeline-plot-light-tooltip.png?w=1024" alt="" class="wp-image-1337" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-timeline-plot-light-tooltip.png?w=1024 1024w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-timeline-plot-light-tooltip.png?w=150 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-timeline-plot-light-tooltip.png?w=300 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-timeline-plot-light-tooltip.png?w=768 768w, https://rakuforprediction.wordpress.com/wp-content/uploads/2025/01/doomsday-clock-timeline-plot-light-tooltip.png 1414w" sizes="(max-width: 1024px) 100vw, 1024px" /></figure> <p><strong>Remark:</strong> The plot <em>should be</em> piecewise constant — simple linear interpolation between the blue points would suggest gradual change of the clock times.</p> <p><strong>Remark:</strong> By hovering with the mouse over the blue points the corresponding descriptions can be seen. We considered using clock-gauges as tooltips, but showing clock-settings reasons is more informative.</p> <p><strong>Remark:</strong> The plot was intentionally made to resemble the <a href="https://en.wikipedia.org/wiki/Doomsday_Clock#Timeline">timeline plot in Doomsday Clock’s Wikipedia page</a>.</p> <p><strong>Remark:</strong> The plot has deficiencies:</p> <ul class="wp-block-list"> <li>Tooltips with arbitrary width <ul class="wp-block-list"> <li>This can be remedied with the (complicated) HTML tooltip procedure described in [AA1]. <ul class="wp-block-list"> <li>But I decided to just make the LLM data extraction to produce short summaries of the descriptions.</li> </ul> </li> </ul> </li> <li>No right vertical axis ticks <ul class="wp-block-list"> <li>The Doomsday Clock timeline plot in Wikipedia and its reproduction in [AAn1] have the “HH::MM” time axis.</li> </ul> </li> </ul> <p>I gave up smoothing out those deficiencies after attempting to fix or address each of them a few times. (It is not that important to figure out Google Charts interface settings for that kind of plots.)</p> <h2 class="wp-block-heading"><a></a>Conclusion</h2> <p>As expected, parsing, plotting, or otherwise processing the Doomsday Clock settings and statements are excellent didactic subjects for textual analysis (or parsing) and temporal data visualization. The visualization could serve educational purposes or provide insights into historical trends of global threats as perceived by experts. (Remember, the clock’s settings are not just data points but reflections of complex global dynamics.)</p> <p>One possible application of the code in this notebook is to make a “web service“ that gives clock images with Doomsday Clock readings. For example, click on this button:</p> <ul class="wp-block-list"> <li><a href="https://www.wolframcloud.com/obj/e28834c9-42a4-4991-9889-243fa39a795d"><strong>Get Doomsday Clock!</strong></a></li> </ul> <h2 class="wp-block-heading"><a></a>References</h2> <h3 class="wp-block-heading"><a></a>Articles, notebooks</h3> <p>[AA1] Anton Antonov, <a href="https://rakuforprediction.wordpress.com/2024/06/18/geographic-data-in-raku-demo/">“Geographic Data in Raku Demo”</a>, (2024), <a href="https://rakuforprediction.wordpress.com/2024/06/18/geographic-data-in-raku-demo/">RakuForPrediction at WordPress</a>.</p> <p>[AAn1] Anton Antonov, <a href="https://community.wolfram.com/groups/-/m/t/3347065">“Doomsday clock parsing and plotting”</a>, (2024), <a href="https://community.wolfram.com/">Wolfram Community</a>.</p> <p>[AAn2] Anton Antonov, <a href="https://community.wolfram.com/groups/-/m/t/3279791">“Making robust LLM computational pipelines from software engineering perspective”</a>, (2024), <a href="https://community.wolfram.com/">Wolfram Community</a>.</p> <h3 class="wp-block-heading"><a></a>Paclets</h3> <p>[AAp1] Anton Antonov, <a href="https://github.com/antononcube/Raku-FunctionalParsers">FunctionalParsers Raku package</a>, (2023-2024), <a href="https://github.com/antononcube">GitHub/antononcube</a>.</p> <p>[AAp2] Anton Antonov, <a href="https://resources.wolframcloud.com/PacletRepository/resources/AntonAntonov/FunctionalParsers/">“FunctionalParsers”</a>, (2023), <a href="https://resources.wolframcloud.com/PacletRepository/">Wolfram Language Paclet Repository</a>.</p> <p>[AAp3] Anton Antonov, <a href="https://github.com/antononcube/Raku-Lingua-NumericWordForms">Lingua::NumericWordForms Raku package</a>, (2021-2024), <a href="https://github.com/antononcube">GitHub/antononcube</a>.</p> <h3 class="wp-block-heading"><a></a>Videos</h3> <p>[AAv1] Anton Antonov, <a href="https://youtu.be/QOsVTCQZq_s">“Robust LLM pipelines (Mathematica, Python, Raku)”</a>, (2024), <a href="https://www.youtube.com/@AAA4prediction">YouTube/@AAA4prediction</a>.</p> <h2><a href="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/">Sparse Matrix Neat Examples in Raku</h2></a> <p><h3>Published by Anton Antonov Antonov on 2024-12-28T16:32:58</h3></p> <h2 class="wp-block-heading"><a></a>Introduction</h2> <p>Sparse matrices are an essential tool in computational mathematics, allowing us to efficiently represent and manipulate large matrices that are predominantly composed of zero elements. In this blog post, we will delve into a few intriguing examples of sparse matrix utilization, specifically in the Raku programming language.</p> <figure class="wp-block-image"><a href="https://youtu.be/kQo3wpiUu6w" target="_blank" rel=" noreferrer noopener"><img data-attachment-id="1323" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/sparse-matrix-neat-examples-thumbnail-small/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/sparse-matrix-neat-examples-thumbnail-small.png" data-orig-size="1000,563" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Sparse-matrix-neat-examples-thumbnail-small" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/sparse-matrix-neat-examples-thumbnail-small.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/sparse-matrix-neat-examples-thumbnail-small.png?w=1000" loading="lazy" width="1000" height="563" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/sparse-matrix-neat-examples-thumbnail-small.png" alt="" class="wp-image-1323" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/sparse-matrix-neat-examples-thumbnail-small.png 1000w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/sparse-matrix-neat-examples-thumbnail-small.png?w=150&h=84 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/sparse-matrix-neat-examples-thumbnail-small.png?w=300&h=169 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/sparse-matrix-neat-examples-thumbnail-small.png?w=768&h=432 768w" sizes="(max-width: 1000px) 100vw, 1000px" /></a></figure> <h3 class="wp-block-heading"><a></a>Examples Covered:</h3> <ol class="wp-block-list"> <li><strong>Random Graph:</strong> <ul class="wp-block-list"> <li>We will explore the adjacency matrix of a random graph generated from a model of social interactions.</li> <li>Additionally, we will overlay adjacency matrices with a shortest path within the graph.</li> </ul> </li> <li><strong>Movie-Actor Bipartite Graph:</strong> <ul class="wp-block-list"> <li>This example involves ingesting data about relationships between actors and movies.</li> <li>We will demonstrate how sparse matrix algebra can facilitate certain information retrieval tasks.</li> </ul> </li> <li><strong>Sparse Matrices Visualization:</strong> <ul class="wp-block-list"> <li>We will discuss techniques for visualizing sparse matrices.</li> </ul> </li> </ol> <p>Support for sparse matrix linear algebra is a hallmark of a mature computational system. Here’s a brief timeline of when some popular systems introduced sparse matrices:</p> <figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>Language</th><th>Initial Introduction</th><th>Confirmed Update</th></tr></thead><tbody><tr><td>MATLAB</td><td>1992</td><td>~</td></tr><tr><td>Mathematica / Wolfram Language</td><td>2003</td><td>updated 2007</td></tr><tr><td>Python</td><td>maybe since 2004</td><td>updated 2006</td></tr><tr><td>R</td><td>maybe since 2011</td><td>updated 2014</td></tr></tbody></table></figure> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Setup</h2> <p>(<em>This setup is similar to the one used in the <a href="https://rakuforprediction.wordpress.com/2024/12/01/graph-neat-examples-in-raku-set-1/">Graph neat examples</a>.</em>)</p> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Random Graph Matrix</h2> <p>Let’s begin by examining a random graph generated using the Watts-Strogatz model. This model is particularly useful for simulating social networks.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% js my $gl = Graph::Random.new: Graph::Distribution::WattsStrogatz.new(20,0.06); my $gp = Graph::Path.new: $gl.find-shortest-path('0','12'), :directed; my $grPlot = js-d3-graph-plot( $gl.edges(:dataset), highlight => [|$gp.vertex-list, |$gp.edge-list], background => '1F1F1F', title-color => 'Silver', edge-thickness => 3, vertex-size => 6, width => 600, force => {charge => {strength => -260, iterations => 2}, y => {strength => 0.2}, collision => {radius => 6, iterations => 10}, link => {distance => 4}} ) </pre></div> <figure class="wp-block-image"><img data-attachment-id="1313" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/random-graph/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph.png" data-orig-size="533,323" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Random-graph" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph.png?w=533" loading="lazy" width="533" height="323" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph.png" alt="" class="wp-image-1313" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph.png 533w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph.png?w=150&h=91 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph.png?w=300&h=182 300w" sizes="(max-width: 533px) 100vw, 533px" /></figure> <p>In the code above, we create a random graph with 20 vertices and a connection probability of 0.06. We also find the shortest path between vertices ‘0’ and ’12’.</p> <h3 class="wp-block-heading"><a></a>Corresponding Matrix</h3> <p>The adjacency matrix of this graph is a sparse matrix, where non-zero elements indicate the presence of an edge between vertices.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% js my $m = Math::SparseMatrix.new(edge-dataset => $gl.edges(:dataset), row-names => $gl.vertex-list.sort(*.Int)); say $m; $m.Array ==> js-d3-matrix-plot(width => 400, margins => 15, :$tick-labels-font-size) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Math::SparseMatrix(:specified-elements(86), :dimensions((20, 20)), :density(0.215)) </pre></div> <figure class="wp-block-image"><img data-attachment-id="1316" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/random-graph-matrix-plot/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-matrix-plot.png" data-orig-size="574,577" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Random-graph-matrix-plot" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-matrix-plot.png?w=298" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-matrix-plot.png?w=574" loading="lazy" width="574" height="577" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-matrix-plot.png" alt="" class="wp-image-1316" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-matrix-plot.png 574w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-matrix-plot.png?w=150&h=150 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-matrix-plot.png?w=298&h=300 298w" sizes="(max-width: 574px) 100vw, 574px" /></figure> <p>Here, we visualize the graph matrix, the shortest path matrix, and the sum of these matrices:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% js my $m2 = Math::SparseMatrix.new(edge-dataset => $gp.edges(:dataset), row-names => $m.row-names); my $m3 = $m.add($m2.multiply(0.75)); # Visualize my %opts = width => 350, margins => {top => 30, left => 16, right => 16, bottom => 16}, :$tick-labels-font-size, :$tick-labels-color, :$title-color, :!tooltip, color-palette => 'Inferno'; [ js-d3-matrix-plot($m.Array, |%opts, title => 'Graph'), js-d3-matrix-plot($m2.Array, |%opts, title => 'Shortest path graph'), js-d3-matrix-plot($m3.Array, |%opts, title => 'Sum') ].join("\n") </pre></div> <figure class="wp-block-image"><img data-attachment-id="1321" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/random-graph-shortest-path-and-sum-matrix-plots/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-shortest-path-and-sum-matrix-plots.png" data-orig-size="1512,513" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Random-graph-shortest-path-and-sum-matrix-plots" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-shortest-path-and-sum-matrix-plots.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-shortest-path-and-sum-matrix-plots.png?w=1008" loading="lazy" width="1512" height="513" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-shortest-path-and-sum-matrix-plots.png" alt="" class="wp-image-1321" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-shortest-path-and-sum-matrix-plots.png 1512w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-shortest-path-and-sum-matrix-plots.png?w=150&h=51 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-shortest-path-and-sum-matrix-plots.png?w=300&h=102 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-shortest-path-and-sum-matrix-plots.png?w=768&h=261 768w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-shortest-path-and-sum-matrix-plots.png?w=1024&h=347 1024w" sizes="(max-width: 1512px) 100vw, 1512px" /></figure> <p>The sum matrix is printed in a “plain” format:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> $m3.print </pre></div> <figure class="wp-block-image"><img data-attachment-id="1315" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/random-graphs-sum-matrix-print/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graphs-sum-matrix-print.png" data-orig-size="1106,614" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Random-graphs-sum-matrix-print" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graphs-sum-matrix-print.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graphs-sum-matrix-print.png?w=1008" loading="lazy" width="1106" height="614" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graphs-sum-matrix-print.png" alt="" class="wp-image-1315" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graphs-sum-matrix-print.png 1106w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graphs-sum-matrix-print.png?w=150&h=83 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graphs-sum-matrix-print.png?w=300&h=167 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graphs-sum-matrix-print.png?w=768&h=426 768w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graphs-sum-matrix-print.png?w=1024&h=568 1024w" sizes="(max-width: 1106px) 100vw, 1106px" /></figure> <p>By comparing the graph and the sum matrix side-by-side, we can better understand the structure and relationships within the graph:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% js [ $grPlot, js-d3-matrix-plot($m3.Array, margins => 16, :$tick-labels-font-size, :$tick-labels-color, width => 400, color-palette => 'Inferno') ].join("\n") </pre></div> <figure class="wp-block-image"><img data-attachment-id="1320" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/random-graph-and-matrix/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-and-matrix.png" data-orig-size="1297,583" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Random-graph-and-matrix" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-and-matrix.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-and-matrix.png?w=1008" loading="lazy" width="1297" height="583" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-and-matrix.png" alt="" class="wp-image-1320" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-and-matrix.png 1297w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-and-matrix.png?w=150&h=67 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-and-matrix.png?w=300&h=135 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-and-matrix.png?w=768&h=345 768w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-graph-and-matrix.png?w=1024&h=460 1024w" sizes="(max-width: 1297px) 100vw, 1297px" /></figure> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Ingest Movie-Actor Data</h2> <p>Next, we will ingest a CSV file containing data about movies and actors. This data will be used to create a bipartite graph.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $file = $*CWD ~ '/Sparse-matrices/dsMovieRecords.csv'; my @dsMovieRecords = data-import($file, 'csv', headers => 'auto'); deduce-type(@dsMovieRecords) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Vector(Assoc(Atom((Str)), Atom((Str)), 6), 40) </pre></div> <h3 class="wp-block-heading"><a></a>Movie Data Table</h3> <p>Here is a tabular representation of the movie data:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html my @field-names = <Movie Actor Genre1 Genre2 Genre3 BoxOffice>; @dsMovieRecords ==> to-html(:@field-names) </pre></div> <figure class="wp-block-image"><img data-attachment-id="1325" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/movie-actor-dataset/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-dataset.png" data-orig-size="974,1115" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Movie-actor-dataset" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-dataset.png?w=262" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-dataset.png?w=895" loading="lazy" width="974" height="1115" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-dataset.png" alt="" class="wp-image-1325" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-dataset.png 974w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-dataset.png?w=131&h=150 131w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-dataset.png?w=262&h=300 262w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-dataset.png?w=768&h=879 768w" sizes="(max-width: 974px) 100vw, 974px" /></figure> <p>A summary of the data:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html records-summary(@dsMovieRecords, :8max-tallies, :!say) ==> to-html(:@field-names) </pre></div> <figure class="wp-block-image"><img data-attachment-id="1306" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/movie-actor-data-summary/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-data-summary.png" data-orig-size="1202,267" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Movie-actor-data-summary" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-data-summary.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-data-summary.png?w=1008" loading="lazy" width="1202" height="267" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-data-summary.png" alt="" class="wp-image-1306" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-data-summary.png 1202w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-data-summary.png?w=150&h=33 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-data-summary.png?w=300&h=67 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-data-summary.png?w=768&h=171 768w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-data-summary.png?w=1024&h=227 1024w" sizes="(max-width: 1202px) 100vw, 1202px" /></figure> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Bipartite Graph</h2> <p>We construct a bipartite graph based on the movie-actor relationships.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my @rules = @dsMovieRecords.map({ $_<Movie> => $_<Actor> }); my $g = Graph.new(@rules) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Graph(vertexes => 27, edges => 40, directed => False) </pre></div> <p>The graph is confirmed to be bipartite:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> $g.is-bipartite </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # True </pre></div> <p>Here is the coloring of the graph:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> .say for $g.bipartite-coloring.classify(*.value) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # 1 => [X2 => 1 The Lord of the Rings: The Fellowship of the Ring => 1 Pirates of the Caribbean: The Curse of the Black Pearl => 1 The Lord of the Rings: The Return of the King => 1 Pirates of the Caribbean: At World's End => 1 X-Men: The Last Stand => 1 The Lord of the Rings: The Two Towers => 1 Pirates of the Caribbean: Dead Man's Chest => 1] # 0 => [Sean Astin => 0 Patrick Stewart => 0 Elijah Wood => 0 Rebecca Romijn => 0 Ian McKellen => 0 Keira Knightley => 0 Orlando Bloom => 0 Famke Janssen => 0 Bill Nighy => 0 Johnny Depp => 0 Jack Davenport => 0 Hugh Jackman => 0 Liv Tyler => 0 Halle Berry => 0 Andy Serkis => 0 Geoffrey Rush => 0 Stellan Skarsgård => 0 Anna Paquin => 0 Viggo Mortensen => 0] </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% js $g.edges(:dataset) ==> js-d3-graph-plot( highlight => @dsMovieRecords.map(*<Actor>).List, :$background, title-color => 'silver', width => 1000, :$edge-thickness, :$vertex-size, vertex-color => 'Red', vertex-label-font-size => 12, vertex-label-color => 'Grey', vertex-label-font-family => 'Helvetica', :!directed, force => {charge => {strength => -680, iterations => 2}, collision => {radius => 10, iterations => 1}, link => {minDistance => 10}} ) </pre></div> <figure class="wp-block-image"><img data-attachment-id="1322" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/movie-actor-bipartite-graph/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-graph.png" data-orig-size="1019,743" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Movie-actor-bipartite-graph" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-graph.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-graph.png?w=1008" loading="lazy" width="1019" height="743" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-graph.png" alt="" class="wp-image-1322" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-graph.png 1019w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-graph.png?w=150&h=109 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-graph.png?w=300&h=219 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-graph.png?w=768&h=560 768w" sizes="(max-width: 1019px) 100vw, 1019px" /></figure> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Sparse Matrix</h2> <p>We create a sparse matrix representing the movie-actor relationships:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my @allVertexNames = [|@dsMovieRecords.map(*<Movie>).unique.sort, |@dsMovieRecords.map(*<Actor>).unique.sort]; my %h = @allVertexNames Z=> ^@allVertexNames.elems; </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # {Andy Serkis => 8, Anna Paquin => 9, Bill Nighy => 10, Elijah Wood => 11, Famke Janssen => 12, Geoffrey Rush => 13, Halle Berry => 14, Hugh Jackman => 15, Ian McKellen => 16, Jack Davenport => 17, Johnny Depp => 18, Keira Knightley => 19, Liv Tyler => 20, Orlando Bloom => 21, Patrick Stewart => 22, Pirates of the Caribbean: At World's End => 0, Pirates of the Caribbean: Dead Man's Chest => 1, Pirates of the Caribbean: The Curse of the Black Pearl => 2, Rebecca Romijn => 23, Sean Astin => 24, Stellan Skarsgård => 25, The Lord of the Rings: The Fellowship of the Ring => 3, The Lord of the Rings: The Return of the King => 4, The Lord of the Rings: The Two Towers => 5, Viggo Mortensen => 26, X-Men: The Last Stand => 6, X2 => 7} </pre></div> <p>The row and column names are sorted, with movie titles first, followed by actor names:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> .say for @allVertexNames </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Pirates of the Caribbean: At World's End # Pirates of the Caribbean: Dead Man's Chest # Pirates of the Caribbean: The Curse of the Black Pearl # The Lord of the Rings: The Fellowship of the Ring # The Lord of the Rings: The Return of the King # The Lord of the Rings: The Two Towers # X-Men: The Last Stand # X2 # Andy Serkis # Anna Paquin # Bill Nighy # Elijah Wood # Famke Janssen # Geoffrey Rush # Halle Berry # Hugh Jackman # Ian McKellen # Jack Davenport # Johnny Depp # Keira Knightley # Liv Tyler # Orlando Bloom # Patrick Stewart # Rebecca Romijn # Sean Astin # Stellan Skarsgård # Viggo Mortensen </pre></div> <p>The sparse matrix of the bipartite graph is constructed:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $m = Math::SparseMatrix.new(edge-dataset => $g.edges(:dataset)) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Math::SparseMatrix(:specified-elements(80), :dimensions((27, 27)), :density(<80/729>)) </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #%js $m.Array ==> js-d3-matrix-plot(width=>400) </pre></div> <figure class="wp-block-image"><img data-attachment-id="1312" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/movie-actor-bipartite-matrix-unclear/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-unclear.png" data-orig-size="490,491" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Movie-actor-bipartite-matrix-unclear" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-unclear.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-unclear.png?w=490" loading="lazy" width="490" height="491" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-unclear.png" alt="" class="wp-image-1312" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-unclear.png 490w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-unclear.png?w=150&h=150 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-unclear.png?w=300&h=300 300w" sizes="(max-width: 490px) 100vw, 490px" /></figure> <p>To clearly show the bipartite nature of the matrix, we restructure it using pre-arranged row and column names:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> $m = $m[@allVertexNames; @allVertexNames] </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Math::SparseMatrix(:specified-elements(80), :dimensions((27, 27)), :density(<80/729>)) </pre></div> <p>The matrix plot now clearly indicates a bipartite graph:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #%js $m.Array ==> js-d3-matrix-plot(width=>400) </pre></div> <figure class="wp-block-image"><img data-attachment-id="1310" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/movie-actor-bipartite-matrix-proper-sort/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-proper-sort.png" data-orig-size="490,490" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Movie-actor-bipartite-matrix-proper-sort" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-proper-sort.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-proper-sort.png?w=490" loading="lazy" width="490" height="490" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-proper-sort.png" alt="" class="wp-image-1310" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-proper-sort.png 490w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-proper-sort.png?w=150&h=150 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-proper-sort.png?w=300&h=300 300w" sizes="(max-width: 490px) 100vw, 490px" /></figure> <p>For an alternative visualization, we can create an HTML “pretty print” of the sparse matrix:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html $m .to-html(:v) .subst('<td>1</td>', '<td><b>●</b></td>', :g) </pre></div> <figure class="wp-block-image"><img data-attachment-id="1324" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/movie-actor-bipartite-matrix-hmtl/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-hmtl.png" data-orig-size="1378,1130" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Movie-actor-bipartite-matrix-HMTL" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-hmtl.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-hmtl.png?w=1008" loading="lazy" width="1378" height="1130" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-hmtl.png" alt="" class="wp-image-1324" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-hmtl.png 1378w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-hmtl.png?w=150&h=123 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-hmtl.png?w=300&h=246 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-hmtl.png?w=768&h=630 768w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/movie-actor-bipartite-matrix-hmtl.png?w=1024&h=840 1024w" sizes="(max-width: 1378px) 100vw, 1378px" /></figure> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Fundamental Information Retrieval Operation</h2> <p>Sparse matrices are particularly useful for information retrieval operations. Here, we demonstrate how to retrieve data about an actor, such as Orlando Bloom.</p> <ul class="wp-block-list"> <li>Retrieve the row/vector corresponding to the actor and transpose it:</li> </ul> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #%html my $m-actor = $m['Orlando Bloom'].transpose; $m-actor.to-html.subst('<td>0</td>','<td> </td>'):g </pre></div> <figure class="wp-block-image"><img data-attachment-id="1318" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/orlando-bloom-movies-in/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/orlando-bloom-movies-in.png" data-orig-size="531,772" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Orlando-Bloom-movies-in" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/orlando-bloom-movies-in.png?w=206" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/orlando-bloom-movies-in.png?w=531" loading="lazy" width="531" height="772" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/orlando-bloom-movies-in.png" alt="" class="wp-image-1318" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/orlando-bloom-movies-in.png 531w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/orlando-bloom-movies-in.png?w=103&h=150 103w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/orlando-bloom-movies-in.png?w=206&h=300 206w" sizes="(max-width: 531px) 100vw, 531px" /></figure> <ul class="wp-block-list"> <li>Multiply the incidence matrix with the actor-vector to find other actors who starred in the same movies:</li> </ul> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% html $m.dot($m-actor).to-html </pre></div> <figure class="wp-block-image"><img data-attachment-id="1319" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/orlando-bloom-actors-with/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/orlando-bloom-actors-with.png" data-orig-size="529,759" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Orlando-Bloom-actors-with" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/orlando-bloom-actors-with.png?w=209" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/orlando-bloom-actors-with.png?w=529" loading="lazy" width="529" height="759" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/orlando-bloom-actors-with.png" alt="" class="wp-image-1319" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/orlando-bloom-actors-with.png 529w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/orlando-bloom-actors-with.png?w=105&h=150 105w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/orlando-bloom-actors-with.png?w=209&h=300 209w" sizes="(max-width: 529px) 100vw, 529px" /></figure> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Matrix Plot (<em>Details</em>)</h2> <p>There are two primary methods for plotting sparse matrices.</p> <h3 class="wp-block-heading"><a></a>Via Tuples</h3> <p>This method uses a heatmap plot specification:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% js my @ds3D = $m.tuples.map({ <x y z tooltip>.Array Z=> [|$_.Array, "⎡{$m.row-names[$_[0]]}⎦ : ⎡{$m.column-names[$_[1]]}⎦ : {$_.tail}"] })».Hash; js-d3-matrix-plot( @ds3D, :$tooltip-background-color, :$tooltip-color, :$background, width => 400) </pre></div> <figure class="wp-block-image"><img data-attachment-id="1309" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/bipartite-matrix-plot-via-heatmap-plot-spec/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-matrix-plot-via-heatmap-plot-spec.png" data-orig-size="488,492" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Bipartite-matrix-plot-via-heatmap-plot-spec" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-matrix-plot-via-heatmap-plot-spec.png?w=298" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-matrix-plot-via-heatmap-plot-spec.png?w=488" loading="lazy" width="488" height="492" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-matrix-plot-via-heatmap-plot-spec.png" alt="" class="wp-image-1309" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-matrix-plot-via-heatmap-plot-spec.png 488w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-matrix-plot-via-heatmap-plot-spec.png?w=150&h=150 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-matrix-plot-via-heatmap-plot-spec.png?w=298&h=300 298w" sizes="(max-width: 488px) 100vw, 488px" /></figure> <p>Here is the corresponding (“coordinates”) list plot:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #%js $m.tuples ==> js-d3-list-plot(:$background, width => 400, :!grid-lines) </pre></div> <figure class="wp-block-image"><img data-attachment-id="1308" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/bipartite-list-plot-coordinates/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-list-plot-coordinates.png" data-orig-size="486,468" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Bipartite-list-plot-coordinates" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-list-plot-coordinates.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-list-plot-coordinates.png?w=486" loading="lazy" width="486" height="468" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-list-plot-coordinates.png" alt="" class="wp-image-1308" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-list-plot-coordinates.png 486w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-list-plot-coordinates.png?w=150&h=144 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-list-plot-coordinates.png?w=300&h=289 300w" sizes="(max-width: 486px) 100vw, 486px" /></figure> <h3 class="wp-block-heading"><a></a>As Dense Matrix</h3> <p>This method visualizes the matrix as a dense array:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #%js $m.Array ==> js-d3-matrix-plot(width => 400) </pre></div> <figure class="wp-block-image"><img data-attachment-id="1307" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/bipartite-dense-matrix/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-dense-matrix.png" data-orig-size="489,489" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Bipartite-dense-matrix" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-dense-matrix.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-dense-matrix.png?w=489" loading="lazy" width="489" height="489" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-dense-matrix.png" alt="" class="wp-image-1307" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-dense-matrix.png 489w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-dense-matrix.png?w=150&h=150 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/bipartite-dense-matrix.png?w=300&h=300 300w" sizes="(max-width: 489px) 100vw, 489px" /></figure> <h3 class="wp-block-heading"><a></a>Larger Sparse Matrix</h3> <p>For larger matrices, a list plot might be more useful, especially if the matrix has a relatively high density.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $gLarge = Graph::Random.new: Graph::Distribution::WattsStrogatz.new(100,0.1); my $mLarge = Math::SparseMatrix.new(edge-dataset => $gLarge.edges(:dataset)); </pre></div> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Math::SparseMatrix(:specified-elements(444), :dimensions((100, 100)), :density(0.0444)) </pre></div> <p>The corresponding graph:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #% js $mLarge.tuples ==> js-d3-list-plot( :$background, width => 600, height => 600, :!grid-lines) </pre></div> <figure class="wp-block-image"><img data-attachment-id="1314" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/large-sparse-matrix-tuples/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/large-sparse-matrix-tuples.png" data-orig-size="726,699" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Large-sparse-matrix-tuples" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/large-sparse-matrix-tuples.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/large-sparse-matrix-tuples.png?w=726" loading="lazy" width="726" height="699" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/large-sparse-matrix-tuples.png" alt="" class="wp-image-1314" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/large-sparse-matrix-tuples.png 726w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/large-sparse-matrix-tuples.png?w=150&h=144 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/large-sparse-matrix-tuples.png?w=300&h=289 300w" sizes="(max-width: 726px) 100vw, 726px" /></figure> <p><strong>Remark:</strong> The list plot might be much more useful for large matrices with (relatively) high density.</p> <p>Tuples dataset:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #%js $mLarge.tuples(:dataset) ==> {rename-columns($_, (<i j x> Z=> <x y z>).Hash)}() ==> js-d3-matrix-plot(:$background, width => 600) </pre></div> <figure class="wp-block-image"><img data-attachment-id="1317" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/large-sparse-matrix-tuples-dataset/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/large-sparse-matrix-tuples-dataset.png" data-orig-size="726,730" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Large-sparse-matrix-tuples-dataset" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/large-sparse-matrix-tuples-dataset.png?w=298" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/large-sparse-matrix-tuples-dataset.png?w=726" loading="lazy" width="726" height="730" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/large-sparse-matrix-tuples-dataset.png" alt="" class="wp-image-1317" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/large-sparse-matrix-tuples-dataset.png 726w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/large-sparse-matrix-tuples-dataset.png?w=150&h=150 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/large-sparse-matrix-tuples-dataset.png?w=298&h=300 298w" sizes="(max-width: 726px) 100vw, 726px" /></figure> <h3 class="wp-block-heading"><a></a>Random Dense Matrix</h3> <p>Lastly, we explore a dense matrix example:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> #%js my @a = random-real(10, 48) xx 12; @a = rand > 0.5 ?? @a.map(*.sort) !! @a.&transpose.map(*.sort.Array).&transpose; say "dimensions : ", dimensions(@a); js-d3-matrix-plot(@a, width => 1600, margins => 1, :$tick-labels-font-size, color-palette => <Turbo Plasma Warm Inferno Oranges>.pick, :$background) </pre></div> <figure class="wp-block-image"><img data-attachment-id="1311" data-permalink="https://rakuforprediction.wordpress.com/2024/12/28/sparse-matrix-neat-examples-in-raku/random-sparse-matrix-ending-light-turbo/#main" data-orig-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-sparse-matrix-ending-light-turbo.png" data-orig-size="1932,493" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Random-sparse-matrix-ending-light-turbo" data-image-description="" data-image-caption="" data-medium-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-sparse-matrix-ending-light-turbo.png?w=300" data-large-file="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-sparse-matrix-ending-light-turbo.png?w=1008" loading="lazy" width="1932" height="493" src="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-sparse-matrix-ending-light-turbo.png" alt="" class="wp-image-1311" srcset="https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-sparse-matrix-ending-light-turbo.png 1932w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-sparse-matrix-ending-light-turbo.png?w=150&h=38 150w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-sparse-matrix-ending-light-turbo.png?w=300&h=77 300w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-sparse-matrix-ending-light-turbo.png?w=768&h=196 768w, https://rakuforprediction.wordpress.com/wp-content/uploads/2024/12/random-sparse-matrix-ending-light-turbo.png?w=1024&h=261 1024w" sizes="(max-width: 1932px) 100vw, 1932px" /></figure> <hr class="wp-block-separator has-alpha-channel-opacity" /> <h2 class="wp-block-heading"><a></a>Reference</h2> <h3 class="wp-block-heading"><a></a>Articles</h3> <p>[AA1] Anton Antonov, <a href="https://mathematicaforprediction.wordpress.com/2015/10/08/rsparsematrix-for-sparse-matrices-with-named-rows-and-columns/">“RSparseMatrix for sparse matrices with named rows and columns”</a>, (2015), <a href="https://mathematicaforprediction.wordpress.com/">MathematicaForPrediction at WordPress</a>.</p> <h3 class="wp-block-heading"><a></a>Packages</h3> <p>[AAp1] Anton Antonov, <a href="https://github.com/antononcube/Raku-Math-SparseMatrix">Math::SparseMatrix Raku package</a>, (2024), <a href="https://github.com/antononcube">GitHub/antononcube</a>.</p> <p>[AAp2] Anton Antonov, <a href="https://github.com/antononcube/Raku-Math-SparseMatrix-Native">Math::SparseMatrix::Native Raku package</a>, (2024), <a href="https://github.com/antononcube">GitHub/antononcube</a>.</p> <p>[AAp3] Anton Antonov, <a href="https://github.com/antononcube/Raku-Graph">Graph Raku package</a>, (2024), <a href="https://github.com/antononcube">GitHub/antononcube</a>.</p> <p>[AAp4] Anton Antonov, <a href="https://github.com/antononcube/Raku-JavaScript-D3">JavaScript::D3 Raku package</a>, (2022-2024), <a href="https://github.com/antononcube">GitHub/antononcube</a>.</p> <h2><a href="https://raku-advent.blog/2024/12/26/the-2024-raku-advent-posts/">The 2024 Raku Advent Posts</h2></a> <p><h3>Published by Elizabeth Mattijsen on 2024-12-26T01:00:00</h3></p> <p class="has-text-align-center">(in chronological order, with comment references)</p> <ul class="wp-block-list"> <li><a href="https://raku-advent.blog/2024/12/01/day-1-rendering-down-for-christmas/">Day 1 – Rendering down for Christmas</a> by <em>Richard Hainsworth</em>.</li> <li><a href="https://raku-advent.blog/2024/12/02/day-2-wat-llm-coding-tool-do-you-want-for-christmas/">Day 2 – WAT LLM coding tool do you want for Christmas?</a> by <em>Nigel Hamilton</em>.</li> <li><a href="https://raku-advent.blog/2024/12/03/day-3-merry-cromas/">Day 3 – Merry Cromas</a> by <em>Steve Roe</em>.</li> <li><a href="https://raku-advent.blog/2024/12/04/day-4-dont-use-forsyth-edwards-notation-to-play-chess-with-llms/">Day 4 – Don’t use Forsyth-Edwards Notation to play chess with LLMs</a> by <em>Anton Antonov</em>.</li> <li><a href="https://raku-advent.blog/2024/12/05/day-5-generating-an-html-dashboard-with-vanilla-raku/">Day 5 – Generating an HTML Dashboard With Vanilla Raku</a> by <em>Coleman McFarland</em>.</li> <li><a href="https://raku-advent.blog/2024/12/06/day-6-creating-a-presentation-hosted-on-a-gemini-capsule/">Day 6 – Creating a presentation hosted on a Gemini capsule</a> by <em>Samuel Chase</em>.</li> <li><a href="https://raku-advent.blog/2024/12/07/day-7-conditionally-writeable-attributes/">Day 7 – Conditionally Writeable Attributes</a> by <em>landyacht</em>.</li> <li><a href="https://raku-advent.blog/2024/12/08/day-8-yet-more-abilities-for-iterables/">Day 8 – Yet More Abilities for Iterables</a> by <em>Mustafa Aydın</em>.</li> <li><a href="https://raku-advent.blog/2024/12/09/day-9-the-end-of-the-year/">Day 9 – The end of the year</a> by <em>Marcel Timmerman</em>.</li> <li><a href="https://raku-advent.blog/2024/12/10/how-to-give-a-raku-talk/">Day 10 – How to give a Raku talk at TPRC – and why you should</a> by <em>Daniel Sockwell</em> (<a href="https://www.reddit.com/r/rakulang/comments/1hayvc9/comment/m1cl6z5/">/r/rakulang</a> comments).</li> <li><a href="https://raku-advent.blog/2024/12/11/day-11-counting-up-concurrency/">Day 11 – Counting up concurrency</a> by <em>Hillel Wayne</em>.</li> <li><a href="https://raku-advent.blog/2024/12/12/day-12-graphs-in-raku/">Day 12 – Graphs in Raku</a> by <em>Anton Antonov</em>.</li> <li><a href="https://raku-advent.blog/2024/12/13/day-13-content-storage-for-raku-distributions/">Day 13 – Content Storage For Raku Distributions</a> by <em>Haytham Elganiny</em>.</li> <li><a href="https://raku-advent.blog/2024/12/14/day-14-playing-around-with-the-new-documentation-components/">Day 14 – Playing around with the new documentation components</a> by <em>Richard Hainsworth</em>.</li> <li><a href="https://raku-advent.blog/2024/12/15/day-15-matching-maps/">Day 15 – Matching Maps</a> by <em>Elizabeth Mattijsen</em>.</li> <li><a href="https://raku-advent.blog/2024/12/16/day-16-revision-gating-in-rakudo-core/">Day 16 – Revision gating in Rakudo core</a> by <em>John Haltiwanger</em>.</li> <li><a href="https://raku-advent.blog/2024/12/17/day-17-chebyshev-polynomials-and-fitting-workflows/">Day 17 – Chebyshev Polynomials and Fitting Workflows</a> by <em>Anton Antonov</em>.</li> <li><a href="https://raku-advent.blog/2024/12/18/day-18-happy-xmas/">Day 18 – Happy<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2122.png" alt="™" class="wp-smiley" style="height: 1em; max-height: 1em;" /> Xmas</a> by <em>Steve Roe</em>.</li> <li><a href="https://raku-advent.blog/2024/12/19/day-19-wrapping-scripts/">Day 19 – Wrapping Scripts</a> by <em>Patrick Böker</em>.</li> <li><a href="https://raku-advent.blog/2024/12/20/day-20-re-introducing-a-raku-plugin-for-intellij-idea/">Day 20 – Re-introducing a Raku plugin for IntelliJ IDEA</a> by <em>John Haltiwanger</em>.</li> <li><a href="https://raku-advent.blog/2024/12/21/day-21-dam-mega-christmas/">Day 21 – Dam Mega Christmas</a> by <em>Steve Roe</em>.</li> <li><a href="https://raku-advent.blog/2024/12/22/day-22-wrapping-a-christmas-present/">Day 22 – Wrapping a Christmas Present</a> by <em>Richard Hainsworth</em>.</li> <li><a href="https://raku-advent.blog/2024/12/23/day-23-santas-print-shop/">Day 23 – Santa’s Print Shop</a> by <em>Tom Browder</em>.</li> <li><a href="https://raku-advent.blog/2024/12/24/day-24-in-search-of-the-essence-of-raku/">Day 24 – In Search of the Essence of Raku</a> by <em>Donald Hunter</em>.</li> <li><a href="https://raku-advent.blog/2024/12/25/day-25-raku-2024-review/">Day 25 – Raku 2024 Review</a> by <em>Elizabeth Mattijsen</em>.</li> </ul> <h2><a href="https://raku-advent.blog/2024/12/25/day-25-raku-2024-review/">Day 25 – Raku 2024 Review</h2></a> <p><h3>Published by Elizabeth Mattijsen on 2024-12-25T01:00:00</h3></p> <p>How time flies. Yet another year has flown by. 2024 was a year of changes, continuations and preparations. Let’s start with the changes:</p> <h3 class="wp-block-heading">Edument<a href="https://github.com/lizmat/articles/blob/main/review-of-2024.md#edument"></a></h3> <p>Edument Central Europe, the branch of Edument that is based in Prague (and led by <em>Jonathan Worthington</em>), decided to stop (commercial) development of its Raku related products: <a href="https://commaide.com/"><code>Comma</code></a> (the IDE for the Raku Programming Language) and <a href="https://cro.services/"><code>Cro</code></a> (a set of libraries for building reactive distributed systems).</p> <h4 class="wp-block-heading">Comma</h4> <p><a href="https://github.com/lizmat/articles/blob/main/review-of-2024.md#comma"></a>The <a href="https://commaide.com/discontinued">announcement</a>:</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <p class="has-small-font-size">With the discrepancy between revenue and development cost to continue being so large, and the prevailing economic environment forcing us to focus on business activities that at least pay for themselves, we’ve made the sad decision to discontinue development of the Comma IDE.</p> </blockquote> <p>Fortunately, <em>Jonathan Worthington</em> was not only able to release the final commercial version as a free download, but was also able to release all of the sources of Comma. This would allow other people to continue Comma development.</p> <p>With <em>John Haltiwanger</em> being the now de-facto project leader, this has resulted in a <a href="https://github.com/ab5tract/comma-plugin/releases/tag/2024.3-beta.1">beta of the open source version of a Raku plugin for IntelliJ IDEA</a>, as described on <a href="https://raku-advent.blog/2024/12/20/day-20-re-introducing-a-raku-plugin-for-intellij-idea/">Day 20</a>.</p> <h4 class="wp-block-heading">Cro<a href="https://github.com/lizmat/articles/blob/main/review-of-2024.md#cro"></a></h4> <p>The <a href="https://web.archive.org/web/20240421151537/https://cro.services/community-transfer">announcement</a>:</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <p class="has-small-font-size">When Edument employees were by far the dominant Cro contributors, it made sense for us to carry the overall project leadership. However, the current situation is that members of the Raku community contribute more. We don’t see this balance changing in the near future.</p> <p class="has-small-font-size">With that in mind, we entered into discussions with the Raku Steering Council, in order that we can smoothly transfer control of Cro and its related projects to the Raku community. In the coming weeks, we will transfer the GitHub organization and release permissions to steering council representatives, and will work with the Raku community infrastructure team with regards to the project website.</p> </blockquote> <p>As the source code of <code>Cro</code> had always been open source, this was more a question of handing over responsibilities. Fortunately the Raku Community reacted: <em>Patrick Böker</em> has taken care of making <code>Cro</code> a true open source project related to Raku, and the <a href="https://cro.raku.org/">associated web site https://cro.raku.org</a> is now being hosted on the Raku infrastructure. With many kudos to the Raku Infra Team!</p> <h3 class="wp-block-heading">A huge thanks</h3> <p><a href="https://github.com/lizmat/articles/blob/main/review-of-2024.md#a-huge-thanks"></a>Sadly, <em>Jonathan Worthington</em> also indicated that they would only remain minimally involved in the further development of <a href="https://github.com/MoarVM/MoarVM">MoarVM</a>, <a href="https://github.com/raku/NQP">NQP</a> and <a href="https://github.com/rakudo/rakudo/">Rakudo</a> in the foreseeable future. As such, (almost) all of their modules were moved to the <a href="https://github.com/raku-community-modules">Raku Community Modules Adoption Center</a>, where they were updated and re-released.</p> <p>It’s hard to overstate the importance of <em>Jonathan Worthington</em>‘s work in the development and implementation of the Raku Programming Language. So on behalf of the current, past and future Raku Community members: <strong>Thank You!</strong></p> <h3 class="wp-block-heading">Issues cleanup</h3> <p><a href="https://github.com/lizmat/articles/blob/main/review-of-2024.md#issues-cleanup"></a>At the beginning of October, <em>Elizabeth Mattijsen</em> decided to take on the large number of open Rakudo issues at that time: <strong>1300+</strong>. This resulted in the closing of more than <strong>500</strong> issues: some just needed closing, some needed tests, and some could be fixed pretty easily.</p> <p>They reported on that work in the <a href="https://dev.to/lizmat/raku-fall-issue-cleanup-lkc">Raku Fall Issue Cleanup</a> blog post. Of the about 800 open issues remaining, almost 300 were marked as “fixed in RakuAST”, and about 100 were marked as “Will be addressed in RakuAST”. Which still leaves about <strong>400</strong> open because of other reasons, so there’s still plenty of work to be done here.</p> <h3 class="wp-block-heading">More cleanup: p6c</h3> <p>The original Raku ecosystem (“p6c”) is in the process of being completely removed. Since March 2023, the ecosystem was no longer being refreshed by <code>zef</code>. But it was still being refreshed by the <a href="https://raku.land/zef:lizmat/Ecosystem::Archive::Update">Raku Ecosystem Archiver</a>. But this stopped in August 2024, meaning that any updates of modules in that ecosystem would go unnoticed from then on.</p> <p>At that time, every author who still had at least one module in the “p6c” ecosystem was given notice by creating an issue in the repository of the first of their modules in the <code><a href="https://raw.githubusercontent.com/Raku/ecosystem/refs/heads/main/META.list">META.list</a></code>. Luckily many authors responded, either by indicating that they would migrate their module(s) to the “zef” ecosystem, or that they were no longer interested in maintaining.</p> <p>Since then, most of the modules of the authors that responded, have been migrated. And work has started on the modules of the authors that did not respond. With the following results: at the beginning of 2024, there were still <em>658</em> modules in the “p6c” ecosystem (now <em>427</em>, 35% less), by <em>230</em> different authors (now <em>138</em>, 40% less).</p> <h3 class="wp-block-heading">Ecosystem statistics</h3> <p>In 2024, <em>558</em> Raku modules have been updated (or first released): up from <em>332</em> in 2023 (an increase of 68%). There are now <em>2304</em> different modules installable by <code>zef </code>by just mentioning their name. And there are now <em>12181</em> different versions of Raku modules available from the <a href="https://github.com/raku/REA?tab=readme-ov-file#raku-programming-language-ecosystem-archive">Raku Ecosystem Archive</a>, up from <em>10754</em> in 2023, which means almost <em>4</em> module updates / day in 2024.</p> <h3 class="wp-block-heading">Rakudo</h3> <p>Rakudo saw about 2000 commits (MoarVM, NQP, Rakudo, doc) this year, which is about the same as in 2023. About one third of these commits were in the development of RakuAST (down from 75% in 2023).</p> <h4 class="wp-block-heading">Under the hood, behind the scenes</h4> <p>A lot of work was done under the hood of the various subsystems of Rakudo. So was the dispatcher logic simplified by introducing <a href="https://github.com/rakudo/rakudo/blob/main/docs/dispatchers.md">several nqp:: shortcuts</a>, which made the dispatcher code a lot more readable and maintainable.</p> <p>The Meta-classes of <code>NQP</code> and <code>Raku</code> also received a bit of a makeover, as most of them hadn’t been touched since the 2015 release: this resulted in better documentation, and some minor performance improvements. Support for <code><a href="https://docs.raku.org/syntax/TWEAK">TWEAK</a></code> methods and a rudimentary <code><a href="https://docs.raku.org/programs/01-debugging#index-entry-dd">dd</a></code> functionality were also added to <code>NQP</code>.</p> <p>The <code>JVM</code> backend also got some TLC in 2024: one under the hood change (by <em>Daniel Green</em>) made execution of Raku code on the JVM backend twice as fast!</p> <p><em>Timo Paulssen</em> made the interface with low-level debuggers such as <code>gdb</code> and <code>lldb</code> a lot less cumbersome on <code>MoarVM</code>, which makes adding / fixing <code>MoarVM</code> features a lot easier!</p> <p>On the MoarVM backend the expression JIT (active on Intel hardware) was disabled by default: it was found to be too unreliable <strong>and</strong> did <em>not</em> provide any execution speed gains. This change made Rakudo on Intel hardware up to 5% faster overall.</p> <p>Also on the MoarVM backend, <em>Daniel Green</em> completed the work on optimizing short strings started by <em>Timo Paulssen</em> and <em>Bart Wiegmans</em>, resulting in about a 2% speed improvement of the compilation of Raku code.</p> <p>Work on the <a href="https://raku.land/zef:raku-community-modules/App::MoarVM::Debug">Remote Debugger</a> (which so far had only been really used as part of Comma) has resumed, now with a much better command line interface. And can now be checked in the language itself with the new <code>VM.remote-debugging</code> method.</p> <p>Some race conditions were fixed: a particularly nasty one on the lazy deserialization of bytecode that was very hard to reproduce, as well as some infiniloops.</p> <p>A lot of work was done on making the Continuous Integration testing produce fewer (and recently hardly any) false positives anymore. Which makes life for core developers a <em>lot</em> easier!</p> <h3 class="wp-block-heading">New traits</h3> <p>Two new <code>Routine</code> traits were added to the Raku Programming Language in 2024.</p> <h4 class="wp-block-heading">is item</h4> <p>The <code>is item</code> trait can be used on <code>@</code> and <code>%</code> sigilled parameters to indicate that a <code>Positional</code> (in the <code>@</code> case) or an <code>Associative</code> (in the <code>%</code> case) is only acceptable in dispatch if it is presented as an item. It only serves as a tie-breaker, so there should always also be a dispatch candidate that would accept the argument when it is <strong>not</strong> itemized. Perhaps an example makes this more clear:</p> <pre class="wp-block-code has-foreground-background-color has-background" style="border-width:8px"><code>multi sub foo(@a) { say "array" } multi sub foo(@a <mark style="background-color:rgba(0, 0, 0, 0);color:#49e112" class="has-inline-color"><strong>is item</strong></mark>) { say "item" } foo [1,2,3]; # array foo <mark style="background-color:rgba(0, 0, 0, 0);color:#53e71e" class="has-inline-color"><strong>$</strong></mark>[1,2,3]; # item</code></pre> <h4 class="wp-block-heading">is revision-gated(“v6.x”)</h4> <p>The <code>is revision-gated</code> trait fulfils a significant part of the promise of the Raku Programming Language to be a 100-year programming language. It allows a developer to add / keep behaviour of a dispatch to a subroutine or method depending on the language level <em><strong>from which it is being called</strong></em>.</p> <p>As with <code>is item</code>, this is implemented as a tie-breaker to be checked only if there are multiple candidates in dispatch that match a given set of arguments.</p> <p>This will allow core and module developers to provide forward compatibility, as well as backward compatibility in their code (as long as the core supports a given language level, of course).</p> <p>In its current implementation, the trait <strong>must</strong> be specified on the <code>proto</code> to allow it to work (this may change in the future), and it should specify the lowest language level it should support. An example of a module “FOO” that exports a “foo” subroutine:</p> <pre class="wp-block-code has-foreground-background-color has-background" style="border-width:8px"><code>unit module FOO; proto sub foo(|) <mark style="background-color:rgba(0, 0, 0, 0);color:#6ce427" class="has-inline-color">is revision-gated("v6.c")</mark> is export {*} multi sub foo() <mark style="background-color:rgba(0, 0, 0, 0);color:#5ee51c" class="has-inline-color">is revision-gated("6.c")</mark> { say "6.c"; } multi sub foo() <mark style="background-color:rgba(0, 0, 0, 0);color:#56de25" class="has-inline-color">is revision-gated("6.d")</mark> { say "6.d" }</code></pre> <p>Then we have a program that uses the “FOO” module and calls the “foo” subroutine. This shows “6.d” because the current default language level is “6.d”.</p> <pre class="wp-block-code has-foreground-background-color has-background" style="border-width:8px"><code>use FOO; foo(); # 6.d</code></pre> <p>However, if this program would like to use language level 6.c semantics, it can indicate so by adding a <code>use v6.c</code> at the start of the program. And get a different result in an otherwise identical program:</p> <pre class="wp-block-code has-foreground-background-color has-background" style="border-width:8px"><code>use <mark style="background-color:rgba(0, 0, 0, 0);color:#3ade22" class="has-inline-color"><strong>v6.c</strong></mark>; use FOO; foo(); # 6.c</code></pre> <p><em>John Haltiwanger</em> has written an extensive blog post about the background, implementation and usage of the <code>is revision-gated</code> trait on <a href="https://raku-advent.blog/2024/12/16/day-16-revision-gating-in-rakudo-core/">Day 16</a>, if you’d like to know more about it.</p> <h3 class="wp-block-heading">Language changes (6.d)</h3> <p>These are some of the more notable changes in language level 6.d: all of them add functionality, so are completely backward compatible.</p> <h4 class="wp-block-heading">Flattening</h4> <p>The <code><a href="https://docs.raku.org/type/Any#method_flat">.flat</a></code> method optionally takes a <code>:hammer</code> named argument, which will deeply flatten any data structure given:</p> <pre class="wp-block-code has-foreground-background-color has-background" style="border-width:8px"><code>my @a = 1, [2, [3,4]]; say @a.flat; # (1 [2 [3 4]]) say @a.flat(<mark style="background-color:rgba(0, 0, 0, 0);color:#3ddd1f" class="has-inline-color"><strong>:hammer</strong></mark>); # (1 2 3 4)</code></pre> <p>One can now also use <code><a href="https://docs.raku.org/type/HyperWhatever">HyperWhatever</a></code> (aka <code>**</code>) in a postcircumfix <code>[ ]</code> for the same semantics:</p> <pre class="wp-block-code has-foreground-background-color has-background" style="border-width:8px"><code>my @a = 1, [2, [3,4]]; say @a[<mark style="background-color:rgba(0, 0, 0, 0);color:#51ed19" class="has-inline-color"><strong>*</strong></mark>]; # (1 [2 [3 4]]) say @a[<mark style="background-color:rgba(0, 0, 0, 0);color:#5be617" class="has-inline-color"><strong>**</strong></mark>]; # (1 2 3 4)</code></pre> <h4 class="wp-block-heading">Min/max/minmax options</h4> <p>The <code><a href="https://docs.raku.org/type/Any#routine_min">.min</a></code> / <code><a href="https://docs.raku.org/type/Any#routine_max">.max</a></code> / <code><a href="https://docs.raku.org/type/Any#routine_minmax">.minmax</a></code> methods now also accept the <code>:by</code> named argument to make it consistent with the sub versions, which should prevent unexpected breakage when refactoring code from a sub form to a method form (as the <code>:by</code> would previously be silently ignored in the method form).</p> <pre class="wp-block-code has-foreground-background-color has-background" style="border-width:8px"><code>say min <5 45 345>, <mark style="background-color:rgba(0, 0, 0, 0);color:#6ae01c" class="has-inline-color"><strong>:by(*.Str)</strong></mark>; # 345 say <5 45 345>.min(<mark style="background-color:rgba(0, 0, 0, 0);color:#72e31c" class="has-inline-color"><strong>*.Str</strong></mark>); # 345 say <5 45 345>.min(<mark style="background-color:rgba(0, 0, 0, 0);color:#6de31f" class="has-inline-color"><strong>:by(*.Str)</strong></mark>); # 345 ADDED</code></pre> <h3 class="wp-block-heading">.are(Type)</h3> <p>The <code><a href="https://docs.raku.org/type/Any#method_are">.are</a></code> method now also accepts a type argument. If called in such a manner, it will return <code>True</code> if all members of the invocant matched the given type, and <code>False</code> if not. Apart from allowing better readable code, it also allows shortcutting if any of the members of the invocant did <strong>not</strong> match the given type. An example:</p> <pre class="wp-block-code has-foreground-background-color has-background" style="border-width:8px"><code>unless <mark style="background-color:rgba(0, 0, 0, 0);color:#68e123" class="has-inline-color"><strong>@args.are(Pair)</strong></mark> { die "All arguments should be Pairs"; }</code></pre> <h3 class="wp-block-heading">Language changes (6.e.PREVIEW)</h3> <p>The most notable additions to the future language level of the Raku Programming Language:</p> <h4 class="wp-block-heading">.nomark</h4> <p>The <code>.nomark</code> method on <code>Cool</code> objects returns a string with the base characters of any composed characters, effectively removing any accents and such:</p> <pre class="wp-block-code has-foreground-background-color has-background" style="border-width:8px"><code>use v6.e.PREVIEW; say "élève"<mark style="background-color:rgba(0, 0, 0, 0);color:#48e124" class="has-inline-color"><strong>.nomark</strong></mark>; # eleve</code></pre> <h4 class="wp-block-heading">:smartcase</h4> <p>The <code>.contains</code> / <code>.starts-with</code> / <code>.ends-with</code> / <code>.index</code> / <code>.rindex</code> / <code>.substr-eq</code> methods now all accept a <code>:smartcase</code> named argument: a conditional <code>:ignorecase</code>. If specified with a <code>True</code> value, it will look at the needle to see if it is all lowercase. If it is, then <code>:ignorecase</code> semantics will be assumed. If there are any uppercase characters in the needle, then normal semantics will be assumed:</p> <pre class="wp-block-code has-foreground-background-color has-background" style="border-width:8px"><code>use v6.e.PREVIEW; say "Frobnicate".contains("frob"); # False say "Frobnicate".contains("frob", <mark style="background-color:rgba(0, 0, 0, 0);color:#43d71f" class="has-inline-color"><strong>:smartcase</strong></mark>); # True say "Frobnicate".contains("FROB", <mark style="background-color:rgba(0, 0, 0, 0);color:#43d71f" class="has-inline-color"><strong>:smartcase</strong></mark>); # False</code></pre> <h4 class="wp-block-heading">IO::Path.stem</h4> <p>The <code>.stem</code> method on <code>IO::Path</code> objects returns the<code> .basename</code> of the object <em>without</em> any extensions, or with the given number of extensions removed:</p> <pre class="wp-block-code has-foreground-background-color has-background" style="border-width:8px"><code>use v6.e.PREVIEW; say "foo/bar/baz.tar.gz".IO.basename; # baz.tar.gz say "foo/bar/baz.tar.gz".IO<mark style="background-color:rgba(0, 0, 0, 0);color:#4aea1b" class="has-inline-color"><strong>.stem</strong></mark>; # baz say "foo/bar/baz.tar.gz".IO<mark style="background-color:rgba(0, 0, 0, 0);color:#65e31d" class="has-inline-color"><strong>.stem(1)</strong></mark>; # baz.tar</code></pre> <h3 class="wp-block-heading">RakuAST</h3> <p>About one third of this year’s work was done on the RakuAST (Raku <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">Abstract Syntax Tree</a>) project. It basically consists of 3 sub-projects, that are heavily intertwined:</p> <ol class="wp-block-list"> <li>Development of RakuAST classes that can be used to represent all aspects of Raku code in an object-oriented fashion.</li> <li>Development of a grammar and an actions class to parse Raku source code and turn that into a tree of instantiated RakuAST objects.</li> <li>Development of new features / bug fixing in the Raku Programming Language and everything else that has become a lot easier with RakuAST.</li> </ol> <h4 class="wp-block-heading">RakuAST classes</h4> <p>There is little more to say about the development of <code><a href="https://docs.raku.org/type/RakuAST">RakuAST</a></code> classes other than that there were <strong>440</strong> of them at the start of the year, and <strong>454</strong> of them at the end of the year. As the development of these classes is still very much in flux, they are not documented yet (other than in the test-files in the <code><a href="https://github.com/rakudo/rakudo/tree/main/t/12-rakuast">/t/12rakuast</a></code> directory).</p> <p>On the other hand, the <code><a href="https://docs.raku.org/type/RakuAST/Doc">RakuAST::Doc</a></code> classes <em>are</em> documented because they have a more or less stable API to allow for the development of <a href="https://raku-advent.blog/2023/12/12/perspectives-on-rakudoc-version-2/">RakuDoc Version 2</a>.</p> <h4 class="wp-block-heading">Raku Grammar / Actions</h4> <p>The work on the Raku Grammar and Actions has been mostly about implementing already existing features. This is measured by the number of Rakudo (<code>make test</code>) and roast (<code>make spectest</code>) test-files that <em>completely</em> pass with the new Raku Grammar and Actions. And these are the changes:</p> <ul class="wp-block-list"> <li><code>make test</code>: 110/151 (72.8%) <strong>→</strong> 140/156 (89.7%)</li> <li><code>make spectest</code>: 980/1356 (72.3%) <strong>→</strong> 1155 / 1359 (85%)</li> </ul> <p>A lot of work was done by <em>Stefan Seifert</em>, picking up the compile-time handling refactor that <em>Jonathan Worthington</em> had started in 2023, but was unable to bring to fruition. TPRF continued the funding for this work.</p> <blockquote class="wp-block-quote has-small-font-size is-layout-flow wp-block-quote-is-layout-flow"> <p class="has-small-font-size">By the way, <a href="https://www.perl.com/article/duckduckgo-donates-25-000-to-the-perl-and-raku-foundation/">DuckDuckGo donated US$ 25000</a> to the foundation to allow this type of development funding to go on! Hint, hint!</p> </blockquote> <p>Like last year, there are still quite a few features left to implement. Although it must be said that many tests are hinging on the implementation of a single feature, and often cause an “avalanche” of additional test-files passing when it gets implemented.</p> <p>If you’d like to try out the new Raku Grammar and Actions, you should set the <code>RAKUDO_RAKUAST</code> environment variable to <strong>1</strong>. The <code>.legacy</code> method on the <code>Raku</code> class will tell you whether the legacy (older) grammar is being used or not:</p> <pre class="wp-block-code has-foreground-background-color has-background" style="border-width:8px"><code>$ raku -e 'say Raku<mark style="background-color:rgba(0, 0, 0, 0);color:#00d084" class="has-inline-color"><strong>.legacy</strong></mark>'<br>True<br>$ <strong><mark style="background-color:rgba(0, 0, 0, 0);color:#f78da7" class="has-inline-color">RAKUDO_RAKUAST=1</mark></strong> raku -e 'say Raku<strong><mark style="background-color:rgba(0, 0, 0, 0);color:#00d084" class="has-inline-color">.legacy</mark></strong>'<br>False</code></pre> <h4 class="wp-block-heading">Localization</h4> <p>A cousin language was created: <code><a href="https://dev.to/finanalyst/creating-a-new-programming-language-draig-503p">Draig</a></code>, allowing you to write Raku code in Welsh!</p> <h4 class="wp-block-heading">Syntax Highlighting</h4> <p>The <code><a href="https://github.com/rakudo/rakudo/blob/main/lib/RakuAST/Deparse/Highlight.rakumod#L680">RakuAST::Deparse::Highlight</a></code> module allows customizable Raku syntax highlighting of Raku code that is syntactically correct. It is definitely <strong>not</strong> usable for real-time syntax highlighting as it a. requires valid Raku code, and b. it <em>may</em> execute Raku code in <code>BEGIN</code> blocks, <code>constant</code> specifications and <code>use</code> statements.</p> <p>A more lenient alternative is the <code><a href="https://raku.land/zef:patrickb/Rainbow">Rainbow</a></code> module by <em>Patrick Böker</em>.</p> <h4 class="wp-block-heading">RakuDoc</h4> <p>The final version of the RakuDoc v2.0 specification can now be <a href="https://raku.github.io/rakudoc">completely rendered</a> thanks to the tireless work of <em>Richard Hainsworth</em> and extensive feedback and proofreading by <em>Damian Conway</em>, as shown in <a href="https://raku-advent.blog/2024/12/01/day-1-rendering-down-for-christmas/">Day 1</a> and <a href="https://raku-advent.blog/2024/12/14/day-14-playing-around-with-the-new-documentation-components/">Day 14</a>.</p> <h3 class="wp-block-heading">Conference / Core Summit</h3> <p>Sadly it has turned out to be impossible to organize a Raku Conference (neither in-person or online), nor was it possible to organize a Raku Core Summit. We’re hoping for a better situation in 2025!</p> <h3 class="wp-block-heading">Beginner Videos</h3> <p>Completely out of the blue, a Dr Raku created about <a href="https://www.youtube.com/@DrRaku-Programming">90 Raku beginner videos</a> on YouTube.</p> <h3 class="wp-block-heading"><a href="https://github.com/lizmat/articles/blob/main/review-of-2022.md#summary"></a>Summary</h3> <p>Looking back, again an amazing amount of work has been done in 2024!</p> <p>Hopefully you will all be able to enjoy the Holiday Season with sufficient R&R, especially <em><a href="https://irclogs.raku.org/raku/gist.html?,2024-04-23Z12:47,2024-04-23Z12:48,2024-04-23Z12:50-0001,2024-04-23Z12:52,2024-04-23Z13:14,2024-04-23Z14:34">Kane Valentine</a></em> (aka <em>kawaii</em>).</p> <p> The next Raku Advent Blog is only 340 days away!</p> <p></p> <h2><a href="https://raku-advent.blog/2024/12/24/day-24-in-search-of-the-essence-of-raku/">Day 24 – In Search of the Essence of Raku</h2></a> <p><h3>Published by donaldhunter on 2024-12-24T01:00:00</h3></p> <p>Amongst all of its features, what do we consider to be the <strong>essence</strong> of Raku? In this advent post we will explore what makes Raku an exciting programming language and see if we can describe its essence.</p> <p>Paul Graham wrote <a href="http://www.paulgraham.com/hundred.html">an essay about the 100 year language</a>, back in 2003. In it he had this to say:</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow" style="padding-top:var(--wp--preset--spacing--30);padding-bottom:var(--wp--preset--spacing--30)"> <p class="has-normal-font-size">Who will design the languages of the future? One of the most exciting trends in the last ten years has been the rise of open-source languages like Perl, Python, and Ruby. Language design is being taken over by hackers. The results so far are messy, but encouraging. There are some stunningly novel ideas in Perl, for example. Many are stunningly bad, but that’s always true of ambitious efforts. At its current rate of mutation, God knows what Perl might evolve into in a hundred years.</p> </blockquote> <p>Larry Wall took Paul Graham’s essay and <a href="https://thenewstack.io/larry-walls-quest-100-year-programming-language/">ran with it</a>:</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow" style="padding-right:var(--wp--preset--spacing--20);padding-left:var(--wp--preset--spacing--20)"> <p>“It has to do with whether it can be extensible, whether it can evolve over time gracefully.”</p> </blockquote> <p>Larry referred to the expressiveness of human languages as his inspiration for Raku:</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <ul class="wp-block-list"> <li>Languages evolve over time. (“It’s okay to have dialects…”)</li> <li>No arbitrary limits. (“And they naturally cover multiple paradigms”)</li> <li>External influences on style.</li> <li>Fractal dimensionality.</li> <li>Easy things should be easy, hard things should be possible.</li> <li>“And, you know, if you get really good at it, you can even speak CompSci.”</li> </ul> </blockquote> <p>Naoum Hankache’s <a href="https://raku.guide/#_what_is_raku">Raku Guide</a> introduces Raku:</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <p class="has-normal-font-size">Raku is a high-level, general-purpose, gradually typed language. Raku is multi-paradigmatic. It supports Procedural, Object Oriented, and Functional programming.</p> </blockquote> <p>The feature list on <a href="https://raku.org/">Raku.org</a> goes further:</p> <ul class="wp-block-list"> <li>Object-oriented programming including generics, roles and multiple dispatch</li> <li>Functional programming primitives, lazy and eager list evaluation, junctions, autothreading and hyperoperators (vector operators)</li> <li>Parallelism, concurrency, and asynchrony including multi-core support</li> <li>Definable grammars for pattern matching and generalized string processing</li> <li>Optional and gradual typing</li> </ul> <h3 class="wp-block-heading" id="headline-1">In Search of the Essence</h3> <p>As we can see, Raku has many rich features. Maybe the essence of Raku lies somewhere among them.</p> <h4 class="wp-block-heading" id="headline-2">Deep Unicode</h4> <p>It’s the 21st Century and programming languages are finally waking up to the fact that we live in a multi-lingual world. Raku is a <a href="https://docs.raku.org/language/unicode">Unicode centric language</a> in a way that no other language manages.</p> <h4 class="wp-block-heading" id="headline-3">Rich Concurrency</h4> <p>Again programming languages are only beginning to catch up with the reality that every computer has multiple cores. Sometimes very many cores. Raku excels, alongside other modern languages, at delivering the tools required for effective concurrent and asynchronous programs.</p> <ul class="wp-block-list"> <li>Reactive programming</li> <li><a href="https://docs.raku.org/language/concurrency#Channels">Channels</a> and <a href="https://docs.raku.org/language/concurrency#Supplies">Supplies</a></li> <li>Async / await</li> <li><a href="https://docs.raku.org/language/statement-prefixes#hyper,_race">Hyper, race and vectorization</a></li> </ul> <h4 class="wp-block-heading" id="headline-4">Multi Paradigm</h4> <p>Raku is a richly functional language that combines the procedural and object oriented encapsulation constructs to span paradigms.</p> <ul class="wp-block-list"> <li><a href="https://docs.raku.org/language/functions#Subroutines">Procedural</a></li> <li><a href="https://docs.raku.org/language/functions#Functions_are_first-class_objects">Functional</a></li> <li><a href="https://docs.raku.org/language/objects">Object Oriented</a></li> </ul> <h4 class="wp-block-heading" id="headline-5">A Cornucopia of Language Features</h4> <ul class="wp-block-list"> <li><a href="https://docs.raku.org/language/glossary#gradual_typing">Gradual typing</a></li> <li><a href="https://docs.raku.org/language/list#Lazy_lists">Lazy evaluation</a></li> <li><a href="https://docs.raku.org/language/glossary#Multi-dispatch">Multi dispatch</a> with longest token matching</li> <li><a href="https://docs.raku.org/language/numerics#Rational">Rational Numbers</a></li> <li><a href="https://docs.raku.org/language/grammars">Grammars</a></li> <li><a href="https://docs.raku.org/language/phasers">Phasers</a></li> </ul> <h4 class="wp-block-heading" id="headline-6">Multi Language with Slangs and Inlines</h4> <ul class="wp-block-list"> <li>Many languages in the same execution unit. No need to be an async/ IPC boundary away.</li> <li>Representational polymorphism</li> </ul> <p>The breadth of the Raku language is indeed impressive. But I don’t think it’s instructive, or even possible to try and distill an essence from such a diverse list of capabilities.</p> <h3 class="wp-block-heading" id="headline-7">What then, is the Essence of Raku?</h3> <p>Perl and Raku both adhere to the famous motto:</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <p class="has-normal-font-size">TMTOWTDI (Pronounced Tim Toady): There is more than one way to do it.</p> </blockquote> <p>This is indeed demonstrably true as shown by Damian Conway whenever he <a href="https://blogs.perl.org/users/damian_conway/2019/09/to-compute-a-constant-of-calculusa-treatise-on-multiple-ways.html">blogs about</a> or <a href="https://github.com/thoughtstream/Resources/">talks about</a> Raku. Indeed in <a href="https://blogs.perl.org/users/damian_conway/2019/05/why-i-love-perl-6.html">Why I love Raku</a> he sums up with this endorsement:</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <p class="has-normal-font-size">More than any other language I know, Raku lets you write code in precisely the way that suits you best, at whatever happens to be your (team’s) current level of coding sophistication, and in whichever style you will later find most readable …and therefore easiest to maintain.</p> </blockquote> <p>How do you identify the essence of a deliberately broad multi-paradigm language? Perhaps the essence is not so much about some aspect of the core language. Perhaps it is much more about the freedom it gives you to choose, instead of the language choosing for you.</p> <p>Like Raku, the community is not opinionated. If you choose a particular approach, the community is there to help you. If you want to learn what’s possible, the community will offer up multiple approaches, as shown by many Stack Overflow answers.</p> <p>Which leaves me with this thought:</p> <p>The essence of Raku is the <strong>freedom</strong> to choose for yourself. And the freedom to choose differently tomorrow.</p> <h2><a href="https://raku-advent.blog/2024/12/23/day-23-santas-print-shop/">Day 23 – Santa’s Print Shop</h2></a> <p><h3>Published by Tom Browder (@tbrowder) on 2024-12-23T01:00:00</h3></p> <p><strong>A lead-free area</strong></p> <p>Elf Nebecaneezer (‘Neb’) was welcoming some young new workers to his domain and doing what old folks like to do: pontificate. (His grandchildren politely, but behind his back, call it “bloviating.”)</p> <p>“In the old days, we used hand-set lead type, then gradually used ever-more-modern methods; now we prepare the content in PDF and send a copy to another company to print the content on real newsprint. It still takes a lot of paper, but much less ink and labor (as well as being safer<sup data-fn="d8f195cb-163d-45c1-99d2-05655dbebb44" class="fn"><a href="#d8f195cb-163d-45c1-99d2-05655dbebb44" id="d8f195cb-163d-45c1-99d2-05655dbebb44-link">1</a></sup>).”</p> <p>“Most young people are very familiar with HTML and online documents, but, unless your browser and printer are very fancy, it’s not easy to create a paper copy of the thing you need to file so that it’s very usable. One other advantage of hard-copy products is that they can be used without the internet or electricity. Remember, the US had a hugely-bad day recently when a major internet service provider had a problem!” [He continued to speak…]</p> <h2 class="wp-block-heading">Inspiration and perspiration</h2> <p><a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#inspiration-and-perspiration"></a>Now let’s get down to ‘brass tacks.’ You all are supposed to be competent Raku users, so I will show you some interesting things you can do with PDF modules. But first, a little background on PDF (Adobe’s Portable Document Format).</p> <p>The PDF was developed by the Adobe company in 1991 to replace the PostScript (PS) format. In fact, The original PS code is at the heart of PDF. Until 2008, Adobe defined and continued to develop PDF. Beginning in 2008, the ISO defined it in its 32000 standard. That standard is about a thousand pages long and can be obtained online at <a href="https://www.pdfa-inc.org/">https://www.pdfa-inc.org</a> at no cost.</p> <p>One other important piece of the mix is the CLI <code>ghostscript</code> interpreter which can be used, among other things, to compress PDF documents.</p> <p>Before we continue, remember, we are a Debian shop, and any specific system software I mention is available in current versions.</p> <h2 class="wp-block-heading">Document creation</h2> <p><a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#document-creation"></a>We are still improving Rakudoc (see the <a href="https://raku-advent.blog/2024/12/14/day-14-playing-around-with-the-new-documentation-components/">recent Raku Advent</a> post by <em>Richard Hainsworth</em>), but it can be used now for general text to produce PDF output. However, for almost any specific printed product needed, we can create the PDF on either a one-time case or a specific design for reuse.</p> <p>A good example of that is the module <code><a href="https://raku.land/zef:tbrowder/PDF::NameTags">PDF::NameTags</a></code> which can be modified to use different layouts, colors, fonts, images, and so forth. Its unique output feature is its reversibility–when printed two-sided, the person’s name shows regardless of whether the badge flips around or not. That module was used to create the name tags we are are wearing on our lanyards, and I created that module! I’ll be using parts of that module as an example as I continue.</p> <p>Actually, I have a GitHub account under an alias, ‘tbrowder’, and I have published several useful modules to help my PDF work. I’ll mention some later.</p> <p>Note I always publish my finished modules via ‘zef’, the standard Raku package manager. Then they are easily available for installation, and they automatically get listed on <a href="https://raku.land/">https://Raku.land</a> and can easily be found. Why do I do that, especially since no one else may care? (As ‘@raiph’ or someone other Raku user once said, modules are usually created for the author’s use.) Because, if it helps someone else, as the Golden Rule says, it may be the right thing to do (as I believe <code><a href="https://github.com/tbrowder/Font-Utils">Font::Utils</a></code> does).</p> <h2 class="wp-block-heading">Fonts</h2> <p><a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#fonts"></a>Fonts used to be very expensive and hard to set type with, so the modern binary font is a huge improvement! Binary fonts come in different formats, and their history is fascinating.</p> <p>This shop prefers OpenType fonts for two practical reasons: (1) they provide extensive Unicode glyph coverage for multi-language use (we operate world-wide as you well know) and (2) they provide kerning for more attractive type setting.</p> <p>By using the <code><a href="https://raku.land/zef:dwarring/HarfBuzz">HarffBuzz</a></code> library, the final PDF will only contain the glyphs actually used by the text (the process is called ‘subsetting’). If a user is not running Debian 12 or later, then he or she can additionally ‘use’ module <code><a href="https://raku.land/zef:tbrowder/Compress::PDF">Compress::PDF</a></code> which provides a subroutine which uses Ghostscript to remove unused glyphs. The module also provides a CLI program, <code>pdf-compress</code>, which does the same thing.</p> <p>For Latin and most European languages we use the following font collections available as Debian packages:</p> <figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>Font</th><th>Debian package</th></tr></thead><tbody><tr><td>GNU Free Fonts</td><td>fonts-freefont-otf</td></tr><tr><td>URW Base 35,</td><td>fonts-urw-base35</td></tr><tr><td>E B Garamond</td><td>fonts-egaramond, fonts-garamond-extra</td></tr><tr><td>Cantarell</td><td>fonts-cantarell</td></tr></tbody></table></figure> <p>For other languages we rely on the vast glyph coverage of Google’s Noto fonts (in TrueType format). Debian has many of those fonts, but we find it easier to find the needed font at Google and download them onto our server as zip archives. After unzipping, we move the desired files to ‘/usr/local/share/fonts/noto’. We currently have these 10 Noto fonts available (plus 175 other variants):</p> <figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>Font</th></tr></thead><tbody><tr><td>NotoSerif-Regular.ttf</td></tr><tr><td>NotoSerif-Bold.ttf</td></tr><tr><td>NotoSerif-Italic.ttf</td></tr><tr><td>NotoSerif-BoldItalic.ttf</td></tr><tr><td>NotoSans-Regular.ttf</td></tr><tr><td>NotoSans-Bold.ttf</td></tr><tr><td>NotoSans-Italic.ttf</td></tr><tr><td>NotoSans-BoldItalic.ttf</td></tr><tr><td>NotoSansMono-Regular.ttf</td></tr><tr><td>NotoSansMono-Bold.ttf</td></tr></tbody></table></figure> <p>Note the file names above are in the family and style order as the Free Fonts in our own <code>$HOME</code> directory.</p> <p>To get a feel for the span of glyphs in some of the fonts, the FreeSerif font has over 1,000 glyphs and can be used for many languages. We typically use that font, and the rest of that font family, for most of our work around Europe and the Americas. For the rest of the world, Google’s Noto fonts should cover all but a tiny fraction of the population.</p> <p>One of the many strengths of Raku is its handling of Unicode as a core entity. You can read about <a href="https://home.unicode.org">Unicode</a> at its website. Of particular interest are the code charts at <a href="https://www.unicode.org/charts">https://www.unicode.org/charts</a>. If you select any chart you will see that the code points are shown in hexadecimal. In Raku the code points are natively in decimal. Fortunately, Raku has a method to do that:</p> <pre class="wp-block-code has-small-font-size"><code># convert decimal 178 to hexadecimal (base 16) say 178.base(16); # OUTPUT: B2 # convert a hexadecimal 'A23' to decimal say 'A23'.parse-base(16); # OUTPUT: 2595</code></pre> <p>Or we can look at Wikipedia where there are charts showing both hexidecimal and decimal code points (<a href="https://en.m.wikipedia.org/wiki/List_of_Unicode_characters">Unicode_chars</a>).</p> <h4 class="wp-block-heading">Font files and utilities<a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#font-files-and-utilities"></a></h4> <ul class="wp-block-list"> <li>module <a href="https://github.com/tbrowder/PDF-FontCollection">PDF::FontCollection</a></li> </ul> <p>Not released yet is my Raku module <code>PDF::FontCollection</code> which encapsulates useful font collections into a single reference list. The module has routines allowing the user to get a loaded font with a short code mnemonically associated with the font collection (a digit, and a one- or two-letter character for its style). It has an installed binary to show its fonts by number, code, and name. However, my most useful module, <code>Font::Utils</code>, has made this module effectively obsolete.</p> <ul class="wp-block-list"> <li>module <code><a href="https://github.com/tbrowder/Font-Utils">Font::Utils</a></code></li> </ul> <p>I now introduce my almost-released module <code>Font::Utils</code> which uses fonts already installed and collects them into a file called <code>font-files.list</code> which is then placed into the user’s <code>$HOME/.Font-Utils</code> directory.</p> <p>Since our servers already have the desired OpenType fonts installed, using <code>Font::Utils</code> is actually more convenient since you can arrange the font list any way you want including: (1) creating your own mnemonic keys for easy reference, (2) deleting or adding data lines, and (3) reordering data lines.</p> <p>Note that the actual OpenType font files are quite large, but a good design will ensure they are not loaded until specifically called for in the program. If they are already loaded, calling the routine will be is a no-op. The <code>Font::Utils</code> module has one such routine.</p> <p>Other non-Latin languages are covered in many freely available font collections, including right-to-left and other orientations along with layouts for users who need that capability (the Noto fonts are a good example). As noted, those can be easily added to your <code>Font::Utils</code> collection.</p> <p>Let’s take a look at the first couple of lines in the default installation of my <code>$HOME/.Font-Utils/font-files.list</code>:</p> <pre class="wp-block-code has-small-font-size"><code># key basename path 1 FreeSerif.otf /usr/share/fonts/opentype/freefont/FreeSerif.otf</code></pre> <p>We see the comments and the first line of data consisting of three fields. The first field is the <strong>unique</strong> code which you may change. The second field is the font file’s basename, and the last field is file font file’s path. You may delete or reorder or add new font entries as you wish.</p> <p>Now let’s say you want to publish some text in Japanese. Oh, you say you don’t know Japanese? And you don’t have a keyboard to enter the characters? No problem!, There is a way to do that.</p> <p>We first find from Wikipedia that some Unicode characters for the Japanese language are in the Hiragan collection, which covers hexadecimal code points <code>3041</code> through <code>3096</code> and <code>3099</code> through <code>309F</code>. Then we create a space-separated string of the characters for each word. We’ll use an arbitrary list of them:</p> <pre class="wp-block-code has-small-font-size"><code>my $jword1 = "3059 306A 306B 306C 305D"; my $jword2 = "3059-305D"; # same as $jword1 but with a hyphen for a range my $jword3 = "306B-306F";</code></pre> <p>Note that we can use a hyphen to indicate a contiguous range of code points. (We could also use decimal code points, but that’s a bit more awkward due to ease and larger number of characters required as well as the confusing use of the joining ‘-‘ with subtraction.)</p> <p>Oops, what font shall we use? I couldn’t find a suitable font with the searches on Debian, so I went online to Google, searched for <code>Hiragana</code>, and found the font with description <code>Noto Serif Japanese</code>.</p> <p>I selected it, downloaded it, and got file <code>Noto_Serif_JP.zip</code>. I created a directory named <code>google-fonts</code> and moved the zip file there where I then unpacked them to get directory <code>Noto_Serif_JP</code> with files:</p> <pre class="wp-block-code has-small-font-size"><code>README.txt NotoSerifJP-VariableFont_wght.ttf OFL.txt static static/NotoSerifJP-Bold.ttf static/NotoSerifJP-SemiBold.ttf static/NotoSerifJP-Medium.ttf static/NotoSerifJP-Light.ttf static/NotoSerifJP-ExtraLight.ttf static/NotoSerifJP-ExtraBold.ttf static/NotoSerifJP-Black.ttf static/NotoSerifJP-Regular.ttf</code></pre> <p>The main font is a variable one, so I tried it to see the results.</p> <h2 class="wp-block-heading">Text<a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#text"></a></h2> <p>There are many ways to lay out text on a page. The most useful for general use is to create reusable text boxes.</p> <h3 class="wp-block-heading">Reusable text boxes<a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#reusable-text-boxes"></a></h3> <pre class="wp-block-code has-small-font-size"><code># define it my PDF::Content::Text::Box $tb .= new( :$text, :$font, :$font-size, :$height, # style it :WordSpacing(5), # extra spacing for Eastern languages ); #...clone it and modify it... $tb.clone(:content-width(200));</code></pre> <p>Use the <code>$page.text</code> context to print a box:</p> <pre class="wp-block-code has-small-font-size"><code>my @bbox; $page.text: { .text-position = $x, $y; # print the box and collect the resulting bounding box coordinates @bbox = .print: $tb;</code></pre> <h2 class="wp-block-heading">Graphics and clipping<a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#graphics-and-clipping"></a></h2> <p>I won’t go into it very much, but you can do almost anything with PDF. The aforementioned <code>PDF::NameTags</code> module has many routines for drawing and clipping. Another of my modules on deck is <code><a href="https://github.com/tbrowder/PDF-GraphicsUtils">PDF::GraphicsUtils</a></code> which will encompass many similar routines as well as the published <code><a href="https://raku.land/zef:tbrowder/PDF::Document">PDF::Document</a></code> module.</p> <h2 class="wp-block-heading">Tricks and hints for module authors<a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#tricks-and-hints-for-module-authors"></a></h2> <h3 class="wp-block-heading">Create a run alias<a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#create-a-run-alias"></a></h3> <p>I was having trouble with the variable syntax needed testing scripts as well as test modules. Raku user and author <em><a href='https://rakuadventcalendar.wordpress.com/mentions/librasteve/' class='__p2-hovercard mention' data-type='fragment-mention' data-username='librasteve'><span class='mentions-prefix'>@</span>librasteve</a></em> suggested creating an alias to do that. The result from my <code>.bash_aliases</code> file;</p> <pre class="wp-block-code has-small-font-size"><code>alias r='raku -I.' # adhoc script Raku run command alias rl='raku -I./lib' # adhoc script Raku run command</code></pre> <h3 class="wp-block-heading">Use a load test<a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#use-a-load-test"></a></h3> <p>I was having problems with modules once and <em>@ugexe</em> suggested always using a load test for checking all modules will compile. Now I always create a test script similar to this:</p> <pre class="wp-block-code has-small-font-size"><code># file: t/0-load-test.t use Test; my @modules = < Font::Utils Font::Utils::FaceFreeType Font::Utils::Misc Font::Utils::Subs >; plan +@modules; for @modules { use-ok $_, "Module $_ can be used okay"; }</code></pre> <p>And run it like this: <code>r t/0*t</code></p> <pre class="wp-block-code has-small-font-size"><code>$ r t/0*t # OUTPUT: 1..4 ok 1 - Module Font::Utils can be used okay ok 2 - Module Font::Utils::FaceFreeType can be used okay ok 3 - Module Font::Utils::Misc can be used okay ok 4 - Module Font::Utils::Subs can be used okay</code></pre> <h3 class="wp-block-heading">‘<code>=finish</code>‘<a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#finish"></a></h3> <p>Use <code>=finish</code> to debug a LONG rakumod file. I use it when I’m in the process of adding a new sub or modifying one and commit some error that causes a panic failure without a clear message. I add the <code>=finish</code> after the first routine and see if it compiles. If it does I move the <code>=finish</code> line to follow the next sub, and so on.</p> <p>When I do get a failure, I have a better idea of where the bad code is. Note I do sometimes have to reorder routines becaause of inter-module dependencies. It’s also a good time to move completely independent routines to a lower-level module as described next.</p> <h3 class="wp-block-heading">Encapsulate small, independent routines<a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#encapsulate-small-independent-routines"></a></h3> <p>Sometimes, as in <code>Font::Utils</code>, I create a single, long file with routines that are dependent on other routines so that it is difficult to tell what is dependent upon what. Then I start creating another, lower-level module that has routines that are non-dependent on higher level modules. You can see that structure in the load test output shown above.</p> <h3 class="wp-block-heading">Use the <code>BEGIN</code> phaser<a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#use-the-begin-phaser"></a></h3> <p>Module authors often need access to the user’s <code>$HOME</code> directory, so use of the <code>BEGIN</code> phaser as a block can make it easier to access it from the <code>Build</code> module, as well as the base modules. Here is that code from the <code>Font::Utils</code> module:</p> <pre class="wp-block-code has-small-font-size"><code>unit module Font::Utils; use PDF::Font::Loader :load-font; our %loaded-fonts is export; our $HOME is export = 0; our $user-font-list is export; our %user-fonts is export; BEGIN { if %*ENV<HOME>:exists { $HOME = %*ENV<HOME>; } else { die "FATAL: The environment variable HOME is not defined"; } if not $HOME.IO.d { die qq:to/HERE/; FATAL: \$HOME directory '$HOME' is not usable. HERE } my $fdir = "$HOME/.Font-Utils"; mkdir $fdir; $user-font-list = "$fdir/font-files.list"; } INIT { if not $user-font-list.IO.r { create-user-font-list-file; } create-user-fonts-hash $user-font-list; create-ignored-dec-codepoints-list; }</code></pre> <p>That <code>BEGIN</code> block creates globally accessible variables and enables easy access for any build script to either create a new user fonts list or check an existing one. The <code>INIT</code> block then uses a routine to create the handy <code>%user-fonts</code> hash after the Raku compilation stage enables it.</p> <p>As <em>@lizmat</em> and others on IRC <strong>#raku</strong> always warn about global variables: danger lurks from possible threaded use. But our ‘use cases’ should not trigger such a problem. However, other routines may, and we have used a fancy module to help the problem: <code><a href="https://raku.land/zef:raku-community-modules/OO::Monitors">OO::Monitors</a></code> by <em>@jnthn</em>. See it used to good effect in the class (monitor) <code>Font::Utils::FaceFreeType</code>.</p> <h3 class="wp-block-heading">Experiment: Embedded links<a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#experiment-embedded-links"></a></h3> <p>Currently the PDF standard doesn’t deal with active links, but my Raku friend <em>David Warring</em> gave me a solution in an email. I think I’ve tried it, and I think it works, but YMMV.</p> <p>David said “Hi Tom, Adding a link can be done. It’s a bit lower level than I’d like and you need to know what to look for. Also needs <code><a href="https://raku.land/zef:dwarring/PDF::API6">PDF::API6</a></code>, rather than <code><a href="https://raku.land/zef:dwarring/PDF::Lite">PDF::Lite</a></code>. I’m looking at the pdfmark documentation. It’s a postscript operator and part of the Adobe SDK. It’s not understood directly by the PDF standard, but it’s just a set of data-structures, so a module could be written for it. I’ll keep looking.”</p> <p>His code:</p> <pre class="wp-block-code has-small-font-size"><code># Example: use PDF::API6; use PDF::Content::Color :&color, :ColorName; use PDF::Annot::Link; use PDF::Page; my PDF::API6 $pdf .= new; my PDF::Page $page = $pdf.add-page; $page.graphics: { .FillColor = color Blue; .text: { .text-position = 377, 515; my @Border = 0,0,0; # disable border display my $uri = 'https://raku.org'; my PDF::Action::URI $action = $pdf.action: :$uri; my PDF::Annot::Link $link = $pdf.annotation( :$page, :$action, # action to follow the link :text("Raku homepage"), # display text (optional) :@Border, ); } } $pdf.save-as: "link.pdf";</code></pre> <h3 class="wp-block-heading">Use modules <code><a href="https://raku.land/zef:skaji/App::Mi6">App::Mi6</a></code> and <code><a href="https://raku.land/zef:tbrowder/Mi6::Helper">Mi6::Helper</a></code><a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#use-modules-appmi6-and-mi6helper"></a></h3> <p>Because of the artistic nature of our work, we often need to create our own modules for new products. In that event, we use module <code>App::Mi6</code> and its binary <code>mi6</code> to ease the labor of managing recurring tasks with module development. By using a logical structure and a <code>dist.ini</code> configuration file, the user can create Markdown files for display on GitHub or GitLab, test his or her test files in directories <code>t</code> and <code>xt</code>, and publish the module on ‘Zef/Fez’ with one command each.</p> <p>By using my module <code>Mi6::Helper</code>, the author can almost instantly create a new module ‘git’ source repository with its structure ready to use app <code>mi6</code> with much boiler plate already complete.</p> <p>I’m also working on a <code>dlint</code> program to detect structural problems in the module’s git repository. It is a linter to check the module repo for the following:</p> <ol class="wp-block-list"> <li>Ensure file paths listed in the <code>META6.json</code> file match those in the module’s <code>/resources</code> directory.</li> <li>Ensure all <code>use X</code> statements have matching <code>depends</code> entries in the <code>META6.json</code> file.</li> <li>Ensure all internal sub module names are correct.</li> </ol> <p>The linter will not correct these problems, but if there is interest, that may be added in a future release.</p> <h2 class="wp-block-heading">Party time: finale<a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#party-time-finale"></a></h2> <p>Neb concluded his presentation:</p> <p>“Bottom line: Raku’s many PDF modules provide fairly easy routines to define any pre-press content needed. Those modules continue to be developed in order to improve ease of use and efficiency. Graphic arts have always been fun to me, but now they are even ‘funner!'”.</p> <h2 class="wp-block-heading">Santa’s Epilogue<a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#santas-epilogue"></a></h2> <p>As I always end these jottings, in the words of Charles Dickens’ Tiny Tim, “<strong>may God bless Us, Every one!</strong>” <em>A Christmas Carol</em>, a short story by Charles Dickens (1812-1870), a well-known and popular Victorian author whose many works include <em>The Pickwick Papers</em>, <em>Oliver Twist</em>, <em>David Copperfield</em>, <em>Bleak House</em>, <em>Great Expectations</em>, and <em>A Tale of Two Cities</em><sup data-fn="e6a5c92c-f4e5-4c10-b82f-40bd621a673c" class="fn"><a href="#e6a5c92c-f4e5-4c10-b82f-40bd621a673c" id="e6a5c92c-f4e5-4c10-b82f-40bd621a673c-link">2</a></sup>.</p> <h2 class="wp-block-heading">Footnotes<a href="https://gist.github.com/tbrowder/3f3a3f33cf4da80d024bf483157917e5#footnotes"></a></h2> <ol class="wp-block-footnotes"><li id="d8f195cb-163d-45c1-99d2-05655dbebb44">Old print shops: When I was in the seventh grade in my mother’s home town of McIntyre, Georgia, where we lived while my Dad was in Japan, we had a field trip to the local print shop of the Wilkinson County News. There the printer had to set lead type by hand, install the heavy frame set of type for the full page on the small, rotary press. Then he hand fed fresh newsprint paper as the apparatus inked the type-frame and then pressed the paper against the inked type. Finally, he removed the inked sheet and replaced it with the next copy. One of the kids noted the sharp pin on the press plate to hold the paper and asked if that ever caused problems. The printer showed us his thumb with a big scar and said that was the mark of an old time press man and just an expected hazard of the trade. <a href="#d8f195cb-163d-45c1-99d2-05655dbebb44-link"><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/21a9.png" alt="↩" class="wp-smiley" style="height: 1em; max-height: 1em;" />︎</a></li><li id="e6a5c92c-f4e5-4c10-b82f-40bd621a673c"><em>A Christmas Carol</em>, a short story by Charles Dickens (1812-1870), a well-known and popular Victorian author whose many works include <em>The Pickwick Papers</em>, <em>Oliver Twist</em>, <em>David Copperfield</em>, <em>Bleak House</em>, <em>Great Expectations</em>, and <em>A Tale of Two Cities</em>. <a href="#e6a5c92c-f4e5-4c10-b82f-40bd621a673c-link"><img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/21a9.png" alt="↩" class="wp-smiley" style="height: 1em; max-height: 1em;" />︎</a></li></ol> <h2><a href="https://raku-advent.blog/2024/12/22/day-22-wrapping-a-christmas-present/">Day 22 – Wrapping a Christmas Present</h2></a> <p><h3>Published by Richard Hainsworth on 2024-12-22T01:00:00</h3></p> <h3 class="wp-block-heading">A few months back</h3> <p><a href="https://gist.github.com/#a-few-months-back"></a>In the smoke-filled (virtual) room of the council of the high (from the smoke) elves, the wizened textualist Geoff, said “All of my stuff is in boxes and containers.” Empty shelves behind him indicated he was moving house.</p> <p>“When you have a complex module,” Geoff continued, “and its difficult to describe how to install it, do all the steps in a container, and show the Dockerfile.”</p> <p>“Aha!” said the newest member, who drew his vorpal sword, and went forth to slay the Jabberwock, aka putting <code>RakuAST::RakuDoc::Render</code> into a container.</p> <blockquote class="wp-block-quote has-small-font-size is-layout-flow wp-block-quote-is-layout-flow"> <p>“Beware the Jabberwock, my son!<br>The jaws that bite, the claws that catch!”</p> </blockquote> <p>After days of wandering the murky jungle of Docker/Alpine/Github/Raku documentation, the baffled elf wondered if he was in another fantasy:</p> <p>“So many rabbit holes to fall down.”</p> <h3 class="wp-block-heading">Rabbit Hole, the First – alpinists beware</h3> <p><a href="https://gist.github.com/#rabbit-hole-the-first---alpinists-beware"></a>Best practice for a container is to chose an appropriate base image. Well obviously, there’s the latest Raku version by the friendly, hard-working gnomes at Rakuland. So, here’s the first attempt at a Dockerfile:</p> <pre class="wp-block-code has-small-font-size"><code>FROM docker.io/rakuland/raku # Copy in Raku source code and build RUN mkdir -p /opt/rakuast-rakudoc-render COPY . /opt/rakuast-rakudoc-render WORKDIR /opt/rakuast-rakudoc-render RUN zef install . -/precompile-install</code></pre> <p>It failed. After peering at the failure message, it seemed that at least one of the dependent modules used by the <code>rakuast-rakudoc-render</code> distribution needs a version of make.</p> <p>That’s easily fixed, just add in <code>build-essentials</code>, the vorpal-sworded elf thought. Something like:</p> <pre class="wp-block-code has-small-font-size"><code>FROM docker.io/rakuland/raku # Install make, gcc, etc. RUN apt-get update -y && \ apt-get install -y build-essential && \ apt-get purge -y # Copy in Raku source code and build RUN mkdir -p /opt/rakuast-rakudoc-render COPY . /opt/rakuast-rakudoc-render WORKDIR /opt/rakuast-rakudoc-render RUN zef install . -/precompile-install</code></pre> <p>Failure! No <code>apt</code>.</p> <p>“How can there not be APT??” the Ubuntu-using elf thought in shock. Turns out that the <code>rakuland/raku</code> image is built on an <em>Alpine</em> base, and Alpine have their own package manager <code>apk</code>.</p> <p>Unfortunately, <em>build-essential</em> is a debian package, but at the bottom of this rabbit hole lurks an <code>apk</code> equivalent package <em>build-base</em>, leading to:</p> <pre class="wp-block-code has-small-font-size"><code>FROM docker.io/rakuland/raku # Install make, gcc, etc. RUN apk add build-base # Copy in Raku source code and build RUN mkdir -p /opt/rakuast-rakudoc-render COPY . /opt/rakuast-rakudoc-render WORKDIR /opt/rakuast-rakudoc-render RUN zef install . -/precompile-install</code></pre> <p>Lo! upon using <a href="https://podman-desktop.io/">the Podman desktop</a> to build an image from the Dockerfile, the process came to a succesful end.</p> <p>But now to make things easier, there needs to be a link to the utility <code>RenderDocs</code>, which takes all the RakuDoc sources from <code>docs/</code> and renders them to <code>$*CWD</code> (unless over-ridden by <code>--src</code> or <code>--to</code>, respectively). It will also render to <em>Markdown</em>unless an alternative format is given.</p> <pre class="wp-block-code has-small-font-size"><code>FROM docker.io/rakuland/raku # Install make, gcc, etc. RUN apk add build-base # Copy in Raku source code and build RUN mkdir -p /opt/rakuast-rakudoc-render COPY . /opt/rakuast-rakudoc-render WORKDIR /opt/rakuast-rakudoc-render RUN zef install . -/precompile-install # symlink executable to location on PATH RUN ln -s /opt/rakuast-rakudoc-render/bin/RenderDocs /usr/local/bin/RenderDocs # Directory where users will mount their documents RUN mkdir /doc # Directory where rendered files go RUN mkdir /to WORKDIR /</code></pre> <p>AND!!! when a container was created using this Dockerfile and <code>run</code> with its own terminal, the utility <strong>RenderDocs</strong> was visible. Running</p> <pre class="wp-block-code has-small-font-size"><code>RenderDocs -h</code></pre> <p>produced the expected output (listing all the possible arguments).</p> <p>Since the entire distribution is included in the container, running</p> <pre class="wp-block-code has-small-font-size"><code>RenderDocs --src=/opt/rakuast-rakudoc-render/docs README</code></pre> <p>will render <em>README.rakudoc</em> in <code>--src</code> to <strong>/to/README.md</strong> because the default output format is <em>MarkDown</em>.</p> <p>“Fab!”, screamed the boomer-generation newbie elf. “It worked”.</p> <p>“Now lets try HTML”, he thought.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> RenderDocs --format=HTML --src=/opt/rakuast-rakudoc-render/docs README </pre></div> <p>Failure: no sass.</p> <p>“<strong>expletive deleted</strong>“, he sighed. “The Jabberwok is not dead!”</p> <p><a href="https://gist.github.com/#rabbit-hole-the-second---architecture-problems"></a>There are two renderers for creating HTML. One produces a single file with minimal CSS so that a normal browser can load it as a file locally and it can be rendered without any internet connection. This renderer is triggered using the option <code>--single</code>. Which the containerised <em>RenderDocs</em> handles without problem.</p> <h3 class="wp-block-heading">Rabbit Hole, the second – architecture problems</h3> <p>But the normal use case is for HTML to be online, using a CSS framework and JS libraries from CDN sources. Since the renderer is more generic, it needs to handle custom CSS in the form of <em>SCSS</em>. This functionality is provided by calling an external program <strong>sass</strong>, which is missing in the container.</p> <p>An internet search yields the following snippet for a container.</p> <pre class="wp-block-code has-small-font-size"><code># install a SASS compiler ARG DART_SASS_VERSION=1.82.0 ARG DART_SASS_TAR=dart-sass-${DART_SASS_VERSION}-linux-x64.tar.gz ARG DART_SASS_URL=https://github.com/sass/dart-sass/releases/download/${DART_SASS_VERSION}/${DART_SASS_TAR} ADD ${DART_SASS_URL} /opt/ RUN cd /opt/ && tar -xzf ${DART_SASS_TAR} && rm ${DART_SASS_TAR} RUN ln -s /opt/dart-sass/sass /usr/local/bin/sass</code></pre> <p>The container image builds nicely, but the <code>RenderDocs</code> command <em><strong>STILL</strong></em> chokes with an unavailable <em>sass</em>.</p> <p>Except that diving into the container’s murky depths with an <code>ls /opt/dart-sass/</code> shows that <code>sass</code> exists!</p> <p>The newbie was stumped.</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <p class="has-small-font-size">So rested he by the Tumtum tree<br>And stood awhile in thought.</p> </blockquote> <p>Turns out that the <em>Alpine</em> distribution uses a different compiler, and the wonderful <em>dart-sass</em> fae provide a suitable binary so a simple change was enough to get <em>sass</em> working in the container.</p> <pre class="wp-block-code has-small-font-size"><code>- ARG DART_SASS_TAR=dart-sass-${DART_SASS_VERSION}-linux-x64.tar.gz + ARG DART_SASS_TAR=dart-sass-${DART_SASS_VERSION}-linux-x64-musl.tar.gz</code></pre> <p><strong>simple</strong> does not mean <em>found at once</em>, but the container contains <em>RenderDocs</em>, which produces <em>markdown</em> and <em>HTML</em> rendered files.</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <p class="has-small-font-size">One, two! One, two! And through and through<br>The vorpal blade went snicker-snack!<br>He left it dead, and with its head<br>He went galumphing back.</p> </blockquote> <p>“I can publish this image so everyone can use it,” the FOSS fanatic elf proclaimed.</p> <p>So the docker container image can be accessed using a <code>FROM</code> or <code>PULL</code> using the URL</p> <pre class="wp-block-code has-small-font-size"><code>docker.io/finanalyst/rakuast-rakudoc-render</code></pre> <h3 class="wp-block-heading">Rabbit Hole, the third – Versions<a href="https://gist.github.com/#rabbit-hole-the-third---versions"></a></h3> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <p class="has-small-font-size">“And hast thou slain the Jabberwock?<br>Come to my arms, my beamish boy!<br>O frabjous day! Callooh! Callay!”</p> </blockquote> <p>“It would be great,” mused the triumphant elf, “if RakuDoc sources, say for a README could be automatically included as the <em>github</em> README.md of repo”.</p> <p>“May be as an action?”</p> <p>Github actions can use containers to process files in a repo. Essentially, in an action, the contents of a repo are copied to a github-workspace, then they can be processed in the webspace, and changes to the workspace have to be committed and pushed back to the repository.</p> <p>With a container, the contents of the workspace need to be made available to the container. Despite some documentation that starting a container in a github action automatically maps the github-workspace to some container directory, the exact syntax is not clear.</p> <p>In order to discover how to deal with the multitude of possibilities, a new version of <em>RenderDocs</em> got written, and a new image generated, and again, and again … Unsurprisingly, between one meal and another, the ever hungry elf forgot which version was being tested.</p> <p>“I’ll just include a <code>--version</code> argument,” thought the elf. “I’ll can ask the Super Orcs!”.</p> <p>And there behold was an <em>SO</em> answer to a similar question, and it was written by no lesser a high elf than the zefish package manager <em>ugexe</em>, not to be confused with the <em>other Saint</em> Nick, the package deliverer.</p> <p>Mindlessly copying the spell fragment into his CLI script as:</p> <pre class="wp-block-code has-small-font-size"><code>multi sub MAIN( :version($v)! = False { say "Using version {$?DISTRIBUTION.meta<version>} of rakuast-rakudoc-render distribution." if $v; }</code></pre> <p>the elf thought all done! “Callooh! Callay!”.</p> <p><em><strong>Except</strong></em> <code>RenderDocs -v</code> generated <em><strong>Any</strong></em>.</p> <p>“SSSSSSSSh-,” the elf saw the ominous shadow of Father Christmas looming, “-ine a light on me”.</p> <p>On the IRC channel, the strong willed <em>Coke</em> pointed out that a script does not have a compile time variable, such as, <strong>$?DISTRIBUTION</strong>. Only a module does.</p> <p>The all-knowing elf wizard <em>@lizmat</em> pointed out that command line scripts should be as short as possible, with the code in a module that exports a <code>&MAIN</code>.</p> <p>Imbibing this wisdom, our protagonist copied the entire script contents of <code>bin/RenderDocs</code> to <code>lib/RenderDocs.rakumod</code>, added a <code>proto sub MAIN(|) is export { {*} }</code>, then made a short command line script with just <code>use RenderDocs</code>.</p> <p>Inside the container terminal:</p> <pre class="wp-block-code has-small-font-size"><code># RenderDocs -v Using version 0.20.0 of rakuast-rakudoc-render distribution.</code></pre> <p>With that last magical idiom, our intrepid elf was ported from one rabbit hole back to the one he had just fallen down.</p> <h3 class="wp-block-heading">Rabbit Hole, the fourth – Actions<a href="https://gist.github.com/#rabbit-hole-the-fourth---actions"></a></h3> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <p class="has-small-font-size">“Beware the Jubjub bird, and shun<br>The frumious Bandersnatch!”</p> </blockquote> <p>“I seem to be going backwards,” our re-ported elf sighed.</p> <p>Once again, the <em>github</em> documentation was read. After much study and heartbreak, our hero discovered a sequence that worked:</p> <ol class="wp-block-list"> <li>Place Lavendar and sandalwood essential oils in a fragrance disperser</li> <li>Prepare a cup of pour over single-origin coffee with a spoon of honey, and cream</li> <li>In a <em>github</em> repo create a directory <code>docs</code> containing a file <code>README.rakudoc</code></li> <li>In the same <em>github</em> repo create the structure</li> </ol> <pre class="wp-block-code has-small-font-size"><code> .github/ workflows/ CreateDocs.yml</code></pre> <ol start="5" class="wp-block-list"> <li>Write the following content to <code>CreateDocs.yml</code></li> </ol> <pre class="wp-block-code has-small-font-size"><code> name: RakuDoc to MD on: # Runs on pushes targeting the main branch push: branches: ["main"] # Allows you to run this workflow manually from the # Actions tab workflow_dispatch: jobs: container-job: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@master with: persist-credentials: false fetch-depth: 0 - name: Render docs/sources uses: addnab/docker-run-action@v3 with: image: finanalyst/rakuast-rakudoc-render:latest registry: docker.io options: -v ${{github.workspace}}/docs:/docs -v ${{github.workspace}}:/to run: RenderDocs</code></pre> <p>After examining the <em>github actions</em> logs, it seemed the rendered files were created, but the repository was not changed.</p> <p>“Perhaps I should have used milk and not cream …” thought our fantasy elf.</p> <p>There is in fact a missing step, committing and pushing from the <em>github-workspace</em> back to the repository. This can be done by adding the following to <code>CreateDocs.yml</code>:</p> <pre class="wp-block-code has-small-font-size"><code>- name: Commit and Push changes uses: Andro999b/<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="b0c0c5c3d8f0c6819e83">[email protected]</a> with: github_token: ${{ secrets.GITHUB_TOKEN }} branch: 'main'</code></pre> <p>Even this did not work! Github refused absolutely to write changes to the repository.</p> <p>The weary elf substituted <em>Lemon grass</em> for <em>Lavender</em> in step 1, and just to be certain changed the repo settings following the instructions from the <a href="https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#configuring-the-default-github_token-permissions">Github grimore </a></p> <ol class="wp-block-list"> <li>select <strong>Settings</strong> in the repo’s main page</li> <li>select <strong>Actions</strong> then <strong>General</strong></li> <li>from the dropdown for <em>GITHUB_TOKEN</em>, select the one for <em>read and write</em> access.</li> <li><strong>Save</strong> settings</li> </ol> <p>The content – at this stage of the tale – of <code>CreateDocs.yml</code> is</p> <pre class="wp-block-code has-small-font-size"><code>name: RakuDoc to MD on: # Runs on pushes targeting the main branch push: branches: ["main"] # Allows you to run this workflow manually from the Actions tab workflow_dispatch: jobs: container-job: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@master with: persist-credentials: false fetch-depth: 0 - name: Render docs/sources uses: addnab/docker-run-action@v3 with: image: finanalyst/rakuast-rakudoc-render:latest registry: docker.io options: -v ${{github.workspace}}/docs:/docs -v ${{github.workspace}}:/to run: RenderDocs - name: Commit and Push changes uses: Andro999b/<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="4e3e3b3d260e387f607d">[email protected]</a> with: github_token: ${{ secrets.GITHUB_TOKEN }} branch: 'main'</code></pre> <p>It worked. “The Christmas present is now available for anyone who wants it”, thought our elf.</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <p class="has-small-font-size">’Twas brillig, and the slithy toves<br>Did gyre and gimble in the wabe:<br>All mimsy were the borogoves,<br>And the mome raths outgrabe.<br>(Jabberwocky, By Lewis Carroll)</p> </blockquote> <p>Remember to <code>git pull</code> for the rendered sources to appear locally as well.</p> <h2 class="wp-block-heading">Rabbit Hole, the fifth – Diagrams</h2> <p>“Wouldn’t it be nice to wrap the present in a ribbon? Why not put diagrams in the Markdown file? “</p> <p>Our elf was on a streak, and fell down another rabbit hole: <em>github</em> does not allow <em>svg</em> in a Markdown file it renders from the repo. “It is impossible,” sighed the tired elf.</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <p class="has-small-font-size">Alice laughed. “There’s no use trying,” she said: “one can’t believe impossible things.”<br>“I daresay you haven’t had much practice,” said the Queen. “When I was your age, I always did it for half-an-hour a day. Why, sometimes I’ve believed as many as six impossible things before breakfast.”<br>(Through the Looking Glass, Lewis Carroll)</p> </blockquote> <p>Diagrams can be created using the <code>dot</code> program of <em>Graphvi</em>z, which is a package that Alpine provides. So, we can create a custom block for RakuAST::RakuDoc::Render that takes a description of a graph, sends it to <code>dot</code>, gets an <em>svg</em> file back and inserts into the output.</p> <p>Except: <em>github</em> will not allow <em>svg</em> directly in a markdown file for security reasons. </p> <p>But: it will allow an <em>svg</em> in a file that is an asset on the repo. So, now all that is needed is to save the svg in a file, reference the file in the text, and copy the asset to the same directory as the Markdown text. </p> <p>Except: the time stamps on the <em>RakuDoc</em> source files and the output files seem to be the same because of the multiple copying from the repo to the actions workspace to the docker container. So: add a <code>--force</code> parameter to <code>RenderDocs</code>.</p> <p>So in Raku impossible things are just difficult.</p> <p>The final content of <code>CreateDocs.yml</code> is now</p> <pre class="wp-block-code has-small-font-size"><code>name: RakuDoc to MD on: # Runs on pushes targeting the main branch push: branches: ["main"] # Allows you to run this workflow manually from the Actions tab workflow_dispatch: jobs: container-job: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@master with: persist-credentials: false fetch-depth: 0 - name: Render docs/sources uses: addnab/docker-run-action@v3 with: image: finanalyst/rakuast-rakudoc-render:latest registry: docker.io options: -v ${{github.workspace}}/docs:/docs -v ${{github.workspace}}:/to run: RenderDocs --src=/docs --to=/to --force - name: Commit and Push changes uses: Andro999b/<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="18686d6b70586e29362b">[email protected]</a> with: github_token: ${{ secrets.GITHUB_TOKEN }} branch: 'main'</code></pre> <p>Try adding a graph to a <em>docs/README.rakudoc</em> in a repo, for instance:</p> <pre class="wp-block-code"><code> =begin Graphviz :headlevel(2) :caption<Simple example> digraph G { main -> parse -> execute; main -> init; main -> cleanup; execute -> make_string; execute -> printf init -> make_string; main -> printf; execute -> compare; } =end Graphviz </code></pre> <p>Now you will have a README with an automatic Table of Contents, all the possibilities of RakuDoc v2, and an index at the end (if you indexed any items using <code>X<></code> markup).</p> <p>(Sigh: All presents leave wrapping paper! A small file called <em>semicolon_delimited_script</em> is also pushed by <em>github’s Commit and Push</em> to the repo.)</p> <p></p> <h2><a href="https://rakudo.org/post/announce-rakudo-release-2024.12"> Rakudo compiler, Release #178 (2024.12)</h2></a> <p><h3>Published on 2024-12-14T00:00:00</h3></p> <h2><a href="https://rakujourney.wordpress.com/2024/11/13/raku-perl-a-reconciliation/">raku & perl – A Reconciliation</h2></a> <p><h3>Published by librasteve on 2024-11-13T22:18:50</h3></p> <p>As I <a href="https://rakujourney.wordpress.com/2024/10/27/raku-htmlfunctional/">reported</a> recently, there seems to be a mutual acceptance of raku and perl in the wake of the name change and a bit of time passing:</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <p>This is a reproduction of the talk I gave today at the awesome <strong>London Perl & Raku Workshop</strong>. I met a couple of cool rakuteers and enjoyed the relaxed congregation of perl and raku folk. Seems like the anger has subsided not least thanks to the name change from perl6 to raku. Thanks to all the sponsors, organisers and attendees.</p> </blockquote> <p>One of the questions I was asked at the workshop by some professional perl coders was “what is Raku and why should I use it?”. I was quite flustered by the question (despite having prepared for it the day before).</p> <ul class="wp-block-list"> <li>What I should have said: <ul class="wp-block-list"> <li><em>raku is awesome for LLM – just google <strong>raku LLM</strong></em></li> </ul> </li> <li>What I actually said: <ul class="wp-block-list"> <li><em>raku Grammars are a unique feature … suggest you explore that</em></li> </ul> </li> <li>What I truly think: <ul class="wp-block-list"> <li><em>in business, the best use cases for raku are for ad hoc scripts, data munging, migration and CLI stuff (just like perl was back in the day) – and maybe a bit of LLM prompt engineering</em></li> </ul> </li> </ul> <p>OK – I’m gonna unpack that last statement a bit. By best, I mean that raku is still not really used at scale or for large projects by large teams. There are many things to recommend raku for these kind of use cases – concurrency, strong types (optional), role-based composition, unicode and so on. Stability, bug stomping and performance are improving all the time.</p> <p>But, in addition to great features, any innovative language has to be proven before a real business will be happy to commit to it as a fundamental technical building block. Businesses are, rightly, risk averse. They will consider aspects such as availability of skilled staff, core team bus factor, ecosystem health, etc. So best, for me, means most practical, most likely to succeed, most added value when real-world constraints are applied.</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"></blockquote> <h2 class="wp-block-heading">WordPress<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2122.png" alt="™" class="wp-smiley" style="height: 1em; max-height: 1em;" /></h2> <p>I run a web design consultancy – we focus on WordPress<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2122.png" alt="™" class="wp-smiley" style="height: 1em; max-height: 1em;" /> and other PHP based technologies. Have I rewritten WordPress<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2122.png" alt="™" class="wp-smiley" style="height: 1em; max-height: 1em;" /> in raku? No indeed. But, since I am an all in rakuteer I have effectively used raku to streamline our business processes:</p> <ul class="wp-block-list"> <li>Used raku XML to extract, transform and load 10,000s of ancient PDF files only accessible via a dying web frontend into <a href="https://pkp.sfu.ca/software/ojs/">OJS</a></li> <li>Used raku <a href="https://github.com/librasteve/raku-CLI-AWS-EC2-Simple">CLI::AWS::EC2-Simple</a> to manage a suite of AWS EC2 instances with regard to terminating/relaunching EC2 servers quickly to save cost (and to tune instance types up and down) – as posted <a href="https://rakujourney.wordpress.com/2022/12/29/raku-cli-aws-postvent/">here</a>.</li> <li>Used raku <a href="https://github.com/librasteve/raku-CLI-Wordpress">CLI::Wordpress</a> to create WordPress<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2122.png" alt="™" class="wp-smiley" style="height: 1em; max-height: 1em;" /> hosting sites on AWS EC2 … launch a new instance of WordPress<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2122.png" alt="™" class="wp-smiley" style="height: 1em; max-height: 1em;" /> & setup ssl certificate with three commands:- <code>rawp setup && rawp launch && rawp renewal</code>. This short performs nginx install, database install, WordPress<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2122.png" alt="™" class="wp-smiley" style="height: 1em; max-height: 1em;" /> install, TLS certificate generation, cron for certificate renewal – yes all that</li> <li>Used raku<a href="https://github.com/librasteve/raku-CLI-Wordpress-Migrator"> CLI::Wordpress::Migrator</a> to migrate production WordPress<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2122.png" alt="™" class="wp-smiley" style="height: 1em; max-height: 1em;" /> sites from AWS EC2 to another hosting provider (<a href="http://kualo.com">kualo.com</a> since you ask) – so db and file backup, download, upload, install, restore, search-replace all with <code>rawm migrate</code> (and a suitable yaml config)</li> </ul> <p>Why do I write and share these modules as FOSS? </p> <p>I like to code in raku – it is truly a pleasure. When I assemble and prove out a set of CLI commands to do a task, I am thinking “this is cool, how can I capture this recipe and run it automatically every time” (ie. I a way to remember what works and refine it). And I hope that by sharing others will be able to benefit from these potted scripts and may wish to extend and refine them in turn.</p> <h2 class="wp-block-heading">CLI::Wordpress::Migrator</h2> <p>In common with the other raku modules listed above, this one works like this:</p> <ul class="wp-block-list"> <li>read config.yaml</li> <li>connect to a server</li> <li>upload a perl script to the server</li> <li>go <code>cat xxx.pl | perl</code> to run it</li> <li>control and monitor the perl with raku on the local machine</li> </ul> <p>CLI::Wordpress::Migrator is a script to migrate a WordPress site from one server to another. This performs export (backup), install, import (restore) and search-replace steps according to the configuration in <code>~/.rawm-config/rawm-config.yaml</code></p> <p>This module installs the raku <code>rawm</code> command for use from the local client command line. (RAku WordPress Migrator).</p> <p>The process involves three systems:</p> <ul class="wp-block-list"> <li><code>from</code> server which is running the source site</li> <li><code>to</code> server which is ready for the migrated site</li> <li>local client which connects to the export / import servers in turn</li> </ul> <p>Here’s the raku MAIN usage:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: perl; gutter: false; title: ; notranslate"> > rawm Usage: ./rawm [--ts=<Str>] [--backup-only] [--download-only] [--upload-only] [--restore-only] [--cleanup-only] [--dry-run] <cmd> <cmd> One of <connect export install import search-replace migrate> --ts=<Str> Enter timestamp of files [Str] eg. --ts='20241025-17-02-42' --backup-only Only perform remote backup --download-only Only perform download (requires timestamp [--ts]) --upload-only Only perform upload (requires timestamp [--ts]) --restore-only Only perform restore (requires timestamp [--ts]) --cleanup-only Only perform remote cleanup --dry-run Do not perform replace </pre></div> <p>Here’s the (sanitised) yaml:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: yaml; gutter: false; title: ; notranslate"> from: user: myusername subdom: sdname domain: mydomain.com key-pub: kpname port: 22 to: user: myusername subdom: sdname domain: mydomain.com key-pub: kpname port: 22 wp-config: locale: en_GB db: name: dbname user: dbuser pass: dbpass prefix: wp_ title: My New WP Installation url: mysite.com admin: user: aduser pass: adpass email: ademail </pre></div> <h2 class="wp-block-heading">Under The Hood</h2> <p>Here’s the <strong>perl</strong> code to perform the remote backup:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: perl; gutter: false; title: ; notranslate"> method perl { my $code = q:to/END/; #!/usr/bin/perl use strict; use warnings; print "Doing remote backup at %NOW%\n"; `wp db --path='../%WP-DIR%' export %BU-DB-FN%`; `tar -czf %BU-FS-FN% ../%WP-DIR%/wp-content`; END $code ~~ s:g/'%NOW%' /{ $.timestamp}/; $code ~~ s:g/'%BU-DB-FN%'/{ $.bu-db-fn }/; $code ~~ s:g/'%BU-FS-FN%'/{ $.bu-fs-fn }/; $code ~~ s:g/'%WP-DIR%' /{ $.server.wp-dir }/; $code } </pre></div> <p>And here’s the <strong>raku</strong> code that runs it:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: perl; gutter: false; title: ; notranslate"> method backup { my $s := $.server; my $proc = Proc::Async.new: :w, qqw|ssh -p { $s.port } -tt -i { $s.key-path } { $s.login }|; my $promise = $proc.start; $proc.say("mkdir { $s.tp-dir }"); $proc.say("cd { $s.tp-dir }"); $proc.say("echo \'{ $.perl }\' > exporter.pl"); $proc.say('cat exporter.pl | perl'); sleep 30; $proc.say("exit"); await $promise; } </pre></div> <p>This is a snippet of the source code at <a href="https://github.com/librasteve/raku-CLI-Wordpress-Migrator">https://github.com/librasteve/raku-CLI-Wordpress-Migrator</a> if you want to see the full story.</p> <p>You are welcome to review it all, fork, PR any changes you would like and use it to manage your WordPress<img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/2122.png" alt="™" class="wp-smiley" style="height: 1em; max-height: 1em;" /> estate!</p> <h2 class="wp-block-heading">Why perl? Why raku?</h2> <p>If you have been paying close attention to my raku module history you will know that often I have the opportunity to install raku on the remote machine and to run that for various tasks (i.e. <code>raws-ec2 setup</code>). But in the case of <code>Migrator</code> the idea is to backup and restore remote machines hosted by a third party firm running cPanel and with tight restrictions on installing non standard languages. Happily, as with pretty much all modern Linux installs, their hosted systems are preloaded with perl. So automating the process of logon, save generated perl file on the remote system and run it is very widely applicable.</p> <p>Meantime, driving this with raku is a very natural choice. Raku features that facilitate this are:</p> <ul class="wp-block-list"> <li>simple out of the box command line execution</li> <li>easy OO that can marshal cmd context and yaml config</li> <li>easy shell out via <code>qqx</code></li> <li>regex text substitution to generate custom perl</li> <li>async framework for coupling to remote processes</li> </ul> <p>Ultimately, technically, perl and raku are very complementary – combining the ubiquity of perl with the expressivity of raku to produce a practical outcome. And both have a familiar look and feel…</p> <p>As usual, comments and feedback very welcome!</p> <p>~librasteve </p> <h2><a href="https://dev.to/melezhik/go-pipelines-with-raku-interfaces-j7o">Go pipelines with Raku interfaces</h2></a> <p><h3>Published by Alexey Melezhik on 2024-11-05T20:47:34</h3></p> <p>So, let's say you're golang developer and want pure Go to write some CICD task:</p> <p><code>cat task.go</code><br> </p> <div class="highlight js-code-highlight"> <pre class="highlight go"><code><span class="k">package</span> <span class="n">main</span> <span class="k">import</span> <span class="s">"fmt"</span> <span class="k">func</span> <span class="n">main</span><span class="p">()</span> <span class="p">{</span> <span class="n">fmt</span><span class="o">.</span><span class="n">Println</span><span class="p">(</span><span class="s">"Hello, pipeline"</span><span class="p">)</span> <span class="p">}</span> </code></pre> </div> <p>Go is cool, but there is one thing that makes it difficult to use in high level scenarios - its verbosity. Passing parameters to go tasks and return them back to main scenario takes some efforts and a lot of boilerplate code. It'd be good to keep main code concise and easy to read. </p> <p><a href="https://raku.org" rel="noopener noreferrer">Rakulang</a> in other hand is perfect language when it comes to munge data in and out, due it's extreme flexibility and expressiveness. </p> <p>In this post I am going to show how to embed golang tasks into CICD pipelines, with a little help of Sparrow framework.</p> <p>Let's, first of all, modify our golang task code, new version will be:</p> <p><code>cat task.go</code><br> </p> <div class="highlight js-code-highlight"> <pre class="highlight go"><code><span class="k">package</span> <span class="n">main</span> <span class="k">import</span> <span class="p">(</span> <span class="s">"fmt"</span> <span class="s">"github.com/melezhik/sparrowgo"</span> <span class="p">)</span> <span class="k">func</span> <span class="n">main</span><span class="p">()</span> <span class="p">{</span> <span class="k">type</span> <span class="n">Params</span> <span class="k">struct</span> <span class="p">{</span> <span class="n">Message</span> <span class="kt">string</span> <span class="p">}</span> <span class="k">type</span> <span class="n">Result</span> <span class="k">struct</span> <span class="p">{</span> <span class="n">Message</span> <span class="kt">string</span> <span class="p">}</span> <span class="k">var</span> <span class="n">params</span> <span class="n">Params</span> <span class="c">// sparrowgo takes care about passing </span> <span class="c">// input parameters from Raku to Go</span> <span class="n">sparrowgo</span><span class="o">.</span><span class="n">Config</span><span class="p">(</span><span class="o">&</span><span class="n">params</span><span class="p">)</span> <span class="c">// read params from pipeline</span> <span class="n">fmt</span><span class="o">.</span><span class="n">Printf</span><span class="p">(</span><span class="s">"Task params: %s</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">params</span><span class="o">.</span><span class="n">Message</span><span class="p">)</span> <span class="c">// return results back to pipeline </span> <span class="n">sparrowgo</span><span class="o">.</span><span class="n">UpdateState</span><span class="p">(</span><span class="o">&</span><span class="n">Results</span><span class="p">{</span><span class="n">Message</span> <span class="o">:</span> <span class="s">"Hello from Go"</span><span class="p">})</span> <span class="p">}</span> </code></pre> </div> <p>All we've done here is utilized Sparrowgo package that "convert" golang task into Sparrow task with a benefits of passing and returning data from and to Rakulang. </p> <p>Finally this how our pipeline will look like, now it's Raku part:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>#!raku my $s = task-run ".", %( :message<Hello from Raku> ); say "Result: ", $s<Message>; </code></pre> </div> <p>High level design.</p> <p>Now, once we have some prove of concept code in place, we can get a high level picture of what our pipeline system could look like:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code> [ Raku scenario to pass and handle data in and out ] \ \ \ task.go -> result -> task.go -> result -> task.go -> ... </code></pre> </div> <p>So, we have the best of two worlds - Raku to write scenarios with less of code and Golang to do all heaving lifting where performance and strict type checking is required. </p> <h2><a href="https://andrewshitov.com/2024/11/02/a-language-a-day/">A Language a Day</h2></a> <p><h3>Published by Andrew Shitov on 2024-11-02T10:15:02</h3></p> <p>I published my new book: A Language a Day, which is a collection of brief overviews to 21 programming languages.</p> <figure class="wp-block-image size-large"><img src="https://andrewshitov.com/wp-content/uploads/2024/11/cover-view-727x1024.jpg" alt="" class="wp-image-7125" srcset="https://andrewshitov.com/wp-content/uploads/2024/11/cover-view-727x1024.jpg 727w, https://andrewshitov.com/wp-content/uploads/2024/11/cover-view-213x300.jpg 213w, https://andrewshitov.com/wp-content/uploads/2024/11/cover-view-768x1082.jpg 768w, https://andrewshitov.com/wp-content/uploads/2024/11/cover-view-1091x1536.jpg 1091w, https://andrewshitov.com/wp-content/uploads/2024/11/cover-view-1454x2048.jpg 1454w, https://andrewshitov.com/wp-content/uploads/2024/11/cover-view-1200x1690.jpg 1200w, https://andrewshitov.com/wp-content/uploads/2024/11/cover-view-scaled.jpg 1818w" sizes="(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px" /></figure> <p>This book provides a concise overview of 21 different programming languages. Each language is introduced using the same approach: solving several programming problems to showcase its features and capabilities. Languages covered in the book: C++, Clojure, Crystal, D, Dart, Elixir, Factor, Go, Hack, Hy, Io, Julia, Kotlin, Lua, Mercury, Nim, OCaml, Raku, Rust, Scala, and TypeScript.</p> <p>Each chapter covers the essentials of a different programming language. To make the content more consistent and comparable, I use the same structure for each language, focusing on the following mini projects:</p> <ol><li><em>Creating a ‘Hello, World!’ program.</em></li><li><em>Implementing a Factorial function using recursion or a functional-style approach.</em></li><li><em>Creating a polymorphic array of objects (a ‘zoo’ of cats and dogs) and calling methods on them.</em></li><li><em>Implementing the Sleep Sort algorithm—while impractical for real-word use, it’s a playful demonstration of language’s concurrency capabilities.</em></li></ol> <p>Each language description follows—where applicable—this pattern:</p> <ol><li><em>Installing a command-line compiler and running a program.</em></li><li><em>Creating and using variables.</em></li><li><em>Defining and using functions.</em></li><li><em>Exploring object-oriented features.</em></li><li><em>Handling exception.</em></li><li><em>Introducing basic concurrency and parallelism.</em></li></ol> <p>You can find all the code examples in this book on GitHub: <a href="https://github.com/ash/a-language-a-day">github.com/ash/a-language-a-day</a>.</p> <p>You can buy it on Amazon or LeanPub as an electronic or Kindle edition, or as a paper hardcover or paperback version. <a href="https://andrewshitov.com/a-language-a-day/">More information with the links to the shops</a>.</p> <h2><a href="https://andrewshitov.com/2024/10/29/london-perl-raku-workshop-2024/">London Perl & Raku Workshop 2024</h2></a> <p><h3>Published by Andrew Shitov on 2024-10-29T17:59:30</h3></p> <p>Last Saturday, October 26, 2024, another edition of the <a href="https://act.yapc.eu/lpw2024/">London Perl & Raku Workshop</a> took place. It was a reopening after a few years break. London Perl Workshop has a long history, started in 2004, so this year it was formally 20 years old.</p> <figure class="wp-block-image size-large"><img src="https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0822-1024x677.jpg" alt="" class="wp-image-7104" srcset="https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0822-1024x677.jpg 1024w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0822-300x198.jpg 300w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0822-768x507.jpg 768w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0822-1200x793.jpg 1200w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0822.jpg 1500w" sizes="(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px" /></figure> <h4>The Covid time</h4> <p>For me, it was also a Perl-and-or-Raku event after a break. The last in-person event was <a href="https://perlcon.eu/">PerlCon 2019</a>, which was an annual European Perl conference that was held in Riga that year. Since then, after the <a href="https://covid.observer">Covid</a>’s strike, I have organised a couple of (well, three of them) <a href="https://conf.raku.org">Raku Conferences</a>, but that was online in Zoom. Online due to lockdown first, and then due to people slowly reinventing the joy of travel.</p> <p>In-between, there were at least two in-person gatherings, another edition of the <a href="http://act.yapc.eu/gpw2024/">German Perl and Raku Workshop</a> in April and the <a href="https://tprc.us/tprc-2024-las/">American Perl and Raku Conference in Las Vegas</a> in June. I did not attend any of them, but looked at the photos and videos, so I was envy enough because I missed it.</p> <h4>To go or not to go</h4> <p>You should not forget that right now, right today, at this very moment, an on-going war against my beloved Ukraine takes place. During the first months, it was not possible to think about anything else, not to mention any entertainment trips or conferences. But time keeps going… and you see less and less blue-and-yellow flags on the streets. Even at <a href="https://www.waterstones.com">Waterstones</a>, the books about that war are located at the very bottom shelves due to the unfortunate nature of alphabetical sorting.</p> <figure class="wp-block-image size-large"><img src="https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0627.jpg" alt="" class="wp-image-7072" srcset="https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0627.jpg 1000w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0627-300x197.jpg 300w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0627-768x505.jpg 768w" sizes="(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px" /><figcaption>From Berlin on top through Russia in the middle to Ukraine at the end of the list.<br>Waterstones in Southampton.</figcaption></figure> <p>When this year’s London Perl Workshop was announced, I decided to go. Then, I decided not to go. In the end, one week before the event, I thought that I’d better go. Because of my uncertainty, I did not manage to submit a talk on time, so even a small 5-minute lightning talk submission (which I did on the day of the event, ha-ha), did not have a chance to went through. The venue, apparently, was rented until 6pm sharp, and there was no room for anything else.</p> <p>So, here are the slides of my talks. I’ve added some comments so that you can enjoy it by following my message.</p> <iframe src="https://www.slideshare.net/slideshow/embed_code/key/3m6jmxUJkj9ikz?startSlide=1" width="597" height="486" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC; border-width:1px; margin-bottom:5px;max-width: 100%;" allowfullscreen=""></iframe><div style="margin-bottom:5px"><strong><a href="https://www.slideshare.net/slideshow/perl-jobs-market-in-2024-how-good-is-it/272859502" title="Perl jobs market in 2024, how good is it?" target="_blank" rel="noopener noreferrer">Perl jobs market in 2024, how good is it?</a></strong> from <strong><a href="https://www.slideshare.net/andy.sh" target="_blank" rel="noopener noreferrer">Andrew Shitov</a></strong></div> <h4>London and more</h4> <p>My journey was quite short, I only had a couple of days in London, but the weather made its present and the plane from Amsterdam landed in Southampton instead. So, I took a chance to see the old walls there and to learn that they offer beans on breakfast at IKEA’s canteen.</p> <figure class="wp-block-image size-large"><img src="https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0224-1024x768.jpg" alt="" class="wp-image-7063" srcset="https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0224-1024x768.jpg 1024w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0224-300x225.jpg 300w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0224-768x576.jpg 768w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0224-1200x900.jpg 1200w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0224.jpg 1500w" sizes="(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px" /><figcaption>Southampton</figcaption></figure> <p>I really enjoy London, it’s so huge that you fill extremely comfortable in it, how weird that may sound. Despite only a limited couple of days in London, I tried to use the time to wander around just as a tourist. </p> <figure class="wp-block-image size-large"><img src="https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0765-768x1024.jpg" alt="" class="wp-image-7065" srcset="https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0765-768x1024.jpg 768w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0765-225x300.jpg 225w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0765-1152x1536.jpg 1152w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0765-1200x1600.jpg 1200w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0765.jpg 1500w" sizes="(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 984px) 61vw, (max-width: 1362px) 45vw, 600px" /></figure> <p>I also escaped from the middle of the conference day to see the urban life around venue’s location.</p> <figure class="wp-block-image size-large"><img src="https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0691-1024x768.jpg" alt="" class="wp-image-7064" srcset="https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0691-1024x768.jpg 1024w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0691-300x225.jpg 300w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0691-768x576.jpg 768w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0691-1200x900.jpg 1200w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0691.jpg 1500w" sizes="(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px" /><figcaption>Brick Lane Upmarket foodcourt</figcaption></figure> <h4>Old friends</h4> <p>On the day before the conference, I met with an ex-colleague, with whom I worked at two different places, both Perl-related. For a few years, he’s been living in London, and I noticed that he looked so much happier since I last saw him. I hope that is because he lives in this great city now, not because he’s programming in Go, not Perl anymore.</p> <p>Of course, I was happy to see people I know from the Perl and/or Raku communities. I did not see most of them at least since 2019, so some of them required a few seconds to recognise me, LOL.</p> <h4>The Conference</h4> <p>The workshop was really well attended. Even taking into account the significant drop in the numbers since the beginning of 2000s, it was more than 100 people, and there were two full tracks of the talks.</p> <figure class="wp-block-image size-large"><img src="https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0714-1024x369.jpg" alt="" class="wp-image-7067" srcset="https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0714-1024x369.jpg 1024w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0714-300x108.jpg 300w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0714-768x277.jpg 768w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0714-1536x554.jpg 1536w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0714-2048x739.jpg 2048w, https://andrewshitov.com/wp-content/uploads/2024/10/IMG_0714-1200x433.jpg 1200w" sizes="(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px" /><figcaption>In the main room</figcaption></figure> <p>Let me not list the talks I attended, as the full <a href="https://act.yapc.eu/lpw2024/schedule">schedule is online</a>, and the recordings should appear soon.</p> <p>There was also a third track, partially filled with regular talks, but partially with a broader-scoped events such as <a href="https://act.yapc.eu/lpw2024/talk/7939">Science Perl Talks</a>. I’ve no idea what they talked about there, but it’s somehow connected with the recently-published first issue of the <a href="https://science.perlcommunity.org/spj">Science Perl Journal</a> and a <a href="https://www.reddit.com/r/perl/comments/1g4bnx0/unsolicited_email_from_science_perl_committee/">traditional</a> <a href="https://www.facebook.com/groups/perlprogrammers/posts/8972284462804262/">drama</a> around its establisher.</p> <p>But the appearance of the new publication is a good thing in the end. I was also surprised, that in 2024, a <a href="https://allperlbooks.com/year/2024">few new publications about Perl appeared</a> or <a href="https://allperlbooks.com/year/2025">are planned to be published</a>. I suspect that some of them issued during the previous years are AI-generated texts, but the rest are quite normal and useful books.</p> <h4>The Next years</h4> <p>There are no decision yet whether there will be another London Perl Workshop in 2025. On the bright side, there should be a soon announcement on the date and location of the German Perl Workshop 2025. Also, there are plans to have a Dutch Perl Workshop 2025 in Utrecht.</p> <p>I am thinking of if it’s time to revitalise the annual European Perl and Raku Conference.</p> <h2><a href="https://rakujourney.wordpress.com/2024/10/27/raku-htmlfunctional/">raku HTML::Functional</h2></a> <p><h3>Published by librasteve on 2024-10-27T16:58:25</h3></p> <p><em>This is a reproduction of the talk I gave today at the awesome <a href="https://act.yapc.eu/lpw2024/">London Perl & Raku Workshop</a>. I met a couple of cool rakuteers and enjoyed the relaxed congregation of perl and raku folk. Seems like the anger has subsided not least thanks to the name change from perl6 to raku. Thanks to all the sponsors, organisers and attendees.</em></p> <h2 class="wp-block-heading">Elm Roots</h2> <p>The more observant members of the raku community will notice that I released a new raku module <a href="https://raku.land/zef:librasteve/HTML::Functional">HTML::Functional</a> a couple of weeks back.</p> <p>The general idea for this came from the <a href="https://www.google.com/url?sa=t&source=web&rct=j&opi=89978449&url=https://podcasts.apple.com/gb/podcast/software-unscripted/id1602572955&ved=2ahUKEwj3yNfMz6yJAxVzRUEAHQxyFioQFnoECAgQAQ&usg=AOvVaw1jDhDJDm_XgglYsjtpBwZA">Software Unscripted</a> podcast created by Richard Feldman … one of the dev team on the <a href="https://elm-lang.org">Elm language</a> – <em>A delightful language<br>for reliable web applications.</em></p> <p>Elm is a functional language that streamlines the development of websites by making the code simpler, more composable, more expressive and more maintainable than raw HTML. I want this for raku!</p> <p><em>Actually I want to code my HTML on the server side with raku whereas Elm compiles to JavaScript and runs in the browser.</em> BUT I <strong>do</strong> like the Elm code style.</p> <h2 class="wp-block-heading">Elm Example</h2> <p>Here’s the HTML from the Elm <a href="https://elmprogramming.com/building-a-simple-page-in-elm.html">example</a>…</p> <figure class="wp-block-image size-large"><img width="1024" height="361" data-attachment-id="1320" data-permalink="https://rakujourney.wordpress.com/screenshot-2024-10-26-at-19-35-52/" data-orig-file="https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.35.52.png" data-orig-size="2504,884" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Screenshot 2024-10-26 at 19.35.52" data-image-description="" data-image-caption="" data-medium-file="https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.35.52.png?w=300" data-large-file="https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.35.52.png?w=782" src="https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.35.52.png?w=1024" alt="" class="wp-image-1320" srcset="https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.35.52.png?w=1024 1024w, https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.35.52.png?w=2045 2045w, https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.35.52.png?w=150 150w, https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.35.52.png?w=300 300w, https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.35.52.png?w=768 768w" sizes="(max-width: 1024px) 100vw, 1024px" /></figure> <p><em>[the code examples in this post may be found in <a href="https://github.com/librasteve/raku-HTML-Functional">https://github.com/librasteve/raku-HTML-Functional</a> … they are presented as images here for the syntax highlighting… zoom Cmd+ your browser to get a better view]</em></p> <p>And here it is in Elm…</p> <figure class="wp-block-image size-large"><img width="1024" height="436" data-attachment-id="1321" data-permalink="https://rakujourney.wordpress.com/screenshot-2024-10-26-at-19-39-41/" data-orig-file="https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.39.41.png" data-orig-size="2474,1054" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Screenshot 2024-10-26 at 19.39.41" data-image-description="" data-image-caption="" data-medium-file="https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.39.41.png?w=300" data-large-file="https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.39.41.png?w=782" src="https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.39.41.png?w=1024" alt="" class="wp-image-1321" srcset="https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.39.41.png?w=1024 1024w, https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.39.41.png?w=2048 2048w, https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.39.41.png?w=150 150w, https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.39.41.png?w=300 300w, https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-19.39.41.png?w=768 768w" sizes="(max-width: 1024px) 100vw, 1024px" /></figure> <p>This functional code style is cleaner and more expressive than classic HTML tags. Elm avoids the need to repeat tags at start and end of a block and eliminates visually noisy angle brackets.</p> <h2 class="wp-block-heading">HTML::Functional</h2> <p>And now for the raku…</p> <figure class="wp-block-image size-large"><img width="1024" height="283" data-attachment-id="1324" data-permalink="https://rakujourney.wordpress.com/screenshot-2024-10-26-at-20-39-07/" data-orig-file="https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-20.39.07.png" data-orig-size="1662,460" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Screenshot 2024-10-26 at 20.39.07" data-image-description="" data-image-caption="" data-medium-file="https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-20.39.07.png?w=300" data-large-file="https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-20.39.07.png?w=782" src="https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-20.39.07.png?w=1024" alt="" class="wp-image-1324" srcset="https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-20.39.07.png?w=1024 1024w, https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-20.39.07.png?w=150 150w, https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-20.39.07.png?w=300 300w, https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-20.39.07.png?w=768 768w, https://rakujourney.wordpress.com/wp-content/uploads/2024/10/screenshot-2024-10-26-at-20.39.07.png 1662w" sizes="(max-width: 1024px) 100vw, 1024px" /></figure> <p>To get this module simply go <code>zef install HTML::Functional</code> and then <code>use HTML::Functional;</code> in your header.</p> <p>This is even cleaner than the Elm and employs a lot of raku functional features implicitly:</p> <ul class="wp-block-list"> <li>HTML tags are implemented as raku functions: <code>div</code>, <code>h1</code>, <code>p</code> and so on</li> <li>parens <code>()</code> are optional in raku function calls</li> <li>HTML tag attributes are passed as raku <a href="https://docs.raku.org/language/signatures#Positional_vs._named_arguments"><code>named</code> argument</a>s</li> <li>HTML tag inners (e.g. the Str in <code>h1</code>) are passed as raku<a href="https://docs.raku.org/language/signatures#Positional_vs._named_arguments"> positional argument</a>s</li> <li>the raku Pair syntax is used for each attribute i.e. <code>:name<value></code> </li> <li>multiple <code>@inners</code> are passed as a literal Array [] – <code>div</code> contains <code>h1</code> and <code>p</code></li> <li>the raku parser looks at functions from the inside out, so <code>strong</code> is evaluated before <code>p</code>, before <code>div</code> and so on</li> <li>semicolon <code>;</code> is used as the Array literal separator to suppress nesting of tags</li> </ul> <p>Normally the items in a raku literal Array are comma , separated. Raku precedence considers that div [h1 x, p y]; is equivalent to div( h1(x, p(y) ) ); … so the <code>p</code> tag is embedded within the <code>h1</code> tag unless parens are used to clarify. But replace the comma <code>,</code> with a semi colon <code>;</code> and predisposition to nest is reversed. So div [h1 x; p y]; is equivalent to div( h1(x), p(y) ). Boy that <a href="https://www.google.com/url?sa=t&source=web&rct=j&opi=89978449&url=https://en.wikipedia.org/wiki/Larry_Wall&ved=2ahUKEwiphdTB8ayJAxWiXUEAHc3uMiQQFnoECB0QAQ&usg=AOvVaw3-rytbqYKn1ldeQY87cavB">Larry Wall </a>was smart!</p> <p>The raku example also shows the power of the raku <a href="https://docs.raku.org/language/quoting#The_Q_lang">Q lang</a> at work:</p> <ul class="wp-block-list"> <li>double quotes <code>""</code> interpolate their contents</li> <li>curlies denote an embedded code block <code>"{fn x}"</code></li> <li>tilde <code>~</code> is for Str concatenation</li> <li>the <a href="https://docs.raku.org/syntax/heredocs%20%3Ato">heredoc</a> form <code>q:to/END/;</code> can be used for verbatim text blocks</li> </ul> <h2 class="wp-block-heading">Demo Video</h2> <p>To see this in action, view the demo video:</p> <figure class="wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-4-3 wp-has-aspect-ratio"><div class="wp-block-embed__wrapper"> <div class="embed-youtube"><iframe title="HTMLFunctionalDemo" width="782" height="587" src="https://www.youtube.com/embed/2WPKqVfchs8?feature=oembed" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe></div> </div></figure> <p>The video showcases IntelliJ IDEA Universal IDE with the latest <a href="https://github.com/ab5tract/comma-plugin">comma-plugin2.0</a> courtesy of @ab5stract.</p> <h2 class="wp-block-heading">Conclusions</h2> <p>HTML::Functional does this:</p> <ul class="wp-block-list"> <li>write web pages in Raku with “Elm-like” style</li> <li>exports HTML tag names as raku subs</li> <li>applies HTML tag attrs as raku attrs</li> <li>HTML::Escape for tag text</li> </ul> <p>The clean HTML code it delivers is in tension with modern event driven web frameworks such as React. As we have seen in other <a href="https://rakujourney.wordpress.com/2024/08/22/htmx-and-raku-cro/">posts</a>, HTML::Functional is even more maintainable if used in conjunction with HTMX.</p> <p>And let’s end with a quote (sadly I did not record the originator)…</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <p>I think i just expressed my thought in a wrong way, haha. I am a functional freak, and the first thing i did was check out Raku’s functional patterns. I was amazed. Raku can be extremely functional, but in my opinion language can be called functional when there’s no other way other than functional for the most part. Raku has great functional support, but the language doesn’t force you into anything, you can do basically anything! A sandbox language, and i am loving it.</p> <cite>anon</cite></blockquote> <p>Comments and feedback welcome!</p> <p>~librasteve</p> <h2><a href="https://rakudo.org/post/announce-rakudo-release-2024.10"> Rakudo compiler, Release #177 (2024.10)</h2></a> <p><h3>Published on 2024-10-24T00:00:00</h3></p> <h2><a href="https://rakujourney.wordpress.com/2024/10/12/raku-burritos/">Raku Burritos</h2></a> <p><h3>Published by librasteve on 2024-10-12T00:18:11</h3></p> <p><strong>Late Edit – thanks to a genius <a href="https://rakujourney.wordpress.com/2024/10/12/raku-burritos/#comment-800">suggestion</a> from @wamba, I have made some late changes to improve the code – specifically the concerns I mentioned in v1 about <code>when</code> clause and <code>lisp ((((</code> are now fixed. Excellent input!!</strong></p> <p>Regular followers of this blog will know that I am on a bit of a Functional tack … I’m highly motivated to improve my functional chops because I am giving a talk at the <a href="http://act.yapc.eu/lpw2024/news/1697">London Perl and Raku Conference</a> shortly entitled <code>Raku HTML::Functional<strong>‎</strong></code> and I just realised that the audience is going to be a bunch of deep perl and raku (linux) experts who know Functional coding inside out. Yikes … hope I don’t get any tough questions from folks who really know their stuff!</p> <p>By contrast, I am a dabbler in Functional. I like the feel of using <code>.map</code> and <code>.grep</code> and so on, but I am on the learning curve. And I am resistant to languages that constantly get in my way since largely I am trying to make code that works rather than to wrestle with compile errors. (And no I do not work in large teams since you ask).</p> <p>So when I saw a recent post on HN, written in F#, I felt challenged to work out what was going on and to try and relate to Raku.</p> <h2 class="wp-block-heading"><a href="https://news.ycombinator.com/item?id=41698907">A Burrito is a Monad</a></h2> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: fsharp; auto-links: false; gutter: false; title: ; quick-code: false; notranslate"> type Meat = Chicken | Beef | Pork | Fish | Veggie type Ingredient = | Cheese | Rice | Beans | Salsa | Guacamole | SourCream | Lettuce | Tomato | Onion | Cilantro | PicoDeGallo type Burrito = Meat option * Ingredient list let (>>=) burrito f = match burrito with | Some meat, ingredients -> f (Some meat, ingredients) | None, _ -> None, [] let returnBurrito (meat, ingredients) = meat, ingredients let tortilla = returnBurrito (Some Veggie, []) let addMeat meat (m, ingredients) = Some meat, ingredients let addIngredient ingredient (meat, ingredients) = meat, ingredient :: ingredients let addMissionBurritoIngredients (meat, ingredients) = meat, Cheese :: Rice :: Beans :: ingredients let holdThe ingredient (meat, ingredients) = meat, List.filter (fun i -> i <> ingredient) ingredients let burrito = tortilla >>= addMeat Chicken >>= addMissionBurritoIngredients >>= holdThe Cheese >>= addIngredient PicoDeGallo >>= addIngredient Salsa >>= addIngredient Guacamole >>= addIngredient SourCream printfn "%A" burrito </pre></div> <p>This from the <a href="http://williamcotton.com/articles/a-burrito-is-a-monad">OP</a> by William Cotton</p> <h2 class="wp-block-heading">Definitely Maybe</h2> <p>Since we are talking Monads, I realised that the raku <a href="https://github.com/masukomi/Definitely">Definitely</a> module written by <code>masukomi</code> would come in handy. This module arose from a <a href="https://rakujourney.wordpress.com/2022/08/16/raku-rust-option-some-none/">post</a> I made here some time back, so it was a good time to revisit.</p> <p><a href="https://github.com/librasteve/raku-Burrito/blob/main/burrito-dm.raku">https://github.com/librasteve/raku-Burrito/blob/main/burrito-dm.raku</a></p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: perl; gutter: false; title: ; notranslate"> use Definitely; enum Meat <Chicken Beef Pork Fish Veggie>; enum Ingredient <Cheese Rice Beans Salsa Guacamole SourCream Lettuce Tomato Onion Cilantro PicoDeGallo>; sub returnBurrito($meat, @ingredients) { $meat, @ingredients } sub tortilla { returnBurrito(something(Veggie), []) } sub add-meat($meat, ($, @ingredients)) { something($meat), @ingredients } sub add-ingredient($ingredient, ($meat, @ingredients)) { $meat, [$ingredient, |@ingredients] } sub add-mission-burrito-ingredients(($meat, @ingredients)) { $meat, [Cheese, Rice, Beans, |@ingredients] } sub hold-the($ingredient, ($meat, @ingredients)) { ($meat, [@ingredients.grep(* != $ingredient)]); } multi infix:«>>=»((None $, @), +@ ) is prec(prec => 'f=') { nothing(),[] } multi infix:«>>=»($burrito, +(&f, *@args)) is prec(prec => 'f=') { f( |@args, $burrito ) } tortilla() >>= (&add-meat, Beef) >>= (&add-mission-burrito-ingredients) >>= (&hold-the, Cheese) >>= (&add-ingredient, PicoDeGallo) >>= (&add-ingredient, Salsa) >>= (&add-ingredient, Guacamole) >>= (&add-ingredient, SourCream) ==> say(); </pre></div> <p>I hope that you will agree that Raku does a generally solid job of handling the translation from F#. </p> <p><s>There are a couple of raised eyebrows around the <code>when {...}</code> clauses and the handling of variadic arity of the passed in function in the <code>match</code> and the lisp-like <code>((((((</code> parens in the application of the custom binder. Otherwise, it is pretty smooth. </s>EDIT – NOW FIXED</p> <p>The Definitely module works well here, I have also <a href="https://github.com/librasteve/raku-Burrito/blob/main/burrito-mr.raku">tried</a> with Rawley Fowlers <a href="https://raku.land/zef:rawleyfowler/Monad-Result">Monad::Result</a> module which was similarly successful.</p> <h2 class="wp-block-heading">Monad Chains and Binding</h2> <p>In this self-study, I leaned on the excellent <a href="https://en.wikipedia.org/wiki/Monad_(functional_programming)#An_example:_Maybe">Wikipedia Monad</a> page, which mentions that a true Monad implementation has three operations:</p> <ol class="wp-block-list"> <li>Monadic Type <ul class="wp-block-list"> <li>A type (Maybe)</li> </ul> </li> <li>Unit operation <ul class="wp-block-list"> <li>A type converter (Just(x))</li> </ul> </li> <li>Bind operation <ul class="wp-block-list"> <li>A combinator for monadic functions ( >>= or .flatMap())</li> </ul> </li> </ol> <p>And it shows chaining of this <code>halve</code> function as an example of chaining with the bind operator in Haskell:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: haskell; gutter: false; title: ; notranslate"> halve :: Int -> Maybe Int halve x | even x = Just (x `div` 2) | odd x = Nothing -- This code halves x twice. it evaluates to Nothing if x is not a multiple of 4 halve x >>= halve </pre></div> <p>So, to improve the Definitely module, I have added a binding operator to be used like this:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: perl; gutter: false; title: ; notranslate"> use Definitely; sub halve(Int $x --> Maybe[Int]) { given $x { when * %% 2 { something( $x div 2 ) } when ! * %% 2 { nothing(Int) } } } say (halve 4) >>= &halve; #1 say (something 32) >>= &halve >>= &halve >>= &halve; #4 say (halve 3) ~~ None; #True </pre></div> <p>Note that the Result::Monad module already has bind and map operators provided.</p> <p><s>For now this is a <a href="https://github.com/masukomi/Definitely/pull/3">PR</a>, feel free to install directly from my fork here if you would like to try it.</s> Now released to zef package installer eco-system… </p> <p><code>zef install <a href="https://github.com/librasteve/Definitely.git">https://github.com/librasteve/Definitely.git</a></code></p> <p>As usual all comments and feedback welcome!</p> <p>~librasteve</p> <p></p> <h2><a href="https://rakudo.org/post/announce-rakudo-release-2024.09"> Rakudo compiler, Release #176 (2024.09)</h2></a> <p><h3>Published on 2024-09-26T00:00:00</h3></p> <h2><a href="https://5ab5traction5.bearblog.dev/introducing-comma-20/">Introducing Comma 2.0...</h2></a> <p><h3>Published by 5ab5traction5 on 2024-09-14T16:51:00</h3></p> <p><em><strong>NOTE:</strong> There is an issue when opening existing Comma projects that were created in earlier versions. Please use <code>New project from Existing Sources...</code> rather than <code>Open</code> and make sure to select <code>Yes</code> when it prompts you about overwriting an existing <code>.idea</code> file in the project directory.</em></p> <p>This <a href='https://github.com/ab5tract/comma-plugin'>release</a> represents a major shift for the Comma project in many ways.</p> <p>From the bottom of my heart, I want to express the deepest gratitude and thanks to Jonathan Worthington (<code>jnthn++</code>), Edument, and all past and future contributors to the Comma project. There's been so much effort put into this codebase and it was an honor to be able to work on it.</p> <p>The most major change is the shift to the <code>IntelliJ Platform Gradle Plugin 2.0</code>. This allows Comma to be built (as a plugin) without cloning the <code>intellij-community</code> repo and downloading it's entire dependency tree!</p> <p>This does seem to preclude building Comma as a standalone IDE, at least for the time being. That appears to be a different beast entirely and we will have to investigate that as the time and tuits allow.</p> <p>Other major changes included updating the code to correct for broken and (some) deprecated API changes, as well as the significant cosmetic adjustment of migrating <code>Perl6</code> to <code>Raku</code>. The latter should be almost entirely finished, but there might be some stragglers that I've missed.</p> <p>Building should be as simple as opening this repository in <code>IntelliJ IDEA</code> (using version <code>2024.2</code> or greater), and selecting <code>build > build</code> from the Gradle build target options. Or, for more immediate gratification, you can select <code>intellij platform > runIde</code>.</p> <p><strong>Update:</strong> If you don't feel like building it yourself, you can now simply download the <a href='https://github.com/ab5tract/comma-plugin/releases/tag/v2.0-beta.1'>plugin zip from GitHub</a>. From inside IntelliJ IDEA, open the <code>Settings > Plugins</code>, find the gear icon, and select <code>Install Plugin from Disk...</code>.</p> <p>Next steps:</p> <ul> <li>Enabling the test suite</li> <li>Migrating the repository to the Raku organization on github</li> <li><del>Setting up multi-platform releases through CI/CD</del></li> <li>Setting up plugin signing and distribution via the IntelliJ Platform</li> <li>Adding NQP as another supported language</li> <li>Converting the entire project to Kotlin?</li> <li><code><Your wishlists go here!></code></li> </ul> <p>Happy hacking! :D</p> <h2><a href="https://rakujourney.wordpress.com/2024/09/15/can-raku-replace-php/">Can Raku replace PHP?</h2></a> <p><h3>Published by librasteve on 2024-09-15T19:18:49</h3></p> <p>Back in ’21 I asked the question <a href="https://rakujourney.wordpress.com/2021/06/09/can-raku-replace-html/">Can Raku replace HTML?</a> As expected that rather click-baity title got a lot of complaints. So I couldn’t resist repeating the meme.</p> <p>If you are wondering, <a href="https://raku.org">Raku</a> can replace PHP <em>literally</em>…</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: perl; gutter: false; title: ; notranslate"> "PHP".subst(/PHP/, 'Raku').say; #Raku </pre></div> <p>BUT that’s beside the point. Just my sense of <a href="https://raku.org/fun/">-Ofun</a> getting out of hand.</p> <p>In recent posts, I have been digging in to HTMX and Raku Cro…</p> <ul class="wp-block-list"> <li><a href="https://rakujourney.wordpress.com/2024/08/22/htmx-and-raku-cro/">HTMX and Raku Cro</a></li> <li><a href="https://rakujourney.wordpress.com/2024/09/01/htmx-and-raku-cro-ii/">HTMX and Raku Cro II</a></li> <li><a href="https://rakujourney.wordpress.com/2024/09/08/htmx-raku-and-pico-css/">HTMX, Raku and Pico CSS</a></li> </ul> <p>And while in the web application frame of mind, I started to think maybe I can use Raku with WordPress, perhaps initially to just write some front end with Raku and HTMX served with Cro and to talk to the WP database backend. (This kind of combination is already a <a href="https://vercel.com/guides/wordpress-with-vercel">thing</a> with WordPress and React).</p> <p>And then that made me think yeah well WordPress (and Laravel, OJS, etc.) continue to be popular and lend PHP a kind of ongoing zombie existence. PHP is not likely to suddenly bust out of its web language niche, so likely over time it will gradually fade away in popularity. And much of the gravity in web development is going to drag PHPers towards JavaScript. And, since I am a PHP coder in my day job, I realised that (like me) many PHPers travellers would rather not get dragged into the JavaScript / React / Composer / Node black hole of complexity. And so maybe Raku and HTMX would one day become a good upgrade path from PHP since it has roots in perl – the original web language – with a friendlier syntax (eg for OO). Even the <code>$</code> sigil for variables, <code>{}</code> curlies and the <code>;</code> semicolon make for a smooth transition from PHP. Maybe in this niche Raku can ultimately replace PHP…</p> <p>Then I started to think about what made PHP the goto language for web developers originally. How would Raku stack up?</p> <h2 class="wp-block-heading"><?php Flashback ?></h2> <p>Remember this:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: xml; gutter: false; title: ; notranslate"> <body> <div class="container"> <h1>Welcome to My Simple PHP Page!</h1> <p> Today is: <?php // Get the current date and time echo date("l, F j, Y, g:i a"); ?> </p> <p> Random number: <?php // Generate a random number between 1 and 100 echo rand(1, 100); ?> </p> </div> </body> </html> </pre></div> <p>The full source of this <strong>index.php</strong> file is in this gist … <a href="https://gist.github.com/librasteve/009831b5cb6821fa6daabb1de19aa4e6">Simple PHP HTML Page</a> </p> <p>To serve this page, you can run a server like this:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: php; gutter: false; title: ; notranslate"> php -S localhost:8000 -t /path/to/directory </pre></div> <p>Horrible though it is, this intertwining of PHP and HTML is what made PHP the goto web language in its heyday. And that got me thinking, could this be done with Raku?</p> <h2 class="wp-block-heading"><?raku as a Replacement ?></h2> <p>So, knowing Raku to be a very flexible language, I made a new module <a href="Evaluate">Cro::WebApp::Evaluate</a>. Here’s the synopsis:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: xml; gutter: false; title: ; notranslate"> <body> <div class="container"> <h1>Welcome to My Simple Raku Page!</h1> <p> Today is: <?raku #Get the current date and time use DateTime::Format; say strftime('%Y-%m-%d %H:%M:%S', DateTime.now); ?> </p> <p> Random number: <?raku #Generate a random number between 1 and 100 say (^100).pick; ?> </p> </div> </body> </pre></div> <p>And here’s how to serve this as <strong>index.raku</strong> using the Raku <a href="https://cro.raku.org">Cro</a> web framework.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: perl; gutter: false; title: ; notranslate"> use Cro::HTTP::Router; use Cro::WebApp::Template; use Cro::WebApp::Evaluate; sub routes() is export { route { evaluate-location 'evaluates'; get -> { evaluate 'index.raku'; } get -> *@path { static 'static', @path; } } } </pre></div> <p>I leave it as an exercise for the reader to show how to have Cro render and serve<strong> index.php</strong> files in a parallel directory and route structure – perhaps for an incremental migration effort.</p> <h2 class="wp-block-heading">Conclusion</h2> <p>Do I expect this new module to be embraced by the PHP community? No. In most cases, I think that hybrid PHP/HTML pages like this have been replaced by templating systems or web frameworks.</p> <p>Am I a little ashamed to have made this module? Yes. Honestly, I would not encourage coders to start using Raku like this – <a href="https://cro.raku.org/docs#Cro::WebApp">Cro Templates</a> would be a better solution for most projects.</p> <p>Are there some point needs where this approach can be applied? Maybe. Since this was a seminal feature of early PHP, I expect that there are some point cases where embedding Raku and HTML will be the cleanest way to (re)package some code. For example where a single dynamic page uses PHP for a database query, wrapping the results as json and then passing the data into a JavaScript function … for some client-side logic with the (eg.) Google Maps API and then dynamic presentation.</p> <p>Is this a module prerequisite for PHPers to migrate to Raku? Probably not. However, I think that the presence of this module can bring some comfort to PHP coders that anything that can be done in PHP can be (re)done in Raku. </p> <p>As usual comments & feedback welcome!</p> <p>~librasteve</p> <h2><a href="https://rakujourney.wordpress.com/2024/09/08/htmx-raku-and-pico-css/">HTMX, Raku and Pico CSS</h2></a> <p><h3>Published by librasteve on 2024-09-08T09:59:05</h3></p> <p>This post is kind of part 3, coming off last week’s thrilling <a href="https://rakujourney.wordpress.com/2024/09/01/htmx-and-raku-cro-ii/">episode</a>.</p> <p>I am a simple sole, I want to reduce the cognitive load in my web projects. The general idea is to go back to the halcyon early days of the web before Netscape dropped the JS-bomb. You know HTML for the layout and CSS for the style. An elegant division of roles.</p> <p>When I read about <a href="http://htmx.org">HTMX</a> it was clear that <a href="http://raku.org">Raku</a> and <a href="http://cro.raku.org">Cro</a> are ideal candidates for the back end HTML assembly, defining routes and serving RESTful APIs. As we have seen in the previous posts, HTMX eliminates the need for JS to make dynamic web content. Lovely.</p> <p>Remember – we are talking simpler ways to build attractive, dynamic, modern websites. While HTMX is well suited to 90% of this, if you are building a webapp like FaceBook or Google Maps, then it’s not for you.</p> <h2 class="wp-block-heading">Pico CSS</h2> <p>But what to do about style and CSS?</p> <p>Well HTMX is neutral to CSS … it can be used with Bootstrap, Tailwind, SASS and so on. But many of these CSS tools have evolved to jam more stuff into the HTML tag attributes.</p> <p>In my mind, the ideal would be something like this for a simple navbar:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: xml; gutter: false; title: ; notranslate"> <nav> <ul> <li><a href="#">About</a></li> <li><a href="#">Services</a></li> <li><a href="#">Products</a></li> </ul> </nav> </pre></div> <p>I had heard that <a href="https://picocss.com">Pico</a> CSS was often used in HTMX projects. And sure enough, the <a href="https://picocss.com/docs/nav">Pico Components</a> have this feel… </p> <p>Here’s Bootstrap for contrast:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: xml; gutter: false; title: ; notranslate"> <nav class="navbar navbar-expand-lg navbar-light bg-light"> <div class="collapse navbar-collapse" id="navbarNav"> <ul class="navbar-nav ms-auto"> <li class="nav-item"> <a class="nav-link" href="#">About</a> </li> <li class="nav-item"> <a class="nav-link" href="#">Services</a> </li> <li class="nav-item"> <a class="nav-link" href="#">Product</a> </li> </ul> </div> </nav> </pre></div> <p>What about Tailwind, also for contrast:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: xml; gutter: false; title: ; notranslate"> <nav class="bg-gray-100"> <div class="flex justify-end"> <ul class="flex space-x-3"> <li class="nav-item"> <a class="text-gray-700 hover:text-gray-900" href="#">About</a> </li> <li class="nav-item"> <a class="text-gray-700 hover:text-gray-900" href="#">Services</a> </li> <li class="nav-item"> <a class="text-gray-700 hover:text-gray-900" href="#">Product</a> </li> </ul> </div> </nav> </pre></div> <p>Bootstrap and Tailwind come at the cost of “more stuff in the HTML tags”.</p> <p>Here’s the Pico CSS example:</p> <figure class="wp-block-image size-large"><img loading="lazy" width="1024" height="484" data-attachment-id="1266" data-permalink="https://rakujourney.wordpress.com/2024/09/08/htmx-raku-and-pico-css/screenshot-2024-09-08-at-09-55-33/" data-orig-file="https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.55.33.png" data-orig-size="1784,844" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Screenshot 2024-09-08 at 09.55.33" data-image-description="" data-image-caption="" data-medium-file="https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.55.33.png?w=300" data-large-file="https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.55.33.png?w=782" src="https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.55.33.png?w=1024" alt="" class="wp-image-1266" srcset="https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.55.33.png?w=1024 1024w, https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.55.33.png?w=150 150w, https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.55.33.png?w=300 300w, https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.55.33.png?w=768 768w, https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.55.33.png 1784w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption class="wp-element-caption"><a href="https://picocss.com/docs/nav">https://picocss.com/docs/nav</a></figcaption></figure> <p>So for our goals, it looks like Pico CSS is on a good track. Their website says:</p> <figure class="wp-block-image size-large"><img loading="lazy" width="1024" height="490" data-attachment-id="1268" data-permalink="https://rakujourney.wordpress.com/2024/09/08/htmx-raku-and-pico-css/screenshot-2024-09-08-at-09-57-28/" data-orig-file="https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.57.28.png" data-orig-size="1556,746" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="Screenshot 2024-09-08 at 09.57.28" data-image-description="" data-image-caption="" data-medium-file="https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.57.28.png?w=300" data-large-file="https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.57.28.png?w=782" src="https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.57.28.png?w=1024" alt="" class="wp-image-1268" srcset="https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.57.28.png?w=1022 1022w, https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.57.28.png?w=150 150w, https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.57.28.png?w=300 300w, https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.57.28.png?w=768 768w, https://rakujourney.wordpress.com/wp-content/uploads/2024/09/screenshot-2024-09-08-at-09.57.28.png 1556w" sizes="(max-width: 1024px) 100vw, 1024px" /></figure> <p>What’s that?</p> <h2 class="wp-block-heading">Semantic HTML!</h2> <p>Looks like my goal all along has been <a href="https://www.semrush.com/blog/semantic-html5-guide/#">Semantic HTML</a> (not that I knew at the time).</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <p>By adding semantic HTML tags to your pages, you provide additional information that helps define the roles and relative importance of the different parts of your page.</p> <p>(As opposed to non-semantic HTML, which uses tags that don’t directly convey meaning.)</p> </blockquote> <p>So having more powerful <em>Semantic</em> HTML is a win.</p> <figure class="wp-block-image size-large"><img src="https://static.semrush.com/blog/uploads/media/cc/85/cc85d452a743e27f68d426df35e4da7d/EN-Semantic-Search-Non-Semantic.webp" alt="" /></figure> <p>Hopefully the figure above is enough of an eye opener for now. There’s much more <a href="https://en.wikipedia.org/wiki/Semantic_HTML">info</a> out there if you are curious. But obviously read the rest of my post first.</p> <p><em>[For hardcore aficionados, I plan to look into Web Components in a future post. I also think that Bootstrap and Tailwind and SASS in general are good companions to HTMX and Raku — but my project and this series of posts <strong>starts</strong> by using Pico CSS to minimize the cognitive load on the style side – later we will come back to these other styling tools].</em></p> <h2 class="wp-block-heading">Real Code</h2> <p>So lets see how this looks in action. All the code for these posts is on <a href="https://github.com/librasteve/raku-HTMX-Examples">GitHub</a> for your perusal and collaboration.</p> <p>I have been using Pico CSS as part of my project to rebuild the HTMX <a href="https://htmx.org/examples/">examples</a> for Raku / Cro largely by translating the Python / Flask <a href="https://github.com/Konfuzian/htmx-examples-with-flask/tree/main">examples</a>. This post draws on the <a href="https://htmx.org/examples/tabs-hateoas/">Tabs HATEOAS</a> one in particular, since I have in mind that I will want a Tab Component in my toolbag but that Pico CSS does not provide one out of the box. Pico does have <a href="https://picocss.com/docs/accordion">Accordions</a> so there is some prior art for inspiration.</p> <p>Anywho, here’s the way the final code ended up.</p> <p>tabs/index.crotmp:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: xml; gutter: false; title: ; notranslate"> <nav> <ul> <li> <a href="#" hx-get="/tabs/tab1" hx-target="#tab">Tab 1</a> </li> <li> <a href="#" hx-get="/tabs/tab2" hx-target="#tab">Tab 2</a> </li> <li> <a href="#" hx-get="/tabs/tab3" hx-target="#tab">Tab 3</a> </li> </ul> </nav> <div id="tab" hx-get="/tabs/tab1" hx-trigger="load"></div> </pre></div> <p>/tabs/tab1.crotmp (tab2 and tab 3 are much the same so I won’t bore you)</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: xml; gutter: false; title: ; notranslate"> <section> <figure> <blockquote> "When you're new to something, you bring an ignorance that can be highly innovative." <footer> <cite>– Rick Rubin</cite> </footer> </blockquote> </figure> </section> </pre></div> <p>/Routes/Examples/Tabs.rakumod to fulfil the <code>hx-get</code> attrs.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: php; gutter: false; title: ; notranslate"> use Cro::HTTP::Router; use Cro::WebApp::Template; sub tabs-routes() is export { route { template-location 'templates/tabs'; get -> { template 'index.crotmp'; } get -> 'tab1' { template 'tab1.crotmp'; } get -> 'tab2' { template 'tab2.crotmp'; } get -> 'tab3' { template 'tab3.crotmp'; } } } </pre></div> <p>And the proof… oh yeah, Pico has built in dark mode <img src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f609.png" alt="😉" class="wp-smiley" style="height: 1em; max-height: 1em;" /></p> <figure class="wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-4-3 wp-has-aspect-ratio"><div class="wp-block-embed__wrapper"> <div class="embed-youtube"><iframe title="Tabs" width="782" height="587" src="https://www.youtube.com/embed/QUB0hECEBZI?feature=oembed" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe></div> </div></figure> <p>Thanks for tuning in, please feel free to like, share or comment.</p> <p>You can find me on the <a href="https://raku.org/community/">Raku Discord and IRC Channels</a>.</p> <p>~librasteve</p> <p></p> <h2><a href="https://gfldex.wordpress.com/2024/05/28/smuggling-pairs/">Smuggling Pairs</h2></a> <p><h3>Published by gfldex on 2024-05-28T21:43:11</h3></p> <p>The <a href="https://stackoverflow.com/questions/78520298/how-to-set-a-named-parameter-to-an-exported-sub-when-importing-a-module">question</a> has been raised, how to get named arguments into <code>sub EXPORT</code> via a <code>use</code>-statement. The ever helpful <a href="https://stackoverflow.com/users/1077672/raiph">raiph</a> provided an answer, which in turn left me with the question, why he didn’t just use a <code>Capture</code> to move the data around. Well, because that doesn’t work. The compiler actually evaluates the expression <code>\(:1a, :2b)</code> into <code>(1, 2)</code> before passing it on to <code>EXPORT</code>.</p> <p>If it’s hard, do it functional!</p> <pre class="wp-block-code"><code># foo.raku use v6.d; constant &transporter = sub { \(:1a, :2b); } use foo &transporter; # lib/foo.rakumod use v6.d; proto sub EXPORT(|) { * } multi sub EXPORT(&transporter) { &EXPORT(|transporter); } multi sub EXPORT(:$a, :$b) { dd $a, $b; Map.new }</code></pre> <p>The idea is to hand a function to <code>use</code> to be called by <code>EXPORT</code>, and then redispatch the value that is produced by that function, to take advantage of Raku´s excellent <a href="https://docs.raku.org/type/Signature">signature binding</a>. The <code>proto</code> and refering to sub <code>EXPORT</code> explicitly is needed because there is also a predefined (and in this case hidden) <code>package</code> called <code>EXPORT</code>.</p> <p>I’m passing on named arguments to <code>EXPORT</code>, but all kinds of stuff could be returned by <code>&transporter</code>. So long as everything is known pretty early on at compile-time. The <code>use</code>-statement is truly an early bird.</p> <p></p> <h2><a href="https://gfldex.wordpress.com/2024/02/25/my-raku-for/">my $*RAKU++ for -> **@ {};</h2></a> <p><h3>Published by gfldex on 2024-02-25T15:31:34</h3></p> <p>As the title states, I made Raku bigger because lol context (that’s how the <a href="https://github.com/Raku/old-design-docs/blob/master/S29-functions.pod#lol">Synopsis</a> is calling <code>**@</code>) makes supporting feed operators fairly easy. I wonder if Larry added this syntax to <code>Signature</code> with that goal in mind. With <a href="https://github.com/rakudo/rakudo/pull/5532">PR#5532</a> the following becomes possible.</p> <pre class="wp-block-code"><code><abc bbc cbc> ==> trans('a' => 'x', 'b' => 'i') ==> say();<br># OUTPUT: (xic iic cic)</code></pre> <p>Armed with this I can make a script of mine a little simpler.</p> <pre class="wp-block-code"><code>use MONKEY-TYPING;<br><br>augment class IO::Path {<br> method trans(|c) {<br> my $from = self.Str;<br> my $to = self.Str.trans(|c);<br><br> self.rename($to) unless $from eq $to<br> }<br>}<br><br>sub rename-whitespace(IO::Path $dir where *.d){<br> dir($dir).grep({ .d || .f && .rw })<br> ==> trans("\c[space]" => "\c[no-break space]", "\c[apostrophe]" => "\c[prime]")<br> ==> sub (*@a) { print '.' for @a}();<br><br> dir($dir).grep({ .d && .rw })».&?ROUTINE;<br>}<br><br>rename-whitespace('.'.IO);<br><br>put '';</code></pre> <p>I don’t like spaces in filenames, as they are often found with audio or video files. Having auto-complete friendly names makes using a CLI less bumpy. By teaching <code>IO::Path</code> to rename files by providing rules, as they are understood by <code>Str.trans</code>, I can use a feed operator to get the job done. (I wouldn’t be surprised to learn, that anonymous subs DWIM here to be emergent behaviour in Raku.)</p> <p>Having another PR that adds <code>.trans</code> to <code>IO::Path</code> is tempting but requires more thought.</p> <h2><a href="https://dev.to/finanalyst/ryuu-a-japanese-dragon-2e7m">Ryuu - a Japanese dragon</h2></a> <p><h3>Published by Richard Hainsworth on 2024-01-30T18:30:58</h3></p> <blockquote> <p>A follow up to <a href="https://dev.to/finanalyst/creating-a-new-programming-language-draig-503p">the Welsh dragon</a>. </p> </blockquote> <h2> Table of Contents </h2> <p>Firing up another localisation<br><br> Steps to Ryuu<br><br> Comments on the Raku program<br><br> More generally about localisation of coding<br><br> If you want to make Ryuu better? </p> <h2> Firing up another localisation </h2> <p>In my previous blog about Y Ddraig, I created a localisation of the Raku Language in Welsh. During a recent conversation, someone mentioned there may be interest in a Japanese localisation, so I thought I would try the same techniques.</p> <blockquote> <p>I do not speak or read or have ever studied Japanese. The localisation given below will be about as clunky and awkward as any can be. I imagine there may be some hilarious stupidities as well.</p> </blockquote> <p>So to be clear, this article is about a proof of concept rather than a real effort to create a production-ready program.</p> <p>However, it took me 40 minutes from start to finish, including setting up <a href="https://github.com/finanalyst/rakuast-L10N-JA">the github repo</a>.</p> <p>Since I like dragons, I named the Japanese cousin to Raku 'Ryuu'. It's a whimsy, not to be treated with much seriousness.</p> <h2> Steps to Ryuu </h2> <p>Basically I created a github repo, copied my existing <a href="https://github.com/finanalyst/rakuast-L10N-CY">Welsh localisation</a> and changed <strong>CY</strong> to <strong>JA</strong>, and <strong>draig</strong> to <strong>ryuu</strong>.</p> <p>Within the <code>automation/</code> directory I used the translation technique explained for Welsh to create the <code>JA</code> file from the template. The <code>translated.txt</code> file needed some manual cleaning, because the English word <em>for</em> has multiple Japanese equivalents. I chose one more or less at random. In addition, Google translate did some strange things to the order of words and numbers in a line.</p> <p>After adapting the files in the <code>bin/</code> directory, and installing the distribution with Raku's <code>zef</code> utility, I ran <code>tr2ryuu</code> on the Raku program <code>simple.raku</code>.</p> <p>A comment about my Welsh blog was that the program in Y Ddraig was not all in Welsh. And here the program is not all in Japanese.</p> <p>Remember that the user-facing part of a program will be in the language of the user, in this case it is English. However, the coder-facing part of the program will be in the language of the coder. Below, the coder interface is in Japanese (or rather my ham-fisted attempt at Japanese).</p> <p>The following is the result (which I put in a file called simple.ryuu):<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>私の $choice; 私の $continue; 私の @bad = <damn stupid nutcase>; リピート { $choice = プロンプト "Type something, like a number, or a string: "; 言う "You typed in ｢" ~ ($choice ~~ 任意(@bad) ?? "*" × $choice.文字 !! $choice) ~ "｣"; 与えられた $choice { いつ "dragon" { 言う "which is 'draig' in Welsh" } いつ任意(@bad) { 言う "wash your mouth with soap" } いつ IntStr { 言う "which evaluates to an integer ", $choice } いつ RatStr { 言う "which evaluates to a rational number ", $choice } デフォルト { 言う "which does not evaluate to a number " } } $continue = プロンプト "Try again? If not type N: " } まで $continue 当量任意(<N n>) </code></pre> </div> <p>What is amazing to me is that when I ran <code>ryuu simple.ryuu</code>, the program ran without error.</p> <h2> Comments on the Raku program </h2> <p>The <code>simple.raku</code> program is obviously trivial, but what I wanted to show are some interesting Raku features. Note how I created an array of words with <code>@bad = <damn stupid nutcase>;</code>, and then later I tested to see whether an input word was one of the array elements.</p> <p>The Raku idiom <code>いつ任意(@bad)</code> or in English <code>when any( @bad )</code> compares the topic variable, in this case the input value, with each array element and creates a junction of Boolean results. The 'any' effectively <code>or</code>'s the result to collapse the junction.</p> <p>Junctions are not common in programming languages, so I thought if there would be problems, then it would be there. So I was surprised to find my Raku program works without error in another language.</p> <h2> More generally about localisation of coding </h2> <p>All the major coding languages are in English. There are, however, coders from all over the world, and the majority of those from non-English speaking nations would have needed to learn English before (or at the same time as) they learnt coding.</p> <p>We are thus creating a new technological elite: those who can understand English (or some subset of it), and those who cannot. The more coding becomes an essential part of life, the greater the ability divide between coders (who speak English) and non-coders will become.</p> <p>The aim of localising a programming language is to provide an entry into coding in a form that is more accessible to every human being, whatever their natural language.</p> <p>However, the aim of this approach is not to eliminate English at every level of complexity, but to provide a sufficiently rich language for most normal coding and educational needs.</p> <p>In addition, by having a canonical language (Raku, which is based on English) into which all localised languages can be translated, what we get is a universal auxiliary language together with a universality of being able to code.</p> <p>Having a single auxiliary language means that a non-English speaking person writing in a localised coding language can translate the program with the problem into Raku, have a developer on the other side of the globe find the problem, and suggest a solution in code, then for that solution to be translated back into the local language.</p> <p>Naturally, a person who wants to learn more about coding, or who needs to delve deeper into the workings of a module, will need to learn English. Learning wider to learn deeper is a normal part of the educational experience.</p> <h2> If you want to make Ryuu better? </h2> <p>Ryuu or however it should be called, absolutely is in need of Tender loving care. Please feel free to use the github issues or PR processes to suggest better translations.</p> <p>At some stage, Ryuu will join the official Raku localisations.</p> <h2><a href="https://dev.to/finanalyst/creating-a-new-programming-language-draig-503p">Creating a new programming language - Draig</h2></a> <p><h3>Published by Richard Hainsworth on 2024-01-15T08:18:08</h3></p> <blockquote> <p>Actually creating a localization of an existing programming language in an existing human language</p> </blockquote> <h3> Table of Contents </h3> <p>Introduction<br> Considerations<br> The plan for y Ddraig (the dragon in Welsh)<br> Constraints and first steps<br> Forwards into Draig and running<br> Completing the translation<br> Backwards to canonical form<br> Drawbacks</p> <h2> Introduction </h2> <p>Nearly all programming languages that are widely used in the world today have English as their base human language.</p> <p>This means that a young person living in a non-English environment must first learn English (if only a limited sub-set of English), and then learn the skills needed for coding. This puts the majority of the humanity at a disadvantage.</p> <p>Would it not be useful to create programming languages that use the script and words of human languages, but which compile into programs that will run with state of the art computer software?</p> <p>Here is how I created a Welsh cousin of Raku, and I called it <strong>y Ddraig</strong> - or <strong><em>The dragon</em></strong>.<sup>1</sup></p> <h2> Considerations </h2> <p>There are some practical obstacles to creating any new programming language, and here are some of the ameliorating reasons why the <strong><em>Raku Programming Language</em></strong> is a good choice to base a new one on.</p> <ul> <li> <p>Different human languages use different writing systems and most need extra letters not covered by the ASCII set</p> <ul> <li> The Unicode system has eliminated the problems of displaying and storing different writing systems.</li> <li> The Raku language has Unicode support at every level. Every number, operator, keyword, function etc can be written with Unicode symbols. There are a very few exceptions, such as <code>;</code>, <code>,</code>, and <code>{}</code>.</li> </ul> </li> <li> <p>Different operating systems</p> <ul> <li> There are at least three major operating systems widely used in the world: Window, PC Linux, and Mac OS.</li> <li> Raku runs on all three</li> </ul> </li> <li> <p>All professional programmers are proficient in English, and so can answer questions about program errors in English. The number of programmers speaking Welsh is quite small, and the same would be true for many other human languages.</p> <ul> <li> This is precisely the problem we are trying to resolve: making the coding profession a possibility for humanity as a whole. But there is a vicious circle, professional programmers work in English, so how can they help a person in another language?</li> <li> The circle can be broken if the interface language, which is the one that programmers work in, can be varied, whilst the code that is run on a computer is completely independent of the interface language.</li> <li> If the interface language can be easily changed, then a program coded in Welsh, but with a problem, can be easily translated (whilst continuing to work as before) into English. The problem can be resolved, corrected, and the solution changed (still working) back to Welsh.</li> </ul> </li> </ul> <h2> The plan for <strong>y Ddraig</strong> </h2> <p>Whilst the plan is to create <strong><em>y Ddraig</em></strong> as a language that can be used with as little English as possible, there are several stages:</p> <ul> <li> <p>First is to create a localization (L10N) of Raku, or a module called <code>L10N::CY</code>.</p> <ul> <li> Inside a Raku program, all that will be needed for a completely Welsh program is for the first line to be use L10N::CY;</li> <li> All subsequent lines will be in Welsh, but will compile and run as a normal Raku program.</li> <li> The program can be easily translated into English, and English Raku programs translated in to Welsh. Simple utilities are given below to do this.</li> </ul> </li> <li><p>Next, the operating system has to be adapted so that a executable called <code>draig</code> is available, which will also mean that in a graphic interface (GUI), double clicking on a file with a file-extension of <code>.draig</code> will run <strong><em>Raku</em></strong> with the <em>L10N::CY</em> module already loaded. This is trivial.</p></li> </ul> <h2> Constraints and first steps </h2> <p>For personal reasons, I stopped using Windows on my PC, and I use Ubuntu Linux exclusively. So, where there are terminal sessions, I shall be showing how I created <strong><em>Y ddraig</em></strong> using a Linux terminal.</p> <p>Since <strong><em>Y ddraig</em></strong> is a Raku cousin, or technically a Raku localization, the Raku language needs to be installed. In addition, it needs to be a version of the language released after December 2023. Information about the installation of Raku, and its package manager <strong>zef</strong>, can be found <a href="https://raku.org">on the Raku website</a>.</p> <p>The first stage is to create the <code>L10N::CY</code> module. It is simply a normal Raku module, which is then installed with the <code>zef</code> package manager.</p> <p>Raku module development is conventionally done by creating a github repository. Working with <strong><em>git</em></strong> is quite simple for the basic functionality, but there is a long learning curve when working with others. But none of that is the topic here.</p> <p>Elizabeth Mattijsen, who is responsible for all this Raku internationalization magic, has created a template internationalization module for the Klingon language (yep: aliens get to be the first to use localizations of a Terran computer language)<sup>2</sup>.</p> <p>So I git cloned the Klingon, and created a github repo for the Welsh. My git nick is <em>finanalyst</em>, so here's the terminal code lines:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>git clone https://github.com/lizmat/L10N-TLH.git rakuast-L10N-Klingon git clone https://github.com/finanalyst/rakuast-L10N-CY.git rakuast-L10N-Welsh </code></pre> </div> <p>In the following, I shall call Elizabeth's repo, the Klingon repo, and mine, the Welsh repo. If you want to create your own language, the convention being followed is to name the language according to an ISO 639-1 supported language code, at least for the foreseeable future. You should also think of an filename extension (like <code>.draig</code> here) for programs in the new language (Raku cousin).</p> <p>The two critical parts of the module are <code>update-localization</code>, and a root text file which we will call the <strong><em>localization map</em></strong>. It should be named by the language code. Here it is called <code>CY</code> for <em>Cymraeg</em> or the Welsh language, for Klingon, it is <code>TLH</code>.</p> <p>The <code>update-localization</code> utility in from the Klingon repo looks for a repo root directory file with 2 or 3 upper case characters. This is taken as the <em>localization map</em> and is automatically converted into all the magical modules.</p> <p>The biggest step is to translate the terms to be stored in <code>CY</code>. The template for the <em>localization map</em> can be found at <a href="https://github.com/Raku/L10N/TEMPLATE">Github Raku localizations</a>. To get this as a local text file, I used the following terminal code to download the template in to my working directory.<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>curl 'https://raw.githubusercontent.com/Raku/L10N/main/TEMPLATE' > CY </code></pre> </div> <p>The pristine form of <code>CY</code> contains a few lines of comment (starting with the characters '# ', note the space), and then a number of sections starting with<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code># KEY TRANSLATION </code></pre> </div> <p>Within each section there is a key and then an English Raku keyword, eg.<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>#adverb-pc-delete delete </code></pre> </div> <p>Note that it has been commented out with single <code>#</code>. This means that the <code>update-localization</code> utility will ignore the line.</p> <p>Now comes the translation part. Each significant commented line (a line with <code>#</code> and no space at the start) has two parts: a KEY and a TRANSLATION, with some spaces between them. The translation process is to substitute the English Raku keyword with the Welsh word, and remove the <code>#</code>. For example, the first significant line becomes<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>adverb-pc-delete dileu </code></pre> </div> <p>When starting the translation process, and to see how the system works, it is sufficient to translate a minimum number of keys. (Eg., for the <em>Draig</em> program below, I only need eleven words.)</p> <p>Once I have enough key words for the program, all that is needed is to run <code>./update-localization</code>. This then creates a directory tree under <code>lib/</code>.</p> <h2> Forwards into Draig and running </h2> <p>Here is a short program in Raku (English cousin), which we store in a file called 'simple.raku' in the root directory of the repo.<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>my $choice; my $continue; my @bad = <damn stupid nutcase>; repeat { $choice = prompt 'Type something, like a number, or a string: '; say 'You typed in ｢' ~ ( $choice ~~ any( @bad ) ?? '*' x $choice.chars !! $choice) ~ '｣'; given $choice { when 'dragon' { say "which is 'draig' in Welsh" } when any( @bad ) { say "wash your mouth with soap" } when IntStr { say "which evaluates to an integer ", $choice } when RatStr { say "which evaluates to a rational number ", $choice } default { say "which does not evaluate to a number "} } $continue = prompt 'Try again? If not type N: '; } until $continue eq any(<N n>) ; </code></pre> </div> <p>Try running it in a terminal where the working directory is the root directory of the repo, thus:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>raku simple.raku </code></pre> </div> <p>If you input some words, it will tell you the input is a string, if you input something naughty (well only one of the three words 'damn stupid nutcase'), you will get another response, and then there are responses depending on whether the number is an integer or a rational.</p> <p>The code uses 11 keywords, which I translated and put into <code>CY</code>. Obviously, there are many strings that form the user interface, and these are hard-coded in this program in English. We are concerned at the moment with the infrastructure keywords that form the programming language.</p> <p>Now lets translate the Raku program using a simple Raku utility called <code>tr2draig</code>. <br> We shall specify here that the Raku program is of the form <code>somename.raku</code> and that we want a Draig program of the form <code>somename.draig</code>. </p> <p>The utility is the following Raku script:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>#!/usr/bin/env raku sub MAIN( $filename where *.IO.f #= source file to be localized to Welsh ) { $filename.IO.extension('draig').spurt: $filename.IO.slurp.AST.DEPARSE("CY") } </code></pre> </div> <p>Breaking the program down, <code>#!/usr/bin/env raku</code> is standard for a script with execute permission.</p> <p><code>$filename where *.IO.f #= ...</code> is a nice Raku idiom for a program called from a terminal. The program expects a string that names a file. It checks that the filename exists and is of type 'f'. If not, then an error message will be provided from the comment following <code>#=</code>.</p> <p><code>$filename.IO.extension('draig').spurt:</code> takes the filename, creates a new file with the extension 'draig' replacing the previous extension (which was 'raku'), then spurts text into it, the text it uses being generated by the expression after the <code>:</code>.</p> <p><code>$filename.IO.slurp.AST.DEPARSE("CY")</code> takes the filename (which has extension 'raku'), makes it into a filehandle, slurps (sucks) in the text that is in the file, parses the text as a Raku program into an Abstract Symbol Tree (AST), and then <code>deparses</code> the symbol tree using the new Welsh keywords into a new program with Welsh.</p> <blockquote> <p>For reasons related to distributing Raku software, I have placed the utility in the <code>bin/</code> directory. There are two ways to get a copy of these files, either by creating a clone of my Github repository (the url is given above), or by installing the Raku distribution, as <code>zef install "L10N::CY"</code>. If <em>zef</em> is set up in a typical way, then the utilities below can be run without specifying the path.</p> </blockquote> <p>The translation utility is run like this<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>bin/tr2draig simple.raku </code></pre> </div> <p>This produces a file <code>simple.draig</code>, which contains<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>fy $choice; fy $continue; fy @bad = <damn stupid nutcase>; ailadrodd { $choice = prydlon "Type something, like a number, or a string: "; dywedyd "You typed in ｢" ~ ($choice ~~ unrhyw(@bad) ?? "*" x $choice.golosg !! $choice) ~ "｣"; a-roddwyd $choice { pryd "dragon" { dywedyd "which is 'draig' in Welsh" } pryd unrhyw(@bad) { dywedyd "wash your mouth with soap" } pryd IntStr { dywedyd "which evaluates to an integer ", $choice } pryd RatStr { dywedyd "which evaluates to a rational number ", $choice } rhagosodedig { dywedyd "which does not evaluate to a number " } } $continue = prydlon "Try again? If not type N: " } hyd $continue eq unrhyw(<N n>) </code></pre> </div> <p>Now we want a way to run <code>draig</code> programs. The easiest way is create another Raku program <code>draig</code>, which we place in the <code>bin/</code> directory. <code>bin/draig</code> has the following content:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>#!/usr/bin/env raku sub draig(*@_) { %*ENV<RAKUDO_RAKUAST> = 1; %*ENV<RAKUDO_OPT> = '-ML10N::CY'; run $*EXECUTABLE, @_; } multi sub MAIN() { draig } multi sub MAIN( $filename where *.IO.f #= source file to be run in Welsh ) { draig $filename } </code></pre> </div> <p>Here's a gloss of the program:<br> <code>sub draig(*@_) {...</code> This is a helper subroutine called later. It sets up environment variables, and preloads the localization module, before running Raku with the Welsh keywords. </p> <p><code>multi sub MAIN()</code> runs the sub draig (above) when no program is given. This puts the user into a REPL, where statements can be input directly, parsed and run immediately. However, <code>draig</code> will run using the Welsh keywords.</p> <p><code>multi sub MAIN(<br> $filename where *.IO.f #= source file to be run in Welsh<br> )</code> handles the case when <code>draig</code> is given a filename. As explained above, the filename is tested for existence.</p> <p>Now try running <code>bin/draig simple.draig</code> in a terminal.</p> <blockquote> <p>If the <code>RakuAST-L10N-CY</code> distribution has been installed with <code>zef</code>, then all you will need is <code>draig simple.draig</code>.</p> </blockquote> <p>The running code produces exactly the same output as the English Raku program. The user interface output is still in English, and for completeness, I should translate all of the text strings to Welsh as well.</p> <h2> Completing the translation </h2> <p>At this point, we can translate any English version of a Raku program into a Draig program, and <code>draig</code> will run it, but only if the Raku program uses the 11 keywords I translated.</p> <p>In order to create a full localization, all of the <strong>Translation</strong> values need to be converted to Welsh. The <strong>first</strong> step (and I really must re-emphasise it is a first step) is to use an automated translation tool. A correct localization will need first-language Welsh speakers to go through the <code>CY</code> file and correct the translations.</p> <blockquote> <p>At the time of writing, the localization has not been properly verified, so it has not yet been added to the official Raku localizations.</p> </blockquote> <p>For the automated translation, I have created the directory <code>automation/</code>. I again downloaded the TEMPLATE into a <code>CY</code> file in the <code>automation/</code> directory.</p> <p>I have written some automation helper utilities, namely:</p> <ol> <li> <code>find-untranslated</code>, takes a CY file and splits it into two new files, with line numbers at the start of each line to help match later. One file is <code>partial.txt</code> with the starting key and comment lines, and the second file is <code>to-be-translated.txt</code>. Both contain approximately 700 lines.</li> <li> <code>combine-translated</code>, takes <code>partial.txt</code> and another file <code>translated.txt</code> (see below) to create a new CY file.</li> </ol> <p>Next I copy/pasted the lines for translation (from the file <code>to-be-translated.txt</code> into <a href="https://translate.google.co.uk/?sl=auto&tl=cy&op=translate">Google's translate to Welsh page</a>. The operation took a couple of copy/pastes due to size limitations, but the text is not overly large.</p> <p>The translated text can be copied straight back to a new file (translated.txt), and then recombined with <code>partials.txt</code> to create CY.</p> <h2> Backwards to canonical form </h2> <p>As mentioned above, suppose a Welsh-speaker using <code>y Ddraig</code> <br> runs into a programming problem, a syntax error or logic not working as the programmer assumes. An English speaking programmer will probably not be able to help. </p> <p>But ... <em>.draig</em> program can be retranslated back to the canonical form of Raku. This is done by a utility called <code>tr2raku</code>. It is almost the inverse of <code>tr2draig</code>, but instead of replacing the file extension <code>.draig</code> with <code>.raku</code>, we add it on to the filename so that its clear it is a canonicalisation of a Raku cousin.</p> <p>The utility <code>bin/tr2raku</code> contains the following contents.<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>#!/usr/bin/env raku sub MAIN( $filename where *.IO.f #= Welsh source file to be turned to canonical form ) { $filename.IO.extension('raku', :0parts).spurt: $filename.IO.slurp.AST("CY").DEPARSE } </code></pre> </div> <p>The difference can be seen that the language signifier (<code>CY</code>) is a parameter to the <code>AST</code> method, rather than the <code>DEPARSE</code> method. </p> <p>There should be no reason why this recipe cannot be applied to Mandarin, Hindi, or Japanese.</p> <h2> Drawbacks </h2> <p>The problems stem from the development history of Raku. Error messages are in English, and so Raku cousins, like Draig, will have English error messages. </p> <p>The problem is not insurmountable, but it will take a lot of translator hours.</p> <p>Another problem is that helper modules, for example, <code>JSON::Fast</code>, which imports/exports structured data from/to <code>.json</code> files into Raku data structures. The module has two main methods <code>to-json</code> and <code>from-json</code>. These names are set by the module, not by Raku.</p> <p>A program in <code>y Ddraig</code> will be able to access all Raku modules without restriction, but it will need to use the canonical (English) names.</p> <p>However, if many Raku localizations come into being, and a user base for them develops, these are all soluble problems.</p> <p>Footnotes</p> <h6> 1 </h6> <p>A reader may wonder why the language is <em>Y ddraig</em>, but <em>draig</em> is given in dictionaries as the translation for <em>dragon</em>. Well ..., <em>draig</em> is a feminine word, and the definite particle <em>Y</em> triggers a mutation in the next feminine word, so <em>d</em> mutates to <em>dd</em>.</p> <h6> 2 </h6> <p>My next project is to create a localization with Egyptian hieroglyphs </p> <h2><a href="https://gfldex.wordpress.com/2023/12/25/autoarraying/">Autoarraying</h2></a> <p><h3>Published by gfldex on 2023-12-25T08:57:55</h3></p> <p>Over on Reddit <a href="https://www.reddit.com/user/zeekar/">zeekar</a> wasn’t too happy about Raku’s love of <code>Seq</code>. It’s immutability can be hindering indeed.</p> <pre class="wp-block-code"><code>my @nums = [ [1..10], ];<br>@nums[0] .= grep: * % 2;<br>@nums[0].push(11); # We can't push to a Seq.</code></pre> <p>I provided a <a href="https://www.reddit.com/r/rakulang/comments/18pv92o/better_way_to_deal_with_nested_arrays_losing/">solution</a> I wasn’t happy with. It doesn’t DWIM and is anything but elegant. So while heavily digesting on my sofa (it is this time of the year), the problem kept rolling around in my head. At first I wanted to wrap <code>Array.grep()</code>, but that would be rather intrusive and likely break Rakudo itself. After quite a bit of thinking, I ended up with the question. How can I have indexable container (aka Array) that will turn each value on assignment into an (sub-)Array?</p> <pre class="wp-block-code"><code>my Array() @foo = [ 1..10, ];<br>dd @foo;<br># Array[Array(Any)] @foo = Array[Array(Any)].new($[1, 2, 3, 4, 5, 6, 7, 8, 9, 10])<br>@foo[0] .= grep: * % 2;<br>@foo[1] = 42;<br>dd @foo;<br># Array[Array(Any)] @foo = Array[Array(Any)].new($[1, 3, 5, 7, 9], $[42])</code></pre> <p>The answer is obvious. By telling the compiler what I want! Coersion-types have become really hard to distinguish from magic.</p> <p>I wish you all a Merry Christmas and the very best questions for 2024.</p> <h2><a href="https://5ab5traction5.bearblog.dev/an-initial-investigation-into-using-zig-to-speed-up-raku-code/">An initial investigation into using Zig to speed up Raku code</h2></a> <p><h3>Published by 5ab5traction5 on 2023-11-27T14:21:37</h3></p> <p><em>Note: This post is also <a href='https://gist.github.com/ab5tract/61e08cfaf0bb3e67b3a3dc009fa3cc7a#file-run-notes-md'>available as a gist</a> if you find that format more readable.</em></p> <h2 id=introduction>Introduction</h2><p>This research was conducted while preparing an upcoming <a href='https://raku-advent.blog/'>Raku Advent Calendar</a> post. The Raku code uses a basic supply pipeline to feed <code>$volume</code> objects through a validation stage that requires a CRC32 check before going to the output sink, which prints the processing time of the validation stage.</p> <p>The "reaction graph" is designed to simulate a stream processing flow, where inputs arrive and depart via Candycane™ queues (that's the name of Santa's Workshop Software's queueing service, in case you weren't familiar).</p> <p>The entire scenario is contrived in that CRC32 was chosen due to native implementation availability in both Raku and Zig, allowing comparison. It's not an endorsement of using CRC32 in address validation to deliver Santa's, or anyone's, packages.</p> <p>Also, thanks to the very helpful folks at <a href='https://ziggit.dev'>ziggit.dev</a> for answering my <a href='https://ziggit.dev/t/heap-allocation-without-a-main-function/2249/6'>newbie question</a> in depth.</p> <h2 id=methodology>Methodology</h2><p>The source code:</p> <ul> <li>Raku - <a href='https://gist.github.com/ab5tract/61e08cfaf0bb3e67b3a3dc009fa3cc7a#file-crc-getter-raku'>crc-getter.raku</a></li> <li>Raku+Zig - <a href='https://gist.github.com/ab5tract/61e08cfaf0bb3e67b3a3dc009fa3cc7a#file-crc-getter-extended-raku'>crc-getter-extended.raku</a>, <a href='https://gist.github.com/ab5tract/61e08cfaf0bb3e67b3a3dc009fa3cc7a#file-main-zig'>main.zig</a></li> </ul> <p>At larger volumes, Raku struggles with the initialization speed of the <code>$volume</code> objects that are instantiated. I replaced the native Raku class with one written in Zig, using the <code>is repr('CStruct')</code> trait in Raku and the <code>extern struct</code> qualifier in Zig.</p> <p>In Zig I use a combination of an arena allocator (for the string passed from Raku) and a memory pool (designed to quicklymake copies of a single type, exactly fitting our use case) to construct <code>Package</code> objects.</p> <p>Additionally, for Raku+Zig the CRC32 hashing routine from Zig's <code>stdlib</code> is used via a tiny wrapper function.</p> <p>A <code>--bad-packages</code> option is provided by both Raku scripts, which makes 10% of the objects have a mismatched address/CRC32 pair.</p> <p>The library tested was compiled with <code>-Doptimize=ReleaseFast</code>.</p> <p>Batches are repeated <code>$batch</code> times, which defaults to 5.</p> <p>All results from an M2 MacBook Pro.</p> <h3 id=caveats>Caveats</h3><p>This test and its is only intended to reflect the case where an object is constructed in Zig based on input from Raku. It is not intended to be a test of Zig's native speed in the creation of structs.</p> <p>There is a call to <code>sleep</code> that gives time -- <code>0.001</code> seconds -- to get the <code>react</code> block up and running before emitting the first <code>True</code> on the <code>$ticker-supplier</code>. This affects overall runtime but <em>not</em> the batch or initialization metrics.</p> <p>The speed of Raku+Zig was so fast that the tool used to measure these details (<code>cmdbench</code>) could not find results in <code>ps</code> for the execution because it had already finished. These are marked as <code>Unmeasured</code>.</p> <p>In the next iteration of this research, there sould be two additional entries in the data tables below for:</p> <ul> <li>Raku+Zig: Raku-managed objects / Zig crc32</li> <li>Raku+Zig: Zig-managed objects / Raku crc32</li> </ul> <h2 id=results>Results</h2><h3 id=10000>10,000</h3><table> <thead> <tr> <th>Volume</th> <th>Edition</th> <th>Runtime</th> <th>Batch Time</th> <th>Initialization</th> <th>Max bytes</th> </tr> </thead> <tbody> <tr> <td>10,000</td> <td>Raku</td> <td>1.072s</td> <td>1: 0.146596686s<br />2: 0.138983732s<br />3: 0.142380065s<br />4: 0.136050775s<br />5: 0.134760525s</td> <td>0.008991746s</td> <td>180240384</td> </tr> <tr> <td>10,000</td> <td>Raku+Zig</td> <td>0.44s</td> <td>1: 0.010978411s<br />2: 0.006575705s<br />3: 0.004145623s<br />4: 0.004280415s<br />5: 0.00468929s</td> <td>0.020358033s</td> <td><code>Unmeasured</code></td> </tr> <tr> <td>10,000</td> <td>Raku<br />(<code>bad-packages</code>)</td> <td>1.112s</td> <td>1: 0.157788932s<br />2: 0.149544686s<br />3: 0.156293433s<br />4: 0.151365477s<br />5: 0.147947436s</td> <td>0.008059955s</td> <td>196263936</td> </tr> <tr> <td>10,000</td> <td>Raku+Zig<br />(<code>bad-packages</code>)</td> <td>0.463s</td> <td>1: 0.031300276s<br />2: 0.01006562s<br />3: 0.010693328s<br />4: 0.011056994s<br />5: 0.010770828s</td> <td>0.010954495s</td> <td><code>Unmeasured</code></td> </tr> </tbody> </table> <h4 id=notes>Notes</h4><p>The Raku+Zig solution wins in performance, but loses the initialization race. Raku is doing a decent showing in comparison to how far it has come performance-wise.</p> <h3 id=100000>100,000</h3><table> <thead> <tr> <th>Volume</th> <th>Edition</th> <th>Overall</th> <th>Batch Time</th> <th>Initialization</th> <th>Max bytes</th> </tr> </thead> <tbody> <tr> <td>100,000</td> <td>Raku</td> <td>7.163s</td> <td>1: 1.360029456s<br />2: 1.32534014s<br />3: 1.353072834s<br />4: 1.346668338s<br />5: 1.351110502s</td> <td>0.062402473s</td> <td>210173952</td> </tr> <tr> <td>100,000</td> <td>Raku+Zig</td> <td>0.75s</td> <td>1: 0.079802007s<br />2: 0.073638176s<br />3: 0.053291894s<br />4: 0.05087652s<br />5: 0.050394687s</td> <td>0.05855585s</td> <td>241205248</td> </tr> <tr> <td>100,000</td> <td>Raku<br />(<code>bad-packages</code>)</td> <td>7.89s</td> <td>1: 1.496982355s<br />2: 1.484494027s<br />3: 1.497365023s<br />4: 1.490810525s<br />5: 1.492416774s</td> <td>0.060026016s</td> <td>209403904</td> </tr> <tr> <td>100,000</td> <td>Raku+Zig<br />(<code>bad-packages</code>)</td> <td>1.076s</td> <td>1: 0.16960934s<br />2: 0.111172493s<br />3: 0.110844786s<br />4: 0.113021202s<br />5: 0.111713535s</td> <td>0.051436311s</td> <td>242450432</td> </tr> </tbody> </table> <h4 id=notes>Notes</h4><p>We see Raku+Zig take first place in everything but memory consumption, which we can assume is a function of using the NativeCall bridge, not to mention my new-ness as a Zig programmer.</p> <h3 id=1000000>1,000,000</h3><table> <thead> <tr> <th>Volume</th> <th>Edition</th> <th>Overall</th> <th>Batch Time</th> <th>Initialization</th> <th>Max bytes</th> </tr> </thead> <tbody> <tr> <td>1,000,000</td> <td>Raku</td> <td>68.081s</td> <td>1: 13.475302627s<br />2: 13.161153845s<br />3: 13.293998956s<br />4: 13.364662217s<br />5: 13.474755295s</td> <td>0.95481884s</td> <td>417103872</td> </tr> <tr> <td>1,000,000</td> <td>Raku+Zig</td> <td>3.758s</td> <td>1: 0.788083286s<br />2: 0.509883905s<br />3: 0.492898873s<br />4: 0.500868284s<br />5: 0.498677495s</td> <td>0.575087671s</td> <td>514064384</td> </tr> <tr> <td>1,000,000</td> <td>Raku+Zig<br />(<code>bad-packages</code>)</td> <td>75.796s</td> <td>1: 14.940173822s<br />2: 14.632683637s<br />3: 14.866796226s<br />4: 15.272903792s<br />5: 15.027481448s</td> <td>0.704549212s</td> <td>396656640</td> </tr> <tr> <td>1,000,000</td> <td>Raku+Zig<br />(<code>bad-packages</code>)</td> <td>6.553s</td> <td>1: 1.362189763s<br />2: 1.061496504s<br />3: 1.069134685s<br />4: 1.062746049s<br />5: 1.061096044s</td> <td>0.528011288s</td> <td>462766080</td> </tr> </tbody> </table> <h4 id=notes>Notes</h4><p>Raku's native CRC32 performance is clearly lagging here. Raku+Zig keeps its domination except in the realm of memory usage. It would be hard to justify using the Raku native version strictly on its reduced memory usage, considering the performance advantage on display here</p> <p>A "slow first batch" problem begins to affect Raku+Zig. Running with <code>bad-packages</code> enabled slows down the Raku+Zig crc32 loop, hinting that there might be some optimizations on either the Raku or the Zig/clang side of things that can't kick in when the looped data is heterogenous.</p> <p>Dynamic runtime optimization <em>sounds</em> more like a Rakudo thing than a Zig thing, though.</p> <h3 id=10000000>10,000,000</h3><table> <thead> <tr> <th>Volume</th> <th>Edition</th> <th>Runtime</th> <th>Batch Time</th> <th>Initialization</th> <th>Max bytes</th> </tr> </thead> <tbody> <tr> <td>10,000,000</td> <td>Raku</td> <td>704.852s</td> <td>1: 136.588638184s<br />2: 136.851019628s<br />3: 138.44696743s<br />4: 139.777040922s<br />5: 139.490784317s</td> <td>13.299274221s</td> <td>2055012352</td> </tr> <tr> <td>10,000,000</td> <td>Raku+Zig</td> <td>38.505s</td> <td>1: 8.843459877s<br />2: 4.84300835s<br />3: 4.991842433s<br />4: 5.077245603s<br />5: 4.939533707s</td> <td>9.375436134s</td> <td>2881126400</td> </tr> <tr> <td>10,000,000</td> <td>Raku<br />(<code>bad-packages</code>)</td> <td>792.1s</td> <td>1: 162.333803401s<br />2: 174.815386318s<br />3: 168.299796081s<br />4: 162.643428135s<br />5: 163.205406678s</td> <td>10.252639311s</td> <td>2124267520</td> </tr> <tr> <td>10,000,000</td> <td>Raku+Zig<br />(<code>bad-packages</code>)</td> <td>65.174</td> <td>1: 14.41616445s<br />2: 11.078961309s<br />3: 10.662389991s<br />4: 11.20240076s<br />5: 10.614430063s</td> <td>6.778600235s</td> <td>2861596672</td> </tr> </tbody> </table> <h4 id=notes>Notes</h4><p>Pure Raku <em>really</em> struggles with a volume of this order of magnitude. But if you add in just a little bit of Zig, you can reasonably supercharge Raku's capabilities.</p> <p>The "slow first batch" for Raku+Zig has been appearing in more understated forms in other tests. Here the first batch is over double the runtime of the second batch. What is causing this?</p> <h3 id=100000000>100,000,000</h3><p>This doesn't seem to work. At least, I'm not patient enough. The process seems to stall, growing and shrinking memory but never finishing.</p> <h2 id=final-thoughts>Final Thoughts</h2><p>This is a preliminary report in blog post form based on a contrived code sample written for another, entirely different blog post. More data and deeper analysis will have to come later.</p> <p>Zig's C ABI compatibility is clearly no put on. It works seamlessly with Raku's NativeCall. Granted, we haven't really pushed the boundaries of what the C ABI can look like but one of the core takeaways is actually that <em>with Zig we can design that interface</em>. In other words, we are in charge of how ugly, or not, it gets. Considering how dead simple the <code>extern struct <-> is repr('CStruct')</code> support is, I don't think the function signatures need to get nearly as gnarly as they get in C.</p> <p>Sussing the truth of that supposition out will take some time and effort in learning Zig. I'm looking forward to it. My first stop will probably be a JSON library that uses Zig. I'm also going to be looking into using Zig as the compiler for Rakudo, as it might simplify our releases significantly.</p> <h2><a href="https://gfldex.wordpress.com/2023/11/10/missing-virtuousness/">Missing Virtuousness</h2></a> <p><h3>Published by gfldex on 2023-11-10T12:43:35</h3></p> <p>According to Larry, laziness is a programmers virtue. The best way to be lazy is having somebody else do it. By my <a href="https://ieji.de/@gfldex/111366016310683643">request</a>, <a href="https://fosstodon.org/@smokemachine/">SmokeMachine</a> kindly <a href="https://github.com/FCO/HTML-Component/commit/89a4236ef9bbd9a0260883d3282c28889fd21e8f">did</a> so. This is not fair. We both should have been lazy and offload the burden to the CORE-team.</p> <p>Please consider the following code.</p> <pre class="wp-block-code"><code>my @many-things = (1..10).List; sub doing-one-thing-at-a-time($foo) { ... } say doing-one-thing-at-a-time(@many-things.all);</code></pre> <p>Rakudo goes out of it’s way to create the illusion that <code>sub doing-one-thing-at-a-time</code> can deal with a <code>Junction</code>. It can’t, the dispatcher does all the work of running code in parallel. There are tricks we can play to untangle a <code>Junction</code>, but there is no guarantee that all values are produced. <code>Junction</code>s are allowed to short-circuit.</p> <p>This was bouncing around in my head for quite some time, until it collided with my thoughts about <code>Range</code>. We may be handling <code>HyperSeq</code> and <code>RaceSeq</code> wrong.</p> <pre class="wp-block-code"><code>my @many-things = (1..10).List; sub doing-one-thing-at-a-time($foo) { ... } say doing-one-thing-at-a-time(@many-tings.<strong>hyper(:degree<10>)</strong>);</code></pre> <p>As with <code>Junction</code>s doing dispatch-magic to make hyper/race just work, moving the handling to the dispatcher would move the decision from the callee to the caller and, as such, from the author of a module to the user. We can do that by hand already with <code>.hyper.grep(*.foo)</code> or other forms of boilerplate. In Raku-land we should be able to do better and provide a generalisation of transforming calls with the help of the dispatcher.</p> <p>I now know what to ask Santa for this year.</p> <h2><a href="https://vrurg.github.io/2023/10/31/json-class-released">JSON::Class:auth Released</h2></a> <p><h3>Published by Vadim Belman on 2023-10-31T00:00:00</h3></p> <p>My version of <code class="language-plaintext highlighter-rouge">JSON::Class</code> is now released. <a href="/2023/10/17/new-json-class">The previous post</a> explains why does this worth a note.</p> <h2><a href="https://gfldex.wordpress.com/2023/10/24/incomplete-ranges/">Incomplete Ranges</h2></a> <p><h3>Published by gfldex on 2023-10-24T19:36:17</h3></p> <p>Lately, some <a href="https://github.com/Raku/problem-solving/issues/391">unhappiness</a> has popped up about <code>Range</code> and it’s incomplete numericaliness. Having just one <a href="https://rakujourney.wordpress.com/2023/10/24/raku-home-on-the-range/">blogpost</a> about it is clearly not enough, given how big <code>Range</code>s can be.</p> <pre class="wp-block-code"><code>say (-∞..∞).elems; # Cannot .elems a lazy list in block <unit> at tmp/2021-03-08.raku line 2629</code></pre> <p>I don’t quite agree with Rakudo here. There are clearly ∞ elements in that lazy list. This could very well be special-cased.</p> <p>The argument has been made, that many operators in Raku tell you what type the returned value will have. Is that so? (This question is always silly or unnecessary.)</p> <pre class="wp-block-code"><code>say (1 + 2&3).WHAT; # (Junction)</code></pre> <p>Granted, <code>Junction</code> is quite special. But so are <code>Range</code>s. Yet, Raku covers the former everywhere but the latter feels uncompleted. Please consider the following code.</p> <pre class="wp-block-code"><code>multi sub infix:<±>(Numeric \n, Numeric \variance --> Range) { (n - variance) .. (n + variance) } say 2.6 > 2 ± 0.5; # True my @heavy-or-light = 25.6, 50.3, 75.4, 88.8; @heavy-or-light.map({ $_ ≤ 75 ± 0.5 ?? „$_ is light“ !! „$_ is heavy“ }).say; # (25.6 is heavy 50.3 is heavy 75.4 is heavy 88.8 is heavy)</code></pre> <p>To me that looks like it should DWIM. It doesn’t, because <code>&infix:«≤»</code> defaults to coercing to <code>Real</code> and then comparing numerically.</p> <p>This could easily be fixed by adding a few more multis and I don’t think it would break any production code. We already provide quite a few good tools for scientists. And those scientists do love their error bars — which are ranges. I would love for them to have another reason to use Raku over … that other language.</p> <h2><a href="https://vrurg.github.io/2023/10/17/new-json-class">A New JSON::Class Module. All New.</h2></a> <p><h3>Published by Vadim Belman on 2023-10-17T00:00:00</h3></p> <p>This will be a short one. I have recently released a family of <code class="language-plaintext highlighter-rouge">WWW::GCloud</code> modules for accessing Google Cloud services. Their REST API is, apparently, JSON-based. So, I made use of the existing <a href="https://raku.land/zef:jonathanstowe/JSON::Class"><code class="language-plaintext highlighter-rouge">JSON::Class</code></a>. Unfortunately, it was missing some features critically needed for my work project. I implemented a couple of workarounds, but still felt like it’s not the way it has to be. Something akin to <a href="https://raku.land/zef:vrurg/LibXML::Class"><code class="language-plaintext highlighter-rouge">LibXML::Class</code></a> would be great to have…</p> <p>There was a big “but” in this. We already have <a href="https://raku.land/?q=XML%3A%3AClass"><code class="language-plaintext highlighter-rouge">XML::Class</code></a>, <code class="language-plaintext highlighter-rouge">LibXML::Class</code>, and the current <code class="language-plaintext highlighter-rouge">JSON::Class</code>. All are responsible for doing basically the same thing: de-/serializing classes. If I wanted another JSON serializer then I had to take into account that <code class="language-plaintext highlighter-rouge">JSON::Class</code> is already taken. There are three ways to deal with it:</p> <ol> <li>Branch the current <code class="language-plaintext highlighter-rouge">JSON::Class</code> and re-implement it as a backward-incompatible version.</li> <li>Give the new module a different name.</li> <li>Implement own version and publish it under my name.</li> </ol> <p>The first two options didn’t appeal to me. The third one is now about to happen.</p> <p>I expect it to be a stress-test for Raku ecosystem as, up to my knowledge, it’s going to be the first case where two different modules share the same name but not publishers.</p> <p>As a little reminder:</p> <ul> <li>To use the old module one would have to have <code class="language-plaintext highlighter-rouge">JSON::Class:auth<zef:jonathanstowe></code> in their dependencies and, perhaps, in their <code class="language-plaintext highlighter-rouge">use</code> statement.</li> <li>The new module will be available as <code class="language-plaintext highlighter-rouge">JSON::Class:auth<zef:vrurg></code>.</li> </ul> <p>There is still some time before I publish it because the documentation is not ready yet.</p> <p>Let’s 🤞🏻.</p> <h2><a href="https://andrewshitov.com/2023/09/05/sorting-numbers-in-raku-with-the-help-of-chatgpt/">Sorting numbers in Raku with the help of ChatGPT</h2></a> <p><h3>Published by Andrew Shitov on 2023-09-05T13:58:16</h3></p> <p>In this post I will show you the solution to the <a href="https://theweeklychallenge.org/blog/perl-weekly-challenge-233/#TASK2">Task 2 of this week’s Weekly Challenge 233</a>. This time I was using the help of ChatGPT, which offered a very nice solution.</p> <div class="wp-block-group"><div class="wp-block-group__inner-container"> <h2 id="TASK2">Task 2: Frequency Sort</h2> <p>You are given an array of integers.</p> <p>Write a script to sort the given array in increasing order based on the frequency of the values. If multiple values have the same frequency then sort them in decreasing order.</p> <p><strong>Example 1</strong></p> <pre class="wp-block-code"><code>Input: @ints = (1,1,2,2,2,3) Ouput: (3,1,1,2,2,2) '3' has a frequency of 1 '1' has a frequency of 2 '2' has a frequency of 3 </code></pre> <p><strong>Example 2</strong></p> <pre class="wp-block-code"><code>Input: @ints = (2,3,1,3,2) Ouput: (1,3,3,2,2) '2' and '3' both have a frequency of 2, so they are sorted in decreasing order. </code></pre> <p><strong>Example 3</strong></p> <pre class="wp-block-code"><code>Input: @ints = (-1,1,-6,4,5,-6,1,4,1) Ouput: (5,-1,4,4,-6,-6,1,1,1)</code></pre> </div></div> <p></p> <p>I asked ChatGPT the following: ‘Create a program. Here is the description of the task.’ To make sure the result is in the Raku programming language, I also added the following instruction in my local settings so that it knows which programming language I prefer by default.</p> <div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img src="https://andrewshitov.com/wp-content/uploads/2023/09/Screenshot-2023-09-05-at-15.49.04-1024x648.png" alt="" class="wp-image-7031" width="484" height="305" srcset="https://andrewshitov.com/wp-content/uploads/2023/09/Screenshot-2023-09-05-at-15.49.04-1024x648.png 1024w, https://andrewshitov.com/wp-content/uploads/2023/09/Screenshot-2023-09-05-at-15.49.04-300x190.png 300w, https://andrewshitov.com/wp-content/uploads/2023/09/Screenshot-2023-09-05-at-15.49.04-768x486.png 768w, https://andrewshitov.com/wp-content/uploads/2023/09/Screenshot-2023-09-05-at-15.49.04.png 1112w" sizes="(max-width: 484px) 85vw, 484px" /></figure></div> <p>The solution generated by ChatGPT was quite good with a small mistake.</p> <p><em><strong>ChatGPT:</strong> Sure, here’s a Raku program that implements the described sorting algorithm for the given task:</em></p> <pre class="wp-block-preformatted">sub frequency-sort(@arr) { my %freq; %freq{$_}++ for @arr; my @sorted = @arr.sort: { %freq{$^b} <=> %freq{$^a} || $^b <=> $^a }; return @sorted; } my @ints = (1, 1, 2, 2, 2, 3); my @sorted = frequency-sort(@ints); say "Input: @ints"; say "Output: @sorted";</pre> <p>First, Raku would not interpolate arrays, so <code>say "Input: @ints"</code> prints the string literary: <code>Input: @ints</code>. Second, in the sorting rule, sorting by frequency must be ascending rather than descending, so we need to switch the placeholder variables <code>$^b</code> and <code>$^a</code>. </p> <p>In the rest, ChatGPT did a good work. It computed the frequency using a loop, and I especially liked how it addressed the requirement to sort by two dimensions.</p> <p>Here is the full program with the above-mentioned corrections:</p> <pre class="wp-block-preformatted">sub frequency-sort(@arr) { my %freq; %freq{$_}++ for @arr; my @sorted = @arr.sort: { <strong>%freq{$^a} <=> %freq{$^b} || </strong> <strong>$^b <=> $^a</strong> }; return @sorted; } my @tests = (1, 1, 2, 2, 2, 3), (2, 3, 1, 3, 2), (-1, 1, -6, 4, 5, -6, 1, 4, 1); say frequency-sort($_) for @tests;</pre> <p>This program prints the following output for the given test cases:</p> <pre class="wp-block-preformatted">$ raku ch-2.raku [3 1 1 2 2 2] [1 3 3 2 2] [5 -1 4 4 -6 -6 1 1 1]</pre> <ul><li><em><a href="https://github.com/ash/perlweeklychallenge-club/tree/master/challenge-233/ash/raku">Code on GitHub</a></em></li><li><em><a href="https://andrewshitov.com/raku-challenges-index/">Navigation to the Raku challenges post series</a></em></li></ul> <h2><a href="https://andrewshitov.com/2023/09/04/working-with-words-in-the-raku-programming-language/">Working with words in the Raku programming language</h2></a> <p><h3>Published by Andrew Shitov on 2023-09-04T19:46:41</h3></p> <p>In this post, I will demonstrate my solution to another <a href="https://theweeklychallenge.org/blog/perl-weekly-challenge-233/#TASK1">Task of The Weekly Challenge, week 233</a>. Here’s how it reads:</p> <div class="wp-block-group"><div class="wp-block-group__inner-container"> <h2>Similar words</h2> <p>You are given an array of words made up of alphabets only.</p> <p>Write a script to find the number of pairs of similar words. Two words are similar if they consist of the same characters.</p> <p><strong>Example 1</strong></p> <pre class="wp-block-code"><code>Input: @words = ("aba", "aabb", "abcd", "bac", "aabc") Output: 2 Pair 1: similar words ("aba", "aabb") Pair 2: similar words ("bac", "aabc") </code></pre> <p><strong>Example 2</strong></p> <pre class="wp-block-code"><code>Input: @words = ("aabb", "ab", "ba") Output: 3 Pair 1: similar words ("aabb", "ab") Pair 2: similar words ("aabb", "ba") Pair 3: similar words ("ab", "ba") </code></pre> <p><strong>Example 3</strong></p> <pre class="wp-block-code"><code>Input: @words = ("nba", "cba", "dba") Output: 0</code></pre> </div></div> <p></p> <p>There’s a slight moment that may be needs extra comments. In the second example all three words constructed of the same two letters, <em>a</em> and <em>b</em>. So, all of the three words match the definition of a ‘similar’ word. But as the task needs to find pairs, we need to construct all the possible pairs out of those three words.</p> <p>In my solution, I chose to use a handy <code>classify</code> method. For an array, it creates a hash, where the keys are the common classifying symbol, and the values are the lists of the input elements that match this classification property.</p> <p>Here is the whole first program together with all the test cases provided in the description. The program maps every word to a corresponding string that consists of the sorted unique letters in the word.</p> <pre class="wp-block-preformatted">my @tests = ["aba", "aabb", "abcd", "bac", "aabc"], ["aabb", "ab", "ba"], ["nba", "cba", "dba"]; for @tests -> @words { say <strong>@words.classify(*.comb.unique.sort.join)</strong>.grep(*.value.elems > 1); }</pre> <p>For example, the word <em>aba</em> will be associated with the key <em>ab</em>. The program prints the following output:</p> <pre class="wp-block-preformatted">$ raku ch-1.raku (ab => [aba aabb] abc => [bac aabc]) (ab => [aabb ab ba]) ()</pre> <p>The format of the output differs from the examples, but it can be enhanced if needed. My goal was to create a compact solution <img src="https://s.w.org/images/core/emoji/12.0.0-1/72x72/1f609.png" alt="😉" class="wp-smiley" style="height: 1em; max-height: 1em;" /></p> <p>But I would assume that you’d be interested in looking at what <code>classify</code> produces. I am also curious. For the same <code>@tests</code>, it returns the following three hashes:</p> <pre class="wp-block-preformatted">{ab => [aba aabb], abc => [bac aabc], abcd => [abcd]} {ab => [aabb ab ba]} {abc => [cba], abd => [dba], abn => [nba]}</pre> <p>As you see, each string was put into one of the classification bins.</p> <p>The second part of the task is to find <em>pairs</em>. After the <code>grep</code>, we already filtered out everything that has less than two elements, so if data passed through this filter, there will be at least one pair. For bigger arrays, we can use another Raku’s built-in mechanism: the <code>combinations</code> method.</p> <p>The updated mail loop of the program looks like this now.</p> <pre class="wp-block-preformatted">for @tests -> @words { say "Test case: ", @words; my %classification = @words.classify(*.comb.unique.sort.join).grep(*.value.elems > 1); my $pairs = 0; for %classification.kv -> $k, $v { <strong>my @pairs = $v.combinations(2);</strong> <strong>$pairs += @pairs.elems;</strong> say "$k: ", @pairs; } say "Answer: $pairs pair{$pairs == 1 ?? '' !! 's'}.\n"; }</pre> <p>The ‘redundant’ code here is added just to have a more detailed output so that we can see which pairs were actually found. Let us look at the output for the initial test cases:</p> <pre class="wp-block-preformatted">$ raku ch-1.raku Test case: [aba aabb abcd bac aabc] ab: [(aba aabb)] abc: [(bac aabc)] Answer: 2 pairs. Test case: [aabb ab ba] ab: [(aabb ab) (aabb ba) (ab ba)] Answer: 3 pairs. Test case: [nba cba dba] Answer: 0 pairs.</pre> <ul><li><a href="https://github.com/ash/perlweeklychallenge-club/tree/master/challenge-233/ash/raku">Code on GitHub</a></li><li><a href="https://andrewshitov.com/raku-challenges-index/">Navigation to the <em>Raku challenges</em> post series</a></li></ul> <h2><a href="https://andrewshitov.com/2023/08/21/a-couple-of-tasks-solved-in-raku/">A couple of tasks solved in Raku</h2></a> <p><h3>Published by Andrew Shitov on 2023-08-21T11:31:17</h3></p> <p>On this page, I’ll briefly cover the solutions to the tasks for this week’s <a href="https://theweeklychallenge.org/blog/perl-weekly-challenge-231/">Weekly Challenge #231</a>.</p> <a name="task1"></a><h2>Task 1</h2> <div class="wp-block-group"><div class="wp-block-group__inner-container"> <p>You are given an array of distinct integers.</p> <p>Write a script to find all elements that is neither minimum nor maximum. Return -1 if you can’t.</p> <p><strong>Example 1</strong></p> <pre class="wp-block-code"><code>Input: @ints = (3, 2, 1, 4) Output: (3, 2) The minimum is 1 and maximum is 4 in the given array. So (3, 2) is neither min nor max. </code></pre> <p><strong>Example 2</strong></p> <pre class="wp-block-code"><code>Input: @ints = (3, 1) Output: -1 </code></pre> <p><strong>Example 3</strong></p> <pre class="wp-block-code"><code>Input: @ints = (2, 1, 3) Output: (2) The minimum is 1 and maximum is 3 in the given array. So 2 is neither min nor max.</code></pre> </div></div> <p></p> <p>Here is my original solution in the Raku programming language.</p> <pre class="wp-block-preformatted">sub solve(@data) { @data.grep: * != (@data.min, @data.max).any }</pre> <p>As the tasks requires that we print <code>-1</code> when there are no elements in the output, let us add an update to satisfy this requirement:</p> <pre class="wp-block-preformatted">sub solve(@data) { <strong>(</strong>@data.grep: * != (@data.min, @data.max).any<strong>) || -1</strong> }</pre> <p>The <code>*</code> in this code will actually replace the <code>$_</code> variable. Would you prefer it, you may use <code>$_</code>, but you’ll need parentheses in this case. So, instead of <code>@data.grep: * != ...</code>, you need <code>@data.grep({$_ != ...})</code>, which may be a less clear code for some people.</p> <p>Finally, let us use some math notation and replace calling the <code>.any</code> method with a ‘contains’ operator:</p> <pre class="wp-block-preformatted">sub solve(@data) { (@data.grep: * <strong>∉</strong> (@data.min, @data.max)) || -1 }</pre> <p>Well, actually, ‘does not contain’. And this is my final solution. </p> <p>Note that you may want to use the <code>.minmax</code> method instead of two calls to <code>.min</code> and <code>.max</code>, but <code>.minmax</code> returns a range, which is not that suitable for this task.</p> <p>Adding some test cases and passing them to the <code>solve</code> function:</p> <pre class="wp-block-preformatted">my @tests = (3, 2, 1, 4), (3, 1), (2, 1, 3); say solve($_) for @tests;</pre> <p>The program prints the expected output:</p> <pre class="wp-block-preformatted">$ raku ch-1.raku (3 2) -1 (2)</pre> <a name="task2"></a><h2>Task 2</h2> <div class="wp-block-group"><div class="wp-block-group__inner-container"> <p>You are given a list of passenger details in the form “9999999999A1122”, where 9 denotes the phone number, A the sex, 1 the age and 2 the seat number.</p> <p>Write a script to return the count of all senior citizens (age >= 60).</p> <h4 id="example-1-1">Example 1</h4> <pre class="wp-block-code"><code>Input: @list = ("7868190130M7522","5303914400F9211","9273338290F4010") Ouput: 2 The age of the passengers in the given list are 75, 92 and 40. So we have only 2 senior citizens. </code></pre> <h4 id="example-2-1">Example 2</h4> <pre class="wp-block-code"><code>Input: @list = ("1313579440F2036","2921522980M5644") Ouput: 0</code></pre> </div></div> <p></p> <p>Apparently, the solution requires extracting information from a string in a specific format. It is not quite clear from the description whether the strings always contains the same number of characters, and thus the age and seat number are always two-digit values. But let’s use this assumption.</p> <p>As we do not need any other information from the ticket code, no need to properly parse it, so I preferred anchoring around the only letter in the string and consider the next two digits as the age. Of course, you may make it simpler and just extract the two digits counting from the end of the string.</p> <pre class="wp-block-preformatted">sub is-sinior($ticket) { ~($ticket ~~ / <strong><alpha> (\d\d)</strong> /)[0] >= 75 }</pre> <p>Unlike Perl 5, Raku ignores spaces in regexes by default, so I added some air to it. On the other hand, extracting matches may seem a bit more complicated.</p> <p>For the first given example (see task’s description), the Match object contains the following information:</p> <pre class="wp-block-preformatted">｢M75｣ alpha => ｢M｣ 0 => ｢75｣</pre> <p>So, I am taking the 0<sup>th</sup> element using <code>[0]</code> and stringily it with the <code>~</code> prefix operator.</p> <p>In essence, the task has been solved. Let’s add the test cases and run them:</p> <pre class="wp-block-preformatted">my @tests = ('7868190130M7522', '5303914400F9211', '9273338290F4010'), ('1313579440F2036', '2921522980M5644'); for @tests -> @tickets { say <a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="674c27130e040c0213144900150217">[email protected]</a>({is-sinior($_)}); }</pre> <p>The program prints:</p> <pre class="wp-block-preformatted">$ raku ch-2.raku 2 0</pre> <p></p> <p class="has-text-align-center">* * *</p> <ul><li><a href="https://github.com/ash/perlweeklychallenge-club/tree/master/challenge-231/ash/raku">Code on GitHub</a></li><li><a href="https://andrewshitov.com/raku-challenges-index/">Navigation to the <em>Raku challenges</em> post series</a></li></ul> <h2><a href="https://dev.to/finanalyst/rakudoc-revision-open-to-comment-4hjb">RakuDoc revision open to comment</h2></a> <p><h3>Published by Richard Hainsworth on 2023-07-31T23:00:00</h3></p> <p>The second stage in the process to update RakuDoc is now over and the third (GAMMA review) stage is starting. In order not to repeat some history, please take a look at <a href="https://dev.to/finanalyst/revising-rakudoc-29d4">Revising Rakudoc</a>.</p> <p>An online version is available of <a href="https://new-raku.finanalyst.org/language/rakudoc">the proposed RakuDoc language</a>.</p> <p>The whole of <a href="//docs.raku.org">the Raku documentation suite</a> is written in RakuDoc.</p> <h2> Improving on a good design </h2> <p>About half of the original design ideas outlined in <a href="https://design.raku.org/S26.html">S26</a> were documented in <a href="https://docs.raku.org/language/pod">current POD6</a>. Some of the ideas were available, but not documented. Some instructions were not realised at all. </p> <p>It should be remembered that RakuDoc is parsed by the compiler (eg. Rakudo) as part of a Raku program, and is then rendered by the renderer (eg. <code>Raku::Pod::Render</code>) into (for example) HTML. When I use the word 'implemented', I mean that a RakuDoc instruction is properly parsed and rendered. Some of the instructions defined in S26 were parsed by Rakudo, but <strong><em>not rendered</em></strong>, and some were <strong><em>not parsed</em></strong> properly or at all, so could not be rendered. </p> <p>The revision process has therefore identified and rectified the parsing deficiencies, and identified the rendering flaws. RakuDoc is correctly parsed only on the most recent versions of Rakudo, which at the time of writing has yet to be released. <code>Raku::Pod::Render</code> still does not handle RakuDoc in its entirety. </p> <h2> Two use cases </h2> <p>It became clear that the RakuDoc serves two inter-related use cases:</p> <ol> <li>Documenting code for developing and maintaining software. <ul> <li>explanations about variables, methods, etc.</li> <li>another program that uses the software as a dependency should be able to access these explanations</li> <li>data which should be included with the software but not be hard coded into it, for example, a unit test needs to populate a data structure with test data.</li> </ul> </li> <li>Documenting the software for use by another user. <ul> <li>Good documentation should have tutorials, code snippets, headings, Tables of content, etc</li> </ul> </li> </ol> <h2> Tables </h2> <p>RakuDoc had a simple table markup, which is very similar to the Markdown syntax. It worked, but the simplicity of the syntax was at the cost of flexibility. </p> <p>Looking around at other ways of specifying a table, we identified two paradigms (there may be more), namely the one used by HTML and the one used by the GTK grid widget. Both of them allow for cells that span more than one column or row, and both allow for embedding (eg. a table inside a cell of a table).</p> <p>After several iterations, a new <em>procedural</em> model was created and rendered. The design allows for spanning and embedding, but it also allows an author to specify a table row by row, or column by column, or even using a mixture of both.</p> <p>An example showing a markup using both rows and columns can be seen in <a href="https://new-raku.finanalyst.org/language/rakudoc#Procedural_description_of_tables">the online draft</a>.</p> <h2> Semantic blocks </h2> <p>A semantic block is a section of text that should be easily available to another software tool, or can be moved around the final document. </p> <p>For example, a section on the authors of a document (including contact or affiliations) is most easily written at the top of the document, but often it is better to place the information towards the bottom of the text. </p> <p>This is done by creating a semantic block (simply by making the calling the block in uppercase letters). The block can be hidden from view by adding the metadata option <code>:hidden</code>. All the data is placed in a special structure.</p> <p>The rendered text can be placed in the document later using the <code>P<></code> instruction, or it can be accessed by another tool that may only be wanting the VERSION or LICENSE. </p> <h2> More metadata options </h2> <p>One of the strengths of RakuDoc is the ability to add optional metadata to blocks of text.</p> <p>The new version of the defining document explains this concept in more detail. Metadata options are optional, with reasonable defaults being assumed. This means that a short form of the block is sufficient in most cases.</p> <p>In the description above, the option <code>:hidden</code> was mentioned. Another example, is <code>:caption</code>. Suppose you want to write a semantic block called <code>=AUTHORS</code> at the start of the document, but you want for it to appear later in the document as <code>Article authors</code>, then you could specify it as follows:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>=for AUTHORS :caption<Article authors> :hidden A. N. Writer, socMedia nic @psuedonym M. Z. Orator, socMedia nic @politician Article text continues Pages later P<semantic: AUTHORS> </code></pre> </div> <p>It is possible to include a link <code>L<for reference see | #A very long title somewhere in the text></code> where the text on the right-hand side of the <code>|</code> is a heading. However, this can become tiresome if you want to include several links to the same place. </p> <p>So, a metadata option <code>:id</code> can be included in a heading. This allows you to do the following:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight plaintext"><code>=for head3 :id<lnk> How to correctly link to other places in a manual Pages of text Properly linking is important, L<see for example|#lnk> </code></pre> </div> <h2> Doing things in line </h2> <p>RakuDoc has instructions for block level text, such as headings, paragraphs, code. </p> <p>Typically blocks will be included in the Table of Contents. <br> It also has <strong><em>markup</em></strong> instructions that work in line, and which do not (typically) affect the ToC. </p> <p>For example, a simple markup instruction is <code>C< text ></code>, which renders like <code>text</code>. I have used the Markdown equivalent here. In RakuDoc, everything between the <code>C<</code> and <code>></code> is verbatim and styled differently to normal text, just like the Markdown code quotes. However, RakuDoc also has <code>V< text ></code> which treats everything inside the angle brackets as verbatim but does not style it differently.</p> <p>A new markup instruction in RakuDoc is <code>M< text | metadata></code>. A renderer will place the <em>text</em> in the rendered text, but will also provide a mechanism for the user to take the metadata and provide new functionality. For instance, <code>M< fa-copy | font awesome v5 ></code> could be interpreted to insert the <code>font-awesome</code> icon called <code>fa-copy</code> into the text. Or <code>M< Buy now | PayPal, database-id ></code> could expose the API for the PayPal payment platform.</p> <h2> How not to be confusing </h2> <p>RakuDoc is inherently customisable. It is also designed to be output neutral (although at the moment HTML is the most common output form). Semantic blocks can be invented within a document, and a renderer can allow for other user-defined blocks and markup instructions to be created. </p> <p>However, RakuDoc is specific about naming rules. A built-in block must be all lower case, and renderers should not allow user-defined blocks to use all lower case. A semantic block is all upper case. And a user-defined block must have at least one upper-case letter and one lower-case letter.</p> <p>All markup instructions, which are inline instructions, must be a single Unicode character with the property UPPER. Built-in markup instructions are the ASCII characters and <code>Δ</code>. All other codes can be used.</p> <p>The naming rules have been created to ensure that even if a user-defined block or markup becomes popular, it is not a part of the RakuDoc standard. Renderers are only required to implement the RakuDoc standard, and may render other blocks, or not.</p> <h2> Wrapping up </h2> <p>These are some of the interesting additions to RakuDoc that are being proposed. There are more.</p> <p>Since the Gamma review stage is now underway, it is almost certain that there may be more changes because the revision is now open to the Raku community for comment and requests. Discussion is open both for the language design and for the explanation of the design. </p> <p>As might be admitted, community requests for changes to the overall design will face significant resistance from the main authors in order to maintain backwards compatibility with the previous version of RakuDoc, and the integrity of the underlying paradigms. New block or inline instructions will be more readily considered, but requests for examples, explanation, and greater clarity will be very much appreciated.</p> <h2><a href="https://vrurg.github.io/2023/07/19/easy-peasy-service-from-a-role">Easy-peasy Service From A Role</h2></a> <p><h3>Published by Vadim Belman on 2023-07-19T00:00:00</h3></p> <p>I was always concerned about making things easier.</p> <p>No, not this way. A technology must be easy to start with, but also be easy in accessing its advanced or fine-tunable features. Let’s have an example of the former.</p>  <h2 id="disclaimer">Disclaimer</h2> <p>This post is a quick hack, no proof-reading or error checking is done. Please, feel free to report any issue.</p> <h1 id="the-task">The Task</h1> <p>Part of my ongoing project is to deal with JSON data and deserialize it into Raku classes. This is certainly a task for <a href="https://raku.land/zef:jonathanstowe/JSON::Class"><code class="language-plaintext highlighter-rouge">JSON::Class</code></a>. So far, so good.</p> <p>The keys of JSON structures tend to use lower camel case which is OK, but we like <a href="https://en.wikipedia.org/wiki/Naming_convention_(programming)">kebabing</a> in Raku. Why not, there is <a href="https://raku.land/zef:jonathanstowe/JSON::Name"><code class="language-plaintext highlighter-rouge">JSON::Name</code></a>. But using it:</p> <ul> <li>Will hide the original names. This would break the principle of easy start because one would rather expect to see them as attributes of an object. Having both the original naming and kebabed would be more desirable.</li> <li>Would require a lot of manual work on my side.</li> </ul> <h1 id="the-assets">The Assets</h1> <p>There are roles. At the point I came to the final solution I was already doing something like<sup id="fnref:fictional-naming" role="doc-noteref"><a href="#fn:fictional-naming" class="footnote" rel="footnote">1</a></sup>:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class SomeStructure does JSONRecord {...} </code></pre></div></div> <p>Then there is <a href="https://raku.land/zef:vrurg/AttrX::Mooish"><code class="language-plaintext highlighter-rouge">AttrX::Mooish</code></a>, which is my lifevest on many occasions:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>use AttrX::Mooish; class Foo { has $.foo is mooish(:alias<bar>); } my $obj = Foo.new: bar => "the answer"; say $obj.foo; # the answer </code></pre></div></div> <p>Apparently, this way it would still be a lot of manual interaction with aliasing, and that’s what I was already doing for a while until realized that there is a bettter way. But be back to this later…</p> <p>And, eventually, there are traits and MOP.</p> <h1 id="the-solution">The Solution</h1> <h2 id="name-translation">Name Translation</h2> <p>That’s the easiest part. What I want is to <code class="language-plaintext highlighter-rouge">makeThisName</code> look like <code class="language-plaintext highlighter-rouge">make-this-name</code>. Ha, big deal!</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>unit module JSONRecord::Utils; our sub kebabify-attr(Attribute:D $attr) { if $attr.name ~~ /<.lower><.upper>/ { my $alias = (S:g/<lower><upper>/$<lower>-$<upper>/).lc given $attr.name.substr(2); ... } } </code></pre></div></div> <p>I don’t export the sub because it’s for internal use mostly. Would somebody need it for other purposes it’s a rare case where a long name like <code class="language-plaintext highlighter-rouge">JSONRecord::Utils::kebabify-attr($attr)</code> must not be an issue.</p> <p><em>The sub is not optimal, it’s what I came up with while expermineting with the approach. The number of method calls and regexes can be reduced.</em></p> <p>I’ll get back later to the yada-yada-yada up there.</p> <h2 id="automate-attribute-processing">Automate Attribute Processing</h2> <p>Now we need a bit of MOP magic. To handle all attributes of a class we need to iterate over them and apply the aliasing. The first what comes to mind is to use role body because it is invoked at the early class composition times:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>unit role JSONRecord; for ::?CLASS.^attributes(:local) -> $attr { # take care of it... } </code></pre></div></div> <p>Note the word “early” I used above. It actually means that when role’s body is executed there are likely more roles waiting for their turn to be composed into the class. So, there are likely more attributes to be added to the class.</p> <p>But we can override <code class="language-plaintext highlighter-rouge">Metamodel::ClassHOW</code> <code class="language-plaintext highlighter-rouge">compose_attributes</code> method of our target <code class="language-plaintext highlighter-rouge">::?CLASS</code> and rest assured no one would be missed:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>unit role JSONRecordHOW; use JSONRecord::Utils; method compose_attributes(Mu \obj, |) { for self.attributes(obj, :local) -> $attr { # Skip if it already has `is mooish` trait applied – we don't want to mess up with user's intentions. next if $attr ~~ AttrX::Mooish::Attribute; JSONRecord::Utils::kebabify-attr($attr); } nextsame } </code></pre></div></div> <h2 id="the-role-does-it-all">The Role Does It All</h2> <p>Basically, that’s all we currently need to finalize the solution. We can still use role’s body to implement the key elements of it:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>unit role JSONRecord; use JSONRecordHOW; unless ::?CLASS.HOW ~~ JSONRecordHOW { ::?CLASS.HOW does JSONRecordHOW; } </code></pre></div></div> <p>Job done! <em>Don’t worry, I haven’t forgot about the yada-yada-yada above!</em></p> <p>But…</p> <p>The original record role name itself is even longer than <code class="language-plaintext highlighter-rouge">JSONRecord</code>, and it consists of three parts. I’m lazy. There are a lot of JSON structures and I want less typing per each. A trait? <code class="language-plaintext highlighter-rouge">is jrecord</code>?</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>unit role JSONRecord; multi sub trait_mod:<is>(Mu:U \type, Bool:D :$jrecord) is export { unless type.HOW ~~ JSONRecordHOW { type.HOW does JSONRecordHOW type.^add_role(::?ROLE); } } </code></pre></div></div> <p>Now, instead of <code class="language-plaintext highlighter-rouge">class SomeRecord does JSONRecord</code> I can use <code class="language-plaintext highlighter-rouge">class SomeRecord is jrecord</code>. In the original case the win is even bigger.</p> <h2 id="the-yada">The Yada???</h2> <p>There is absolutely nothing funny about it. Just a common way to keep a reader interested!</p> <p>Seriously.</p> <p>The reason for the yada in that snippet is to avoid a distraction from the primary purpose of the example. Here is what is going on there:</p> <p>I want <code class="language-plaintext highlighter-rouge">AttrX::Mooish</code> to do the dirty work for me. Eventually, what is needed is to apply the <code class="language-plaintext highlighter-rouge">is mooish</code> trait as shown above. But the traits are just subs. Therefore all is needed now is to:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&trait_mod:<is>($attr, :mooish(:$alias)); </code></pre></div></div> <p>Because this is what Raku does internally when encounters <code class="language-plaintext highlighter-rouge">is mooish(:alias(...))</code>. The final version of the kebabifying sub is:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>our sub kebabify-attr(Attribute:D $attr) { if $attr.name ~~ /<.lower><.upper>/ { my $alias = (S:g/<lower><upper>/$<lower>-$<upper>/).lc given $attr.name.substr(2); &trait_mod:<is>($attr, :mooish(:$alias)); } } </code></pre></div></div> <p>Since the sub is used by the HOW above, we can say that the <code class="language-plaintext highlighter-rouge">&trait_mod<is></code> would be called at compile time<sup id="fnref:not-always" role="doc-noteref"><a href="#fn:not-always" class="footnote" rel="footnote">2</a></sup>.</p> <h2 id="the-use">The Use</h2> <p>Now, it used to be:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class SomeRecord does JSONRecord { has $.aLongAttrName is mooish(:alias<a-long-attr-name>); has $.shortname; } </code></pre></div></div> <p>Where, as you can see, I had to transfer JSON key names to attribute names, decide where aliasing is needed, add it, and make sure no mistakes were made or attributes are missed.</p> <p>With the above rather simple tweaks:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class SomeRecord is jrecord { has $.aLongAttrName; has $.shortname; } </code></pre></div></div> <p>Job done.</p> <h2 id="the-stupidy">The Stupidy</h2> <p>Before I came down to this solution I’ve got 34 record classes implemented using the old approach. Some are little, some are quite big. But it most certainly could’ve taken much less time would I have the trait at my disposal back then…</p> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:fictional-naming" role="doc-endnote"> <p>Naming is totally fictional. <a href="#fnref:fictional-naming" class="reversefootnote" role="doc-backlink">↩</a></p> </li> <li id="fn:not-always" role="doc-endnote"> <p>Most likely, but there are exceptions. It barely changes a lot, but certainly falls out of the scope of this post. <a href="#fnref:not-always" class="reversefootnote" role="doc-backlink">↩</a></p> </li> </ol> </div> <h2><a href="https://vrurg.github.io/2023/07/05/another-article-before-a-pause">Another Article Before A Break</h2></a> <p><h3>Published by Vadim Belman on 2023-07-05T00:00:00</h3></p> <p>I have managed to finish one more article in the <a href="/arfb.html">Advanced Raku For Beginners</a> series, this time about <a href="/arfb-publication/09-type-and-object-composition/">type and object composition in Raku</a>.</p>  <p>It’s likely to take a long before I can write another.</p> <h2><a href="https://vrurg.github.io/2023/07/04/did-you-know-that">Did you know that…</h2></a> <p><h3>Published by Vadim Belman on 2023-07-04T17:24:07</h3></p> <p>Once, long ago, coincidentally a few people were asking the same question: <em>how do I get a method object of a class?</em></p>  <p>Answers to the question would depend on particular circumstances of the code where this functionality is needed. One would be about using MOP methods like <code class="language-plaintext highlighter-rouge">.^lookup</code>, the other is to use method name and indirect resolution on invocant: <code class="language-plaintext highlighter-rouge">self."$method-name"(...)</code>. Both are the most useful, in my view. But sometimes declaring a method as <code class="language-plaintext highlighter-rouge">our</code> can be helpful too:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class Foo { our method bar {} } say Foo::<&bar>.raku; </code></pre></div></div> <p>Just don’t forget that this way we always get the method of class <code class="language-plaintext highlighter-rouge">Foo</code>, even if a subclass overrides method <code class="language-plaintext highlighter-rouge">bar</code>.</p> <h2><a href="https://dev.to/finanalyst/revising-rakudoc-29d4">Revising Rakudoc</h2></a> <p><h3>Published by Richard Hainsworth on 2023-06-30T23:00:00</h3></p> <p>In the earliest days of Raku, Damian Conway specified a documentation markup language to accompany it. Since it was modeled on Perl's POD it was called <strong><code><sound of trumpets and dramatic pause></code></strong> POD6.</p> <p>The <a href="https://design.raku.org/S26.html">Specification of POD6 (S26)</a> was mostly incorporated without much extra explanation in the documentation suite. In this way, the description of POD6 was itself was an illustration of many of the features it documented, and some that it did not document.</p> <p>Since Raku is defined by its test suite, and not its documentation, there were other details of POD6 in the tests that were not documented, even in S26. </p> <p>Raku developed and morphed, but POD6 remained. The tooling for rendering the documentation sources needed updating, and the documentation site had to be modernised.</p> <h1> Upgrading the renderer </h1> <p>A project of mine was to upgrade the basic renderer that would transform POD6 to HTML, but allow for developers to customise the templates for each type of POD6 block type. (The first <code>Pod::To::HTML</code> renderer hard-coded representations of POD6 markup, eg. <code>B<this is bold></code> was <code><strong>this is bold</strong></code> and could not be changed.) </p> <p>It turned out that S26 allowed for much more than had been included in the first documentation sources, including custom blocks and custom markup.</p> <p>The project to upgrade the original HTML renderer morphed into <a href="https://github.com/finanalyst/raku-pod-render">Raku::Pod::Render</a>, and transforming a directory full of individual documentation sources into an interlinked and searchable set of documents required another layer of tooling <a href="https://github.com/finanalyst/collection">Collection</a>. For example, collecting together all the pages that can be grouped as tutorials, or reference, or language, and creating a separate page for them automatically. </p> <p>I covered these two projects in <a href="https://www.youtube.com/watch?v=Rmuyb9kSQhY">a presentation to RakuCon 2022</a>.</p> <p>Some of the original ideas in S26 had not been implemented, such as aliases and generic numbering. Other ideas had become outdated, such as a way to specify document encoding, which is now solved with Unicode. </p> <p>In addition, RakuAST (see <a href="https://dev.to/lizmat/rakuast-for-early-adopters-576n">RakuAST for early adopters</a> ) is on the horizon, which will radically change the speed of documentation processing.</p> <p>There are also two implementations of POD6, one in Raku and one in Javascript, namely <a href="https://github.com/podlite/podlite-desktop">Alexandr Zahatski's Podlite</a>.</p> <h1> Introducing Rakudoc </h1> <p>This was an ideal time to revisit POD6 and recast it into Rakudoc - new name for the markup language, and its new file extension ".rakudoc".</p> <p>I was invited to the first Raku Core Summit and I put together a presentation about the changes I thought needed to be made based on my own experience, but also using comments from other developers.</p> <p>We came to a number of consensus agreements about the minimal changes that were needed, and some extra functionality to handle new questions, such as documentation versioning.</p> <p>It was also clear that Rakudoc (aka POD6) has two separate parts: components that interact closely with the program being documented, and components that will be rendered separately into HTML (or an ebook). The documentation file needs to make this clear. </p> <p>I have now written the first draft of the revision and the documentation file that encapsulates it. An HTML version can be found at <a href="https://new-raku.finanalyst.org/language/rakudoc">new-raku.finanalyst.org/language/rakudoc</a>, alongside the <a href="https://new-raku.finanalyst.org/language/pod">old documentation file</a> and the <a href="https://new-raku.finanalyst.org/language/tables">simple table implementation</a>. I am planning future blogs to describe some of the proposed revisions.</p> <p>However, none of the revisions will break existing POD6, so Rakudoc should be backwards compatible with POD6. The version at <code>new-raku</code> is a VERY early first draft, and it will go through several review stages.</p> <p>The first Raku Core Summit was organised by <a href="https://mastodon.social/@lizmat">Elizabeth Mattijsen</a> and hosted by Elizabeth and Wendy at their home. It was a really good meeting and I am sincerely grateful for their generosity and hospitality. The summit was also supported by <a href="https://www.perlfoundation.org/">The Perl and Raku Foundation</a>, <a href="https://rootprompt.at">Rootprompt</a>, and <a href="https://www.edument.se">Edument</a>.</p> <h2><a href="https://6guts.wordpress.com/2023/06/18/recollections-from-the-raku-core-summit/">Recollections from the Raku Core Summit</h2></a> <p><h3>Published by jnthnwrthngtn on 2023-06-18T15:58:04</h3></p> <p>The first Raku Core Summit, a gathering of folks who work on “core” Raku things, was held on the first weekend of June, and I was one of those invited to attend. It’s certainly the case that I’ve been a lot less active in Raku things over the last 18 months, and I hesitated for a moment over whether to go. However, even if I’m not so involved day to day in Raku things at the moment, I’m still keen to see the language and its ecosystem move forward, and – having implemented no small amount of the compiler and runtime since getting involved in 2007 – I figured I’d find something useful to do there!</p> <p>The area I was especially keen to help with is RakuAST, something I started, and that I’m glad I managed to bring far enough that others could see the potential and were excited enough to pick it up and run with it.</p> <p>One tricky aspect of implementing Raku is the whole notion of BEGIN time (of course, this is also one of the things that makes Raku powerful and thus is widely used). In short, BEGIN time is about running code during the compile time, and in Raku there’s no separate meta-language; anything you can do at runtime, you can (in principle) do at compile time too. The problem at hand was what to do about references from code running at compile time to lexically scoped symbols in the surrounding scope. Of note, that lexical scope is still being compiled, so doesn’t really exist yet so far as the runtime is concerned. The current compiler deals with this by building up an entire flattened table of everything that is visible, and installing it as a fake outer scope while running the BEGIN-time code. This is rather costly, and the hope in RakuAST was to avoid this kind of approach in general.</p> <p>A better solution seemed to be at hand by spotting such references during compilation, resolving them, and fixating them – that is, they get compiled as if they were lookups into a constant table. (This copies the suggested approach for quasiquoted code that references symbols in the lexical scope of where the quasiquoted code appears.) This seemed promising, but there’s a problem:</p> <pre class="wp-block-code"><code>my $x = BEGIN %*ENV<DEBUG> ?? -> $x { note "Got $x"; foo($x) } !! -> $x { foo($x) };</code></pre> <p>It’s fine to post-declare subs, and so there’s no value to fixate. Thankfully, the generalized dispatch mechanism can ride to the rescue; we can:</p> <ol class="wp-block-list"> <li>Create a placeholder object with an attribute to hold the resolution</li> <li>Compile the lookup into a use of a dispatcher that reads this attribute and indicates that this is a constant result of the dispatch (so it is stored in the inline cache, and after specialization will be just as cheap as any other sub call). If the attribute is not set, that means we tried to run the code before declaring the sub, and the object can carry a bit of extra metadata in order to give a good error message.</li> <li>Keep track of this object in the compiler, and – upon declaration of the sub – install it into the placeholder object.</li> <li>Give an error if we reach the end of the compilation unit with an unfilled placeholder.</li> </ol> <p>When compiling Raku code, timing is everything. I knew this and tried to account for it in the RakuAST design from the start, but a couple of things in particular turned out a bit awkward.</p> <ul class="wp-block-list"> <li>Every node wanting to do something at BEGIN time would implement RakuAST::BeginTime, and in doing so would have its method to perform a BEGIN-time action called at the appropriate point. So far so good, for most program elements. However, it turns out that some larger program elements want to do things at compile time both at the point they start <em>and</em> at the point they end. I’d added a “I want my BEGIN time before my children” mechanism, but that didn’t help program elements that wanted action at both points. Thus, I started a branch that introduces parse time. For “leaf” elements that is the same as BEGIN time, but for things like packages and routines, which have a bunch of stuff on the inside, it happens before we go parsing their inner scope.</li> <li>The RakuAST tree is an object graph where every node knows its children, but nodes do not reference their parents. However, some program elements need to find out about their enclosing context, for example a method wants to know about the class it is being declared in. Thus I came up with a notion of attach targets (things that child nodes want to discover) and attaching nodes (the children that want to “attach” to a parent – I’m not sure I got the naming right here, in hindsight). I guess with my IDE work it also appealed that one might be able to introspect these kinds of relationships for tooling purposes – although in reality the attaching work was highly imperative anyway. But with parse and begin time clarified, it also seemed that attachment work could happen in either (or do different work in both), but also it was hazy exactly when attachment happened, and it could end up happening multiple times, which was fragile. Thus, while the notion of attachment targets should survive – probably with a better name – the need for a RakuAST::Attaching went away. My branch also took on its elimination.</li> </ul> <p>I got a decent way into this restructuring work during the core summit, and hope to find time soon to get it a bit further along (I’ve been a mix of busy, tired, and had an eye infection to boot since getting back from the summit, so thus far there’s not been time for it).</p> <p>I also took part in various other discussions and helped with some other things; those that are probably most worth mentioning are:</p> <ul class="wp-block-list"> <li>There was quite a bit of talk about Raku Doc (formerly Pod6) and tidying up some aspects of its design and implementation. I’m pleased to see it is getting a rather cleaner implementation in the RakuAST-based compiler frontend. (Story illustrating why: when developing the Comma IDE I built a small and very specialized Raku grammar to Java lexer/parser compiler, in order to nail the Raku language’s syntactic structure, and I followed what Rakudo’s grammar did relatively closely in all but two places: operator parsing – because it just needed to be different for the IntelliJ platform’s tree builder engine – and Pod6, because it was easier to read the spec and implement it afresh than it was to decipher Rakudo’s implementation of it!)</li> <li>There was a long-standing problem that looked like the regex engine massively leaked memory in certain cases, but nobody could pin down the leak. The reason was that it wasn’t actually leaking, it was just creating sufficient backtracking state to go quadratic in the size of the input string. MoarVM only stores an array header in the nursery (the region of memory it allocates in, and whose fullness is the trigger for doing a GC run); the array body is allocated using a standard allocator. There are good things about this, but a less good thing is that if you allocate loads of large arrays in quick succession, you’ll allocate lots of memory, but not eat much of the nursery, and so it won’t be cleaned up very soon. The regex in question was doing exactly that: since it captured into an inner cursor, and cursors are in principle immutable, then it ended up cloning the array at every backtracking step, and doing so in a fairly tight loop. Closer examination revealed that the cloning of the backtrack stack was, however, overly defensive; eliminating that copying led to a huge memory and time improvement. Still, it remains to make such large array allocations pressure the GC more (we already do a similar kind of thing for big integers).</li> <li>I provided a few MoarVM hints that helped leont get support for asynchronous UNIX domain sockets implemented. Apparently that unblocks having an asynchronous Postgres driver, which would be most welcome for use in Cro applications, where the request handling is asynchronous but the database queries end up growing the thread pool by really blocking threads.</li> </ul> <p>Thanks goes to Liz for organizing the summit, to Wendy for keeping everyone so well fed and watered, to the rest of attendees for many interesting discussions over the three days, to TPRF and Rootprompt for sponsoring the event, and to Edument for supporting my attendance.</p> <h2><a href="http://brrt-to-the-future.blogspot.com/2023/06/retrospective-of-moarvm-jit.html">Retrospective of the MoarVM JIT</h2></a> <p><h3>Published by Bart Wiegmans on 2023-06-10T15:33:00</h3></p> <p style="text-align: left;">Hi hackers! Today the MoarVM JIT project is nearly 9 years old. I was inspired by Jonathan's <a href="https://youtu.be/3lxuCr54ULE" target="_blank">presentation</a> reflecting on the development of MoarVM, to do the same for the MoarVM JIT, for which I have been responsible.<br /></p><p style="text-align: left;">For those who are unfamiliar, what is commonly understood as 'JIT compilation' for virtual machines is performed by two components in MoarVM.<br /></p><ul style="text-align: left;"><li style="text-align: left;">A framework for runtime type specialization ('<a href="https://6guts.wordpress.com/2017/11/05/moarvm-specializer-improvements-part-3-optimizing-code/">spesh</a>')</li><li style="text-align: left;">A native code generation backend for the specialized code (the 'JIT').</li></ul><p>This post refers only to the native code generation backend component. It, too, is split into two mostly-independent systems:</p><ul style="text-align: left;"><li>A backend that emits code directly from MoarVM instructions from machine code templates (the 'lego' JIT compiler).<br /></li><li>Another backend that transforms MoarVM instructions into an expression-based intermediate representation and compiles machine code based on that (the 'expression' compiler).<br /></li></ul><div style="text-align: left;"><h4 style="text-align: left;">Things that worked well</h4><ul style="text-align: left;"><li style="text-align: left;">Using <a href="https://luajit.org/dynasm.html" target="_blank">DynASM</a> for code generation. Even though we had to extend it to <a href="https://brrt-to-the-future.blogspot.com/2015/09/most-significant-bits.html">support register selection</a>, using DynASM saved a lot of time compared to generating code 'from scratch'.<br /></li><li style="text-align: left;">Textual opcode templates for the expression compiler (mapping from MoarVM instructions to the JIT intermediate representation)</li><li style="text-align: left;"><a href="https://brrt-to-the-future.blogspot.com/2018/09/template-compiler-update.html" target="_blank">Type analysis</a> for opcode templates (preventing bugs)<br /></li><li style="text-align: left;">Tiling for <a href="https://brrt-to-the-future.blogspot.com/2015/07/tiles-and-compiler-compilers.html">instruction selection</a></li><li style="text-align: left;">Using the <a href="https://brrt-to-the-future.blogspot.com/2018/06/controlled-stack-hacking-for-moarvm-jit.html" target="_blank">on-stack return address</a> as a current position marker (for the purpose of exception handling, lexical variable analysis etc.)</li></ul><h4 style="text-align: left;"> Things that didn't work so well </h4><ul style="text-align: left;"><li style="text-align: left;">Testing. MoarVM doesn't have a separate test suite but relies on the NQP and Rakudo tests. For the JIT compiler, this is essentially 'testing in production'. In hindsight, specific unit and integration tests would have been beneficial.</li><li style="text-align: left;">Using an unordered intermediate representation for the 'expression' JIT. Against my expectation this <a href="https://brrt-to-the-future.blogspot.com/2019/03/something-about-ir-optimization.html">prevented the implementation</a> of common optimizations. And it made it nearly impossible to extend the 'expression' JIT to code segments longer than a basic block.<br /></li><li style="text-align: left;">Failing to deprecate the 'legacy' JIT compiler. </li><li style="text-align: left;">Despite some attempts I never truly succeeded in sharing the owernship of the JIT. <br /></li></ul><h4 style="text-align: left;">What's kind of ugly</h4><ul style="text-align: left;"><li style="text-align: left;">The 'expression' IR uses a completely different memory model (linear array with integer indexes) from the 'spesh' subsystem of which it is supposedly a part (which uses a more straightforward object graph coupled with arena allocation). The reason to do this was that the expression IR expands each MoarVM instruction into many IR instructions, and if every IR instruction nodes had to be pointer-based the memory costs would have been significant. But it's still kind of messy.</li><li style="text-align: left;">Some features in MoarVM (like exception handling) rely on being able to identify the section of code which is currently being executed... which is a reasonable choice for an interpreter but not great for compiled code.<br /></li></ul><h4 style="text-align: left;">How did we get here?</h4><p>One one hand, as a result of my limited experience, time and resources, and on the other hand as a result of the design of MoarVM.</p><p>MoarVM was originally designed as a traditional interpreter for a high level language (much like the Perl interpreter). Meaning that it has a large number of different instructions and many instructions operate on high-level data structures like strings, arrays and maps (as opposed to pointers and machine words).<br /></p><p>This is by no means a bad or outdated design. Frequently executed routines (string manipulation, hash table lookups etc.) are implemented using an efficient language (C) and driven by a language that is optimized for usability (Raku). This design is also used in modern machine learning frameworks. More importantly, this was a reasonable design because it is a good target for the Rakudo compiler.<br /></p><p>For the JIT compiler, this means two things:</p><ul style="text-align: left;"><li>Mapping the large number of VM instructions to machine code (or IR) becomes a significant challenge.</li><li>The high level routines and data structures used by the interpreter are mostly opaque to the compiler.</li></ul><p style="text-align: left;">The machine code generated by the JIT compiler then will mostly consists of consecutive function calls to VM routines, which is not the type of code where a compiler can really improve performance much. <br /></p><p style="text-align: left;">In other words, suppose 50% of runtime is spent in interpretation overhead (instruction decoding and dispatch), and 50% is spent in VM routines, then removing interpretation overhead via JIT compilation will at best result in a twofold increase in performance. For many programs, the observed performance increase will be even less.</p><p style="text-align: left;">Mind that I'm specifically refering to the improvement due to machine code generation, and <i>not</i> to those due to type specialization, inlining etc. (the domain of 'spesh'). These latter features have resulted in much more significant performance improvements.<br /></p><h4 style="text-align: left;">Was it worth it?</h4>I think it was.<br /><p style="text-align: left;">For me personally, it was a tremendously valuable learning experience which led directly to my current career, writing SQL compilers for Google Cloud.</p><p style="text-align: left;">For the Raku community, even if we never realized the performance improvements that I might have hoped at the start, I hope that the JIT project (as it exists) has been valuable, if for no other reason than identifying the challenges of JIT compilation for MoarVM. A future effort may be able to do better based on what we learned; and I hope my blog posts are a useful resource from that perspective. <br /></p><h4 style="text-align: left;">What's next?</h4><p style="text-align: left;">Assuming that time and resources were not an issue:<br /></p><ul style="text-align: left;"><li style="text-align: left;">I'd start by adding tests to the JIT backend. This might take the shape of extracting (parts of) the JIT to a separate project, which MoarVM would link to.</li><li style="text-align: left;">I'd re-engineer the expression IR to be linear ordered instructions, at which point we should be able to construct IR for multiple basic blocks.</li><li style="text-align: left;">I'd deprecate the legacy JIT so we only have one backend left.</li></ul><p>If any of this comes to pass, you'll find my report on it right here. Thanks for reasding and until then! <br /></p><p> </p></div> <h2><a href="https://5ab5traction5.bearblog.dev/i-raku-easy-subroutine-shortcuts-to-class-constructors/">I 🫀 Raku - Easy subroutine shortcuts to class constructors</h2></a> <p><h3>Published by 5ab5traction5 on 2023-01-27T20:21:49</h3></p> <h1 id=context>Context</h1><p>I decided to write a simple (but <a href='http://irontavern.com/2012/07/02/dcc-funky-dice/'>funky</a>) dice roller over the holidays. This led to a number of fun diversions in Raku code that all deserve some highlighting.</p> <p>Today I'd like to share a pending PR I have for <code>GTK::Simple</code> that I believe highlights one of Raku's strength: compositional concision. That is, code that does a lot in very few words thanks to the composition of various concise-on-their-own details of the language.</p> <h2 id=codegtksimplecode-is-pretty-chill-already><code>GTK::Simple</code> is pretty chill already</h2><p>This module does a good job of translating the C experience of <a href='https://docs.gtk.org/gtk3/getting_started.html'>writing a GTK application</a> into an idiomatic Raku version. It is both easily usable as well as quickly extensible, should you find some corner case of the (massive) GTK that isn't covered.</p> <p>I'll be updating with some more in depth discussion of recent changes that have been merged recently.</p> <p>(Note to self: Fully implementing the afore-linked GTK application in Raku would make for an interesting exercise.)</p> <h2 id=but-what-if-it-became-emeven-chillerem>... but what if it became <em>even chiller?</em></h2><p>In <code>GTK::Simple</code>, we map GTK classes into top-level <code>GTK::Simple</code> definitions. So <a href='https://docs.gtk.org/gtk3/class.MenuItem.html'><code>MenuItem</code></a> becomes <code>GTK::Simple::MenuItem</code>.</p> <p>This leads to code such as:</p> <div class="highlight"><pre><span></span>use GTK::Simple; my $app = GTK::Simple::App.new(title => "A fresh new app"); $app.set-content( GTK::Simple::HBox.new( GTK::Simple::Label.new(text => "Looking good in that GTK theme") ) ); $app.run; </pre></div> <p>Should my PR be accepted, it will allow a short-hand syntax that makes things almost a bit <a href='http://shoesrb.com/'>Shoes-like</a> in simplicity.</p> <div class="highlight"><pre><span></span>use GTK::Simple :subs; my $app = app :title("An alternative"); $app.set-content( h-box( label(:text("Shiny slippers")) ) ); $app.run; </pre></div> <h1 id=adding-the-mappings>Adding the mappings</h1><p>The mapping rule is a simple CamelCase to kebab-case conversion of the class names.</p> <p>The entire code for adding this feature is as follows:</p> <div class="highlight"><pre><span></span># Exports above class constructors, ex. level-bar => GTK::Simple::LevelBar.new my module EXPORT::subs { for GTK::Simple::.kv -> $name, $class { my $sub-name = '&' ~ ($name ~~ / (<:Lu><:Ll>*)* /).values.map({ .Str.lc }).join("-"); OUR::{ $sub-name } := sub (|c) { $class.new(|c) }; } } </pre></div> <h3 id=my-module-exportsubs>my module EXPORT::subs {</h3><p>This line of code utilizes a Raku convention that allows for custom behavior through specific instructions from the importer of a module, provided in the form of Pair objects (<code>:subs</code> is shorthand for <code>subs => True</code>).</p> <p>When the importer specifies <code>use GTK::Simple :subs</code>, it looks for a module with the pair's key as the name inside of the imported module. This is often a generated module thanks to the <code>export</code> trait. <code>sub foo() is export :named-option {...}</code> generates a <code>my</code>-scoped module <code>Module::named-option</code> that instructs the importer to include <code>foo</code> in its own <code>my</code> scope.</p> <h3 id=for-gtksimplekv-name-class>for GTK::Simple::.kv -> $name, $class {</h3><p>This is an example of compositional concision right here. The dangling <code>::</code> is shorthand for <code>.WHO</code>, which when called on a module returns the package meta-object of the module.<sup class="footnote-ref" id="fnref-1"><a href="#fn-1">1</a></sup> Holding this meta-object, we can call <code>.kv</code> to get the keys (names) and values (package objects) of the packages within that scope.</p> <p>Because we have loaded all (and only) our relevant classes into the <code>GTK::Simple</code> package scope, the meta-object for that package handily provides a <code>.kv</code> method that delivers the class names as keys and the actual class objects as keys, zipped together.</p> <p>If there were anything else <code>our</code> scoped in <code>GTK::Simple</code>, it would be in this list too (module and class definitions are <code>our</code> scoped by default). So depending on how complex your module is, you might have to write some filters or guards to make sure you were only processing the actual class objects.</p> <p>Thanks to a meta-object system designed to be both easy to use <em>and</em> transparently hidden from view until needed, there's nothing to make things feel complicated here.<sup class="footnote-ref" id="fnref-2"><a href="#fn-2">2</a></sup></p> <h3 id=my-math-xmlnshttpwwww3org1998mathmathml-displayinlinemrowmismimiumimibmimox02212mominmimiamimimmimiemimsupmox0003dmomix02032mimsupmsupmimimix02032mimsupmtextx000a0mtextmo-stretchyfalsex00028momrowmathname-lullemem-valuesmap-strlc-join>my <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mi>s</mi><mi>u</mi><mi>b</mi><mo>−</mo><mi>n</mi><mi>a</mi><mi>m</mi><mi>e</mi><msup><mo>=</mo><mi>′</mi></msup><msup><mi>&</mi><mi>′</mi></msup><mtext> </mtext><mo stretchy="false">(</mo></mrow></math>name ~~ / (<:Lu><:Ll><em>)</em> /).values.map({ .Str.lc }).join("-");</h3><p>This line makes me happy. The crux of the code lies in the advanced Raku regex syntax. <code><:Lu></code> and <code><:Li></code> are built in character classes in Raku that represent uppercase and lowercase characters, respectively. These are Unicode aware, so no worries there.</p> <p>The rest is straight-forward: take the <code>.values</code> of the <code>Match</code> object, map them to strings while also lower-casing them and then join the resulting list of strings into a compound string in kebab-case.</p> <p>We prepend the <code>&</code> sigil to the name in order to register it as a subroutine when it is brought into the importer's scope.</p> <h3 id=our-sub-name-sub-c-classnewc>OUR::{ $sub-name } := sub (|c) { $class.new(|c) };</h3><p>Here's where the actual mapping takes place. <code>OUR::{ $sub-name }</code> is creating a (name for a) sub dynamically in the shared <code>OUR</code> scope between the importer and the imported modules. The import process is such that these dynamically defined modules become available in the scope of the importing module (it's <code>MY</code> scope).</p> <p><code>sub (|c) { $class.new(|c) }</code> says create an anonymous subroutine that passes it's arguments (which are, of course, an object themselves) exactly as-is to the constructor of <code>$class</code>.</p> <h2 id=not-necessarily-obvious>Not necessarily obvious</h2><p>Now, I do understand that there are aspects of this code that are not obvious: the dangling <code>::</code> syntax, the necessity of making a <code>my</code> scoped module with a special name of <code>EXPORT</code>, and perhaps the use of the <code>OUR::</code> package accessor.</p> <p>It's probably not code that I will necessarily write directly from memory next time I want to dynamically load a bunch of subs into an importer's scope.</p> <p>At the same time, I would argue that all of these examples very <em>idiomatic</em> to the language: little 'escape hatches' into deeper layers of the language that work as expected once you encounter them. All of this without muddying the waters of what's "in" a module by, for instance, providing methods directly on package objects that would provide similar functionality.<sup class="footnote-ref" id="fnref-3"><a href="#fn-3">3</a></sup></p> <p>Raku really is a well-thought over programming language. It's also massive, which can be intimidating. It helps to manage this massivity when pieces fit together in carefully planned ways. In other words, once I had that dangling <code>::</code>/<code>WHO</code> object to call <code>.kv</code> on, everything else I had to do just fell together.</p> <h2 id=testing-this>Testing this</h2><p>In order to ensure that everything was indeed working in a systematic way, I needed to write some tests.</p> <div class="highlight"><pre><span></span>use GTK::Simple :subs; # Other modules are pulled into GTK::Simple namespace by now that we do not want to test sub skip-test($name) { state $skip-set = set '&' X~ <app simple raw native-lib g-d-k common property-facade>; $name (elem) $skip-set } for GTK::Simple::.kv -> $name, $class { my $sub-name = '&' ~ ($name ~~ / (<:Lu><:Ll>*)* /).values.map({ .Str.lc }).join("-"); next if skip-test($sub-name); my $widget; lives-ok { $widget = ::{$sub-name}(:label("For Button(s)"), :uri("For LinkButton")) }, "There is a subroutine in scope called '$sub-name'"; ok $widget ~~ $class, "'$sub-name' returns a { $class.^name } object"; } </pre></div> <p>Here I created a <code>state</code> variable (only defined once per scope, so we don't re-do the variable assignment with each call) to hold a set of names that we want to skip. If the argument is an element of said set, skip the test.</p> <h3 id=sub-name>::{$sub-name}</h3><p>Once again <code>::</code> shows up to be a Do-What-I-Mean shortcut. This time it is for the package of our current lexical scope. So it means essentially the same thing: let me access the package as if it were a "stash", giving us semantics equivalent to hashes (eg, <code>::{$sub-name}</code>). A lexical lookup is begun until the subroutine is found in lexical scope. Since the import process implanted it there in lexical scope, it returns with the sub stored with name <code>$sub-name</code>.</p> <p>We pass two positional arguments to the constructor because they are either required for some constructors and or ignored by those that don't require them.</p> <h2 id=conclusion>Conclusion</h2><p>That wraps up a fairly lengthy post about a very short snippet of code. Please stay tuned for other posts about changes that were made such that my awesome, funky dice roller could exist in ease and comfort.</p> <h3 id=footnotes>Footnotes</h3><section class="footnotes"> <ol> <li id="fn-1"><p>When <code>.WHO</code> is called on an initialized object, it returns it's respective class object.<a href="#fnref-1" class="footnote">↩</a></p></li> <li id="fn-2"><p>Compare that to <a href='https://stackoverflow.com/a/10994416'>this Java</a> or even <a href='https://www.baeldung.com/java-find-all-classes-in-package'>that Java</a> without the <code>Reflections</code> library.<a href="#fnref-2" class="footnote">↩</a></p></li> <li id="fn-3"><p>In Ruby, the same would be done with <code>MyModule.constants.select {|c| MyModule.const_get(c).is_a? Class}</code>. It's <code>MyModule.constants</code> and <code>MyModule.const_get</code> that in my opinion shows a muddier abstraction between package objects and the meta-objects that govern and represent them.<a href="#fnref-3" class="footnote">↩</a></p></li> </ol> </section> <h2><a href="https://5ab5traction5.bearblog.dev/overdue-appreciation-for-rakudo-pkg/">Overdue Apprecation for rakudo-pkg</h2></a> <p><h3>Published by 5ab5traction5 on 2021-03-16T00:00:00</h3></p> <p>It's been over a month since I first came across -- finally -- a clean way to present anyone who runs Linux with a simple, clean, non-"virtualenv" installation of <code>raku</code>: <a href='https://github.com/nxadm/rakudo-pkg'><code>rakudo-pkg</code></a> to the rescue!</p> <h2 id=coderakudo-pkgcode-vs-a-virtual-environment-like-coderakubrewcode><code>rakudo-pkg</code> vs a virtual environment like <code>rakubrew</code></h2><p>There was always a bit of an icky feeling related to relying on <code>rakubrew</code>(and <code>rakudobrew</code> before it) for requiring inquiring minds to first ignore what their system offered them through official channels and instead install some in order to have access to anything remotely resembling an up-to-date version of the <code>raku</code> runtime (or <code>perl6</code> before it).</p> <p>Unfortunately, in the case of most official package repositories, the latest officially available versions were often ancient<sup class="footnote-ref" id="fnref-1"><a href="#fn-1">1</a></sup>. It's heartening to note, however, that this situation has improved significantly since the official debut of the "finished" language as <code>6.c</code> half a decade ago. Still the official repositories lag far behind the improvements that are made, even today.</p> <p>In my opinion, it is one thing to encounter a <code>virtualenv</code>-style tool after you have hit some limitation with running the system installation of a language. But being exposed to adding a whole new mothball to your home directory and login shell configuration as a requirement to just trying out a language is not the strongest look in terms of an advocacy perspective.</p> <p>Having a dedicated system path for the tools also fixes issues related to tools that do not inherit environment variables created by executing all of those tweaks stashed in <code>.bash_profile</code> or (in my case) <code>.config/fish/config.fish</code>.</p> <p>A virtualenv approach is also particularly un-desirable as it is potentially resolvable through guarantees made at the language design layer around Raku's approach to module and language versioning.</p> <h2 id=important-details-about-raku-versioning>Important details about Raku versioning</h2><p>Raku naturally shows it's previous life as that caterpillar <em>formerly-known-as-Perl-6</em> most strongly when you encounter its own versioning.</p> <p><code>use v6.c</code> is guaranteed to access a historical standard of Raku behavior. <code>use v6.*</code> optimistically says "use *Whatever* version you consider the newest". <code>use v6.d</code> gives you guarantees that the language won't start spitting deprecation warnings pertaining to later versions, starting with <code>v6.e</code>, while also doing everything exactly as <code>v6.d</code> intended even on a newer release.</p> <h3 id=free-project-ideas-below>Free project ideas below</h3><p>It would be interesting to stress test the implicit and explicit language level guarantees of Raku by dog-fooding an old fashioned "smoke test" on our own with regard to the claims made in the designs of the language versioning and the module repositories and authorities concepts. A sort of "distributed DarkPAN simulator" for Raku in the 2020s.</p> <p>The CompUnit repositories and module authorities are ideas that intend to make backward compatibility easier in a world where sometimes you want to run a locally patched variant of a public module that is otherwise identical (or even wildly incompatible) and other times you want to be able to run two different versions of a library <em>side-by-side</em> -- at the same time.</p> <p>A/A testing of library upgrades at vanguard for a bit before rolling out to the fleet, anyone? (That's a different, likely far more profitable, library idea for you, my intrepid reader).</p> <h2 id=ok-but-where>Ok, but where???</h2><p>Check out the <a href='https://nxadm.wordpress.com/2021/02/11/rakudo-pkg-v2/'>blog post announcement</a> of the new GitHub Task based release flow and the latest iteration of the <a href='https://github.com/nxadm/rakudo-pkg'><code>rakudo-pkg</code></a> project.</p> <section class="footnotes"> <ol> <li id="fn-1"><p>It was a long road to the first official release, so it is not at all fair to blame distribution maintenance teams to not bother with ensuring that the bleeding edge version of a still-baking language was--or is--easily accessible. Things have gotten better since the release of <code>6.c</code>.<a href="#fnref-1" class="footnote">↩</a></p></li> </ol> </section> <h2><a href="https://5ab5traction5.bearblog.dev/fixing-raku-unicode-display-issues-windows/">Fixing Raku Unicode display issues in Windows</h2></a> <p><h3>Published by 5ab5traction5 on 2020-12-26T19:07:50</h3></p> <p>I've been using Windows 10 for a while as I wait to install a new m2 SSD in this laptop to provide a dedicated place for Linux. I've noticed some very strange and disappointing issues with Unicode characters when running Raku from a terminal.</p> <p>Thanks to <a href='https://webchat.freenode.net/?channels=#raku'>#raku on Freenode</a>, I managed to find a solution:</p> <div class="highlight"><pre><span></span>chcp 65001 </pre></div> <p>This changes the Unicode code page to 65001 and magically fixes the issues I was seeing.</p> <h2 id=update>UPDATE</h2><p>To make the change more permanent, it is possible to use change some registry key values under <code>HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CodePage</code>. Modify <code>ACP</code>, <code>MACCP</code>, and <code>OEMCP</code> all to value <code>65001</code>, give the OS a reboot, <em>et voila!</em></p> <p>Thanks to the ever-present raiph for his <a href='https://www.reddit.com/r/rakulang/comments/kko35z/short_blog_post_fixing_raku_unicode_display/gh3yp1c/?utm_source=reddit&utm_medium=web2x&context=3'>reddit comment</a> which pointed me to a Stack Overflow question from a user facing <a href='https://stackoverflow.com/questions/55179824/perl6-rakudo-how-to-handle-special-characters-from-file'>the same problem</a>, which in turn pointed to <a href='https://stackoverflow.com/questions/56419639/what-does-beta-use-unicode-utf-8-for-worldwide-language-support-actually-do'>the solution</a> provided for a question from a C# programmer.</p> <h2><a href="https://6guts.wordpress.com/2021/09/29/the-new-moarvm-dispatch-mechanism-is-here/">The new MoarVM dispatch mechanism is here!</h2></a> <p><h3>Published by jnthnwrthngtn on 2021-09-29T16:16:31</h3></p> <p>Around 18 months ago, I set about working on the largest set of architectural changes that Raku runtime MoarVM has seen since its inception. The work was most directly triggered by the realization that we had no good way to fix a certain semantic bug in dispatch without either causing huge performance impacts across the board <em>or</em> increasingly complexity even further in optimizations that were already riding their luck. However, the need for something like this had been apparent for a while: a persistent struggle to optimize certain Raku language features, the pain of a bunch of performance mechanisms that were all solving the same kind of problem but each for a specific situation, and a sense that, with everything learned since I founded MoarVM, it was possible to do better.</p> <p>The result is the development of a new generalized dispatch mechanism. An overview can be found in my Raku Conference talk about it (<a href="https://jnthn.net/papers/2021-trc-dispatch.pdf">slides</a>, <a href="https://www.youtube.com/watch?v=yRFyGDVHl0E">video</a>); in short, it gives us a far more uniform architecture for all kinds of dispatch, allowing us to deliver better performance on a range of language features that have thus far been glacial, as well as opening up opportunities for new optimizations.</p> <p>Today, this work has been merged, along with the matching changes in NQP (the Raku subset we use for bootstrapping and to implement the compiler) and Rakudo (the full Raku compiler and standard library implementation). This means that it will ship in the October 2021 releases.</p> <p>In this post, I’ll give an overview of what you can expect to observe right away, and what you might expect in the future as we continue to build upon the possibilities that the new dispatch architecture has to offer.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/a894b2e9496ef54d2ad1fbbe60b95010#the-big-wins"></a>The big wins</h3> <p>The biggest improvements involve language features that we’d really not had the architecture to do better on before. They involved dispatch – that is, getting a call linked to a destination efficiently – but the runtime didn’t provide us with a way to “explain” to it that it was looking at a dispatch, let alone with the information needed to have a shot at optimizing it.</p> <p>The following graph captures a number of these cases, and shows the level of improvement, ranging from a factor of 3.3 to 13.3 times faster.</p> <figure class="wp-block-image size-large"><img width="601" height="318" data-attachment-id="598" data-permalink="https://6guts.wordpress.com/graph-1/" data-orig-file="https://6guts.wordpress.com/wp-content/uploads/2021/09/graph-1.png" data-orig-size="601,318" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="graph-1" data-image-description="" data-image-caption="" data-medium-file="https://6guts.wordpress.com/wp-content/uploads/2021/09/graph-1.png?w=300" data-large-file="https://6guts.wordpress.com/wp-content/uploads/2021/09/graph-1.png?w=601" src="https://6guts.wordpress.com/wp-content/uploads/2021/09/graph-1.png?w=601" alt="Graph showing benchmark results, described textually below" class="wp-image-598" srcset="https://6guts.wordpress.com/wp-content/uploads/2021/09/graph-1.png 601w, https://6guts.wordpress.com/wp-content/uploads/2021/09/graph-1.png?w=150 150w, https://6guts.wordpress.com/wp-content/uploads/2021/09/graph-1.png?w=300 300w" sizes="(max-width: 601px) 100vw, 601px" /></figure> <p>Let’s take a quick look at each of these. The first, <code>new-buf</code>, asks how quickly we can allocate <code>Buf</code>s.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> for ^10_000_000 { Buf.new } </pre></div> <p>Why is this a dispatch benchmark? Because <code>Buf</code> is not a class, but rather a role. When we try to make an instance of a role, it is “punned” into a class. Up until now, it works as follows:</p> <ol class="wp-block-list"><li>We look up the <code>new</code> method</li><li>The <code>find_method</code> method would, if needed, create a pun of the role and cache it</li><li>It would return a forwarding closure that takes the arguments and gives them to the same method called on the punned class, or spelt in Raku code, <code>-> $role-discarded, |args { $pun."$name"(|args) }</code></li><li>This closure would be invoked with the arguments</li></ol> <p>This had a number of undesirable consequences:</p> <ol class="wp-block-list"><li>While the pun was cached, we still had a bit of overhead to check if we’d made it already</li><li>The arguments got slurped and flattened, which costs something, and…</li><li>…the loss of callsite shape meant we couldn’t look up a type specialization of the method, and thus lost a chance to inline it too</li></ol> <p>With the new dispatch mechanism, we have a means to cache constants at a given program location <em>and</em> to replace arguments. So the first time we encounter the call, we:</p> <ol class="wp-block-list"><li>Get the role pun produced if needed</li><li>Resolve the <code>new</code> method on the class punned from the role</li><li>Produce a dispatch program that caches this resolved method and also replaces the role argument with the pun</li></ol> <p>For the next thousands of calls, we interpret this dispatch program. It’s still some cost, but the method we’re calling is already resolved, and the argument list rewriting is fairly cheap. Meanwhile, after we get into some hundreds of iterations, on a background thread, the optimizer gets to work. The argument re-ordering cost goes away completely at this point, and <code>new</code> is so small it gets inlined – at which point the buffer allocation is determined dead and so goes away too. Some remaining missed opportunities mean we still are left with a loop that’s not quite empty: it busies itself making sure it’s really OK to do nothing, rather than just doing nothing.</p> <p>Next up, multiple dispatch with <code>where</code> clauses.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> multi fac($n where $n <= 1) { 1 } multi fac($n) { $n * fac($n - 1) } for ^1_000_000 { fac(5) } </pre></div> <p>These were really slow before, since:</p> <ol class="wp-block-list"><li>We couldn’t apply the multi-dispatch caching mechanism <em>at all</em> as soon as we had a <code>where</code> clause involved</li><li>We would run <code>where</code> clauses twice in the event the candidate was chosen: once to see if we should choose that multi candidate, and once again when we entered it</li></ol> <p>With the new mechanism, we:</p> <ol class="wp-block-list"><li>On the first call, calculate a multiple dispatch plan: a linked list of candidates to work through</li><li>Invoke the one with the <code>where</code> clause, in a mode whereby if the signature fails to bind, it triggers a dispatch resumption. (If it does bind, it runs to completion)</li><li>In the event of a bind failure, the dispatch resumption triggers, and we attempt the next candidate</li></ol> <p>Once again, after the setup phase, we interpret the dispatch programs. In fact, that’s as far as we get with running this faster for now, because the specializer doesn’t yet know how to translate and further optimize this kind of dispatch program. (That’s how I know it currently stands no chance of turning this whole thing into another empty loop!) So there’s more to be had here also; in the meantime, I’m afraid you’ll just have to settle for a factor of ten speedup.</p> <p>Here’s the next one:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> proto with-proto(Int $n) { 2 * {*} } multi with-proto(Int $n) { $n + 1 } sub invoking-nontrivial-proto() { for ^10_000_000 { with-proto(20) } } </pre></div> <p>Again, on top form, we’d turn this into an empty loop too, but we don’t quite get there yet. This case wasn’t so terrible before: we did get to use the multiple dispatch cache, however to do that we also ended up having to allocate an argument capture. The need for this also blocked any chance of inlining the <code>proto</code> into the caller. Now that is possible. Since we cannot yet translate dispatch programs that resume an in-progress dispatch, we don’t yet get to further inline the called <code>multi</code> candidate into the <code>proto</code>. However, we now have a design that will let us implement that.</p> <p>This whole notion of a dispatch resumption – where we start doing a dispatch, and later need to access arguments or other pre-calculated data in order to do a next step of it – has turned out to be a great unification. The initial idea for it came from considering things like <code>callsame</code>:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> class Parent { method m() { 1 } } class Child is Parent { method m() { 1 + callsame } } for ^10_000_000 { Child.m; } </pre></div> <p>Once I started looking at this, and then considering that a complex <code>proto</code> also wants to continue with a dispatch at the <code>{*}</code>, and in the case a <code>where</code> clauses fails in a <code>multi</code> it <em>also</em> wants to continue with a dispatch, I realized this was going to be useful for quite a lot of things. It will be a bit of a headache to teach the optimizer and JIT to do nice things with resumes – but a great relief that doing that once will benefit multiple language features!</p> <p>Anyway, back to the benchmark. This is another “if we were smart, it’d be an empty loop” one. Previously, <code>callsame</code> was very costly, because each time we invoked it, it would have to calculate what kind of dispatch we were resuming and the set of methods to call. We also had to be able to locate the arguments. Dynamic variables were involved, which cost a bit to look up too, and – despite being an implementation details – these also leaked out in introspection, which wasn’t ideal. The new dispatch mechanism makes this all rather more efficient: we can cache the calculated set of methods (or wrappers and multi candidates, depending on the context) and then walk through it, and there’s no dynamic variables involved (and thus no leakage of them). This sees the biggest speedup of the lot – and since we cannot yet inline away the <code>callsame</code>, it’s (for now) measuring the speedup one might expect on using this language feature. In the future, it’s destined to optimize away to an empty loop.</p> <p>A module that makes use of <code>callsame</code> on a relatively hot path is <code>OO::Monitors,</code>, so I figured it would be interesting to see if there is a speedup there also.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> use OO::Monitors; monitor TestMonitor { method m() { 1 } } my $mon = TestMonitor.new; for ^1_000_000 { $mon.m(); } </pre></div> <p>A <code>monitor</code> is a class that acquires a lock around each method call. The module provides a custom meta-class that adds a lock attribute to the class and then wraps each method such that it acquires the lock. There are certainly costly things in there besides the involvement of <code>callsame</code>, but the improvement to <code>callsame</code> is already enough to see a 3.3x speedup in this benchmark. Since <code>OO::Monitors</code> is used in quite a few applications and modules (for example, Cro uses it), this is welcome (and yes, a larger improvement will be possible here too).</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/a894b2e9496ef54d2ad1fbbe60b95010#caller-side-decontainerization"></a>Caller side decontainerization</h3> <p>I’ve seen some less impressive, but still welcome, improvements across a good number of other microbenchmarks. Even a basic multi dispatch on the <code>+</code> op:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $i = 0; for ^10_000_000 { $i = $i + $_; } </pre></div> <p>Comes out with a factor of 1.6x speedup, thanks primarily to us producing far tighter code with fewer guards. Previously, we ended up with duplicate guards in this seemingly straightforward case. The <code>infix:<+></code> multi candidate would be specialized for the case of its first argument being an <code>Int</code> in a <code>Scalar</code> container and its second argument being an immutable <code>Int</code>. Since a <code>Scalar</code> is mutable, the specialization would need to read it and then guard the value read before proceeding, otherwise it may change, and we’d risk memory safety. When we wanted to inline this candidate, we’d also want to do a check that the candidate really applies, and so also would deference the <code>Scalar</code> and guard its content to do that. We can and do eliminate duplicate guards – but these guards are on two distinct reads of the value, so that wouldn’t help.</p> <p>Since in the new dispatch mechanism we can rewrite arguments, we can now quite easily do caller-side removal of <code>Scalar</code> containers around values. So easily, in fact, that the change to do it took me just a couple of hours. This gives a lot of benefits. Since dispatch programs automatically eliminate duplicate reads and guards, the read and guard by the multi-dispatcher and the read in order to pass the decontainerized value are coalesced. This means less repeated work prior to specialization and JIT compilation, and also only a single read and guard in the specialized code after it. With the value to be passed already guarded, we can trivially select a candidate taking two bare <code>Int</code> values, which means there’s no further reads and guards needed in the callee either.</p> <p>A less obvious benefit, but one that will become important with planned future work, is that this means <code>Scalar</code> containers escape to callees far less often. This creates further opportunities for escape analysis. While the MoarVM escape analyzer and scalar replacer is currently quite limited, I hope to return to working on it in the near future, and expect it will be able to give us even more value now than it would have been able to before.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/a894b2e9496ef54d2ad1fbbe60b95010#further-results"></a>Further results</h3> <p>The benchmarks shown earlier are mostly of the “how close are we to realizing that we’ve got an empty loop” nature, which is interesting for assessing how well the optimizer can “see through” dispatches. Here are a few further results on more “traditional” microbenchmarks:</p> <figure class="wp-block-image size-large"><img width="600" height="270" data-attachment-id="599" data-permalink="https://6guts.wordpress.com/graph2/" data-orig-file="https://6guts.wordpress.com/wp-content/uploads/2021/09/graph2.png" data-orig-size="600,270" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="graph2" data-image-description="" data-image-caption="" data-medium-file="https://6guts.wordpress.com/wp-content/uploads/2021/09/graph2.png?w=300" data-large-file="https://6guts.wordpress.com/wp-content/uploads/2021/09/graph2.png?w=600" src="https://6guts.wordpress.com/wp-content/uploads/2021/09/graph2.png?w=600" alt="Graph showing benchmark results, described textually below" class="wp-image-599" srcset="https://6guts.wordpress.com/wp-content/uploads/2021/09/graph2.png 600w, https://6guts.wordpress.com/wp-content/uploads/2021/09/graph2.png?w=150 150w, https://6guts.wordpress.com/wp-content/uploads/2021/09/graph2.png?w=300 300w" sizes="(max-width: 600px) 100vw, 600px" /></figure> <p>The complex number benchmark is as follows:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $total-re = 0e0; for ^2_000_000 { my $x = 5 + 2i; my $y = 10 + 3i; my $z = $x * $x + $y; $total-re = $total-re + $z.re } say $total-re; </pre></div> <p>That is, just a bunch of operators (multi dispatch) and method calls, where we really do use the result. For now, we’re tied with Python and a little behind Ruby on this benchmark (and a surprising 48 times faster than the same thing done with Perl’s <code>Math::Complex</code>), but this is also a case that stands to see a huge benefit from escape analysis and scalar replacement in the future.</p> <p>The hash read benchmark is:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my %h = a => 10, b => 12; my $total = 0; for ^10_000_000 { $total = $total + %h<a> + %h<b>; } </pre></div> <p>And the hash store one is:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my @keys = 'a'..'z'; for ^500_000 { my %h; for @keys { %h{$_} = 42; } } </pre></div> <p>The improvements are nothing whatsoever to do with hashing itself, but instead look to be mostly thanks to much tighter code all around due to caller-side decontainerization. That can have a secondary effect of bringing things under the size limit for inlining, which is also a big help. Speedup factors of 2x and 1.85x are welcome, although we could really do with the same level of improvement again for me to be reasonably happy with our results.</p> <p>The line-reading benchmark is:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $fh = open "longfile"; my $chars = 0; for $fh.lines { $chars = $chars + .chars }; $fh.close; say $chars </pre></div> <p>Again, nothing specific to I/O got faster, but when dispatch – the glue that puts together all the pieces – gets a boost, it helps all over the place. (We are also decently competitive on this benchmark, although tend to be slower the moment the UTF-8 decoder can’t take it’s “NFG can’t possibly apply” fast path.)</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/a894b2e9496ef54d2ad1fbbe60b95010#and-in-less-micro-things"></a>And in less micro things…</h3> <p>I’ve also started looking at larger programs, and hearing results from others about theirs. It’s mostly encouraging:</p> <ul class="wp-block-list"><li>The long-standing <code>Text::CSV</code> benchmark <code>test-t</code> has seen roughly 20% improvement (thanks to lizmat for measuring)</li><li>A simple <code>Cro::HTTP</code> test application gets through about 10% more requests per second</li><li>MoarVM contributor dogbert did comparative timings of a number of scripts; the most significant improvement saw a drop from 25s to 7s, most are 10%-30% faster, some without change, and only one that slowed down.</li><li>There’s around 2.5% improvement on compilation of <code>CORE.setting</code>, the standard library. However, a big pinch of salt is needed here: the compiler itself has changed in a number of places as part of the work, and there were a couple of things tweaked based on looking at profiles that aren’t really related to dispatch.</li><li>Agrammon, an application calculating farming emissions, has seen a slowdown of around 9%. I didn’t get to look at it closely yet, although glancing at profiling output the number of deoptimizations is relatively high, which suggests we’re making some poor optimization decisions somewhere.</li></ul> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/a894b2e9496ef54d2ad1fbbe60b95010#smaller-profiler-output"></a>Smaller profiler output</h3> <p>One unpredicted (by me), but also welcome, improvement is that profiler output has become significantly smaller. Likely reasons for this include:</p> <ol class="wp-block-list"><li>The dispatch mechanism supports producing value results (either from constants, input arguments, or attributes read from input arguments). It entirely replaces an earlier mechanism, “specializer plugins”, which could map guards to a target to invoke, but always required a call to something – even if that something was the identity function. The logic was that this didn’t matter for any really hot code, since the identity function will trivially be inlined away. However, since profile size of the instrumenting profiler is a function of the number of paths through the call tree, trimming loads of calls to the identity function out of the tree makes it much smaller.</li><li>We used to make lots of calls to the <code>sink</code> method when a value was in sink context. Now, if we see that the type simply inherits that method from <code>Mu</code>, we elide the call entirely (again, it would inline away, but a smaller call graph is a smaller profile).</li><li>Multiple dispatch caching would previously always call the <code>proto</code> when the cache was missed, but would then not call an onlystar <code>proto</code> again when it got cache hits in the future. This meant the call tree under many multiple dispatches was duplicated in the profile. This wasn’t just a size issue; it was a bit annoying to have this effect show up in the profile reports too.</li></ol> <p>To give an example of the difference, I took profiles from Agrammon to study why it might have become slower. The one from before the dispatcher work weighed in at 87MB; the one with the new dispatch mechanism is under 30MB. That means less memory used while profiling, less time to write the profile out to disk afterwards, and less time for tools to load the profiler output. So now it’s faster to work out how to make things faster.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/a894b2e9496ef54d2ad1fbbe60b95010#is-there-any-bad-news"></a>Is there any bad news?</h3> <p>I’m afraid so. Startup time has suffered. While the new dispatch mechanism is more powerful, pushes more complexity out of the VM into high level code, and is more conducive to reaching higher peak performance, it also has a higher warmup time. At the time of writing, the impact on startup time seems to be around 25%. I expect we can claw some of that back ahead of the October release.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/a894b2e9496ef54d2ad1fbbe60b95010#what-will-be-broken"></a>What will be broken?</h3> <p>Changes of this scale always come with an amount of risk. We’re merging this some weeks ahead of the next scheduled monthly release in order to have time for more testing, and to address any regressions that get reported. However, even before reaching the point of merging it, we have:</p> <ul class="wp-block-list"><li>Ensured it passes the specification test suite, both in normal circumstances, but also under optimizer stressing (where we force it to prematurely optimize everything, so that we tease out optimizer bugs and – given how many poor decisions we force it to make – deoptimization bugs too)</li><li>Used <code>blin</code> to run the tests of ecosystem modules. This is a standard step when preparing Rakudo releases, but in this case we’ve aimed it at the <code>new-disp</code> branches. This found a number of regressions caused by the switch to the new dispatch mechanism, which have been addressed.</li><li>Patched or sent pull requests to a number of modules that were relying on unsupported internal APIs that have now gone away or changed, or on other implementation details. There were relatively few of these, and happily, many of them were fixed up by migrating to supported APIs (which likely didn’t exist at the time the modules were written).</li></ul> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/a894b2e9496ef54d2ad1fbbe60b95010#what-happens-next"></a>What happens next?</h3> <p>As I’ve alluded to in a number of places in this post, while there are improvements to be enjoyed right away, there are also new opportunities for further improvement. Some things that are on my mind include:</p> <ul class="wp-block-list"><li>Reworking callframe entry and exit. These are still decidedly too costly. Various changes that have taken place while working on the new dispatch mechanism have opened up new opportunities for improvement in this area.</li><li>Avoiding megamorphic pile-ups. Micro-benchmarks are great at hiding these. In fact, the <code>callsame</code> one here is a perfect example! The point we do the resumption of a dispatch is inside <code>callsame</code>, so all the inline cache entries of resumptions throughout the program stack up in one place. What we’d like is to have them attached a level down the callstack instead. Otherwise, the level of <code>callsame</code> improvement seen in micro-benchmarks will not be enjoyed in larger applications. This applies in a number of other situations too.</li><li>Applying the new dispatch mechanism to optimize further constructs. For example, a method call that results in invoking the special <code>FALLBACK</code> method could have its callsite easily rewritten to do that, opening the way to inlining.</li><li>Further tuning the code we produce after optimization. There is an amount of waste that should be relatively straightforward to eliminate, and some opportunities to tweak deoptimization such that we’re able to delete more instructions and still retain the ability to deoptimize.</li><li>Continuing with the escape analysis work I was doing before, which should now be rather more valuable. The more flexible callstack/frame handling in place should also unblock my work on scalar replacement of <code>Int</code>s (which needs a great deal of care in memory management, as they may box a big integer, not just a native integer).</li><li>Implementing specialization, JIT, and inlining of dispatch resumptions.</li></ul> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/a894b2e9496ef54d2ad1fbbe60b95010#thank-you"></a>Thank you</h3> <p>I would like to thank <a href="https://www.perlfoundation.org/">TPF</a> and their donors for providing the funding that has made it possible for me to spend a good amount of my working time on this effort.</p> <p>While I’m to blame for the overall design and much of the implementation of the new dispatch mechanism, plenty of work has also been put in by other MoarVM and Rakudo contributors – especially over the last few months as the final pieces fell into place, and we turned our attention to getting it production ready. I’m thankful to them not only for the code and debugging contributions, but also much support and encouragement along the way. It feels good to have this merged, and I look forward to building upon it in the months and years to come.</p> <h2><a href="https://6guts.wordpress.com/2021/04/15/raku-multiple-dispatch-with-the-new-moarvm-dispatcher/">Raku multiple dispatch with the new MoarVM dispatcher</h2></a> <p><h3>Published by jnthnwrthngtn on 2021-04-15T09:54:30</h3></p> <p>I recently wrote about the <a href="https://6guts.wordpress.com/2021/03/15/towards-a-new-general-dispatch-mechanism-in-moarvm/">new MoarVM dispatch mechanism</a>, and in that post noted that I still had a good bit of Raku’s multiple dispatch semantics left to implement in terms of it. Since then, I’ve made a decent amount of progress in that direction. This post contains an overview of the approach taken, and some very rough performance measurements.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/20be85843ca5d903e50e3b87c1cefde5#my-goodness-thats-a-lot-of-semantics"></a>My goodness, that’s a lot of semantics</h3> <p>Of all the kinds of dispatch we find in Raku, multiple dispatch is the most complex. Multiple dispatch allows us to write a set of candidates, which are then selected by the number of arguments:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> multi ok($condition, $desc) { say ($condition ?? 'ok' !! 'not ok') ~ " - $desc"; } multi ok($condition) { ok($condition, ''); } </pre></div> <p>Or the types of arguments:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> multi to-json(Int $i) { ~$i } multi to-json(Bool $b) { $b ?? 'true' !! 'false' } </pre></div> <p>And not just one argument, but potentially many:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> multi truncate(Str $str, Int $chars) { $str.chars < $chars ?? $str !! $str.substr(0, $chars) ~ '...' } multi truncate(Str $str, Str $after) { with $str.index($after) -> $pos { $str.substr(0, $pos) ~ '...' } else { $str } } </pre></div> <p>We may write <code>where</code> clauses to differentiate candidates on properties that are not captured by nominal types:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> multi fac($n where $n <= 1) { 1 } multi fac($n) { $n * fac($n - 1) } </pre></div> <p>Every time we write a set of <code>multi</code> candidates like this, the compiler will automatically produce a <code>proto</code> routine. This is what is installed in the symbol table, and holds the candidate list. However, we can also write our own <code>proto</code>, and use the special term <code>{*}</code> to decide at which point we do the dispatch, if at all.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> proto mean($collection) { $collection.elems == 0 ?? Nil !! {*} } multi mean(@arr) { @arr.sum / @arr.elems } multi mean(%hash) { %hash.values.sum / %hash.elems } </pre></div> <p>Candidates are ranked by narrowness (using topological sorting). If multiple candidates match, but they are equally narrow, then that’s an ambiguity error. Otherwise, we call narrowest one. The candidate we choose may then use <code>callsame</code> and friends to defer to the next narrowest candidate, which may do the same, until we reach the most general matching one.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/20be85843ca5d903e50e3b87c1cefde5#multiple-dispatch-is-everywhere"></a>Multiple dispatch is everywhere</h3> <p>Raku leans heavily on multiple dispatch. Most operators in Raku are compiled into calls to multiple dispatch subroutines. Even <code>$a + $b</code> will be a multiple dispatch. This means doing multiple dispatch efficiently is really important for performance. Given the riches of its semantics, this is potentially a bit concerning. However, there’s good news too.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/20be85843ca5d903e50e3b87c1cefde5#most-multiple-dispatches-are-boring"></a>Most multiple dispatches are boring</h3> <p>The overwhelmingly common case is that we have:</p> <ul class="wp-block-list"><li>A decision made only by the number of arguments and nominal types</li><li>No <code>where</code> clauses</li><li>No custom <code>proto</code></li><li>No <code>callsame</code></li></ul> <p>This isn’t to say the other cases are unimportant; they are really quite useful, and it’s desirable for them to perform well. However, it’s also desirable to make what savings we can in the common case. For example, we don’t want to eagerly calculate the full set of possible candidates for every single multiple dispatch, because the majority of the time only the first one matters. This is not just a time concern: recall that the new dispatch mechanism stores dispatch programs at each callsite, and if we store the list of all matching candidates at each of those, we’ll waste a lot of memory too.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/20be85843ca5d903e50e3b87c1cefde5#how-do-we-do-today"></a>How do we do today?</h3> <p>The situation in Rakudo today is as follows:</p> <ul class="wp-block-list"><li>If the dispatch is decided by arity and nominal type only, and you don’t call it with flattening args, it’ll probably perform quite decently, and perhaps even enjoy inlining of the candidate and elimination of duplicate type checks that would take place on the slow path. This is thanks to the <code>proto</code> holding a “dispatch cache”, a special-case mechanism implemented in the VM that uses a search tree, with one level per argument.</li><li>If that’s the case but it has a custom <code>proto</code>, it’s not too bad either, though inlining isn’t going to be happening; it can still use the search tree, though</li><li>If it uses <code>where</code> clauses, it’ll be slow, because the search tree only deals in finding one candidate per set of nominal types, and so we can’t use it</li><li>The same reasoning applies to <code>callsame</code>; it’ll be slow too</li></ul> <p>Effectively, the situation today is that you simply <em>don’t</em> use <code>where</code> clauses in a multiple dispatch if its anywhere near a hot path (well, and if you know where the hot paths are, and know that this kind of dispatch is slow). Ditto for <code>callsame</code>, although that’s less commonly reached for. The question is, can we do better with the new dispatcher?</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/20be85843ca5d903e50e3b87c1cefde5#guard-the-types"></a>Guard the types</h3> <p>Let’s start out with seeing how the simplest cases are dealt with, and build from there. (This is actually what I did in terms of the implementation, but at the same time I had a rough idea where I was hoping to end up.)</p> <p>Recall this pair of candidates:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> multi truncate(Str $str, Int $chars) { $str.chars < $chars ?? $str !! $str.substr(0, $chars) ~ '...' } multi truncate(Str $str, Str $after) { with $str.index($after) -> $pos { $str.substr(0, $pos) ~ '...' } else { $str } } </pre></div> <p>We then have a call <code>truncate($message, "\n")</code>, where <code>$message</code> is a <code>Str</code>. Under the new dispatch mechanism, the call is made using the <code>raku-call</code> dispatcher, which identifies that this is a multiple dispatch, and thus delegates to <code>raku-multi</code>. (Multi-method dispatch ends up there too.)</p> <p>The record phase of the dispatch – on the first time we reach this callsite – will proceed as follows:</p> <ol class="wp-block-list"><li>Iterate over the candidates</li><li>If a candidate doesn’t match on argument count, just discard it. Since the shape of a callsite is a constant, and we calculate dispatch programs at each callsite, we don’t need to establish any guards for this.</li><li>If it matches on types and concreteness, note which parameters are involved and what kinds of guards they need.</li><li>If there was no match or an ambiguity, report the error without producing a dispatch program.</li><li>Otherwise, having established the type guards, delegate to the <code>raku-invoke</code> dispatcher with the chosen candidate.</li></ol> <p>When we reach the same callsite again, we can run the dispatch program, which quickly checks if the argument types match those we saw last time, and if they do, we know which candidate to invoke. These checks are very cheap – far cheaper than walking through all of the candidates and examining each of them for a match. The optimizer may later be able to prove that the checks will always come out true and eliminate them.</p> <p>Thus the whole of the dispatch processes – at least for this simple case where we only have types and arity – can be “explained” to the virtual machine as “if the arguments have these exact types, invoke this routine”. It’s pretty much the same as we were doing for method dispatch, except there we only cared about the type of the first argument – the invocant – and the value of the method name. (Also recall from the previous post that if it’s a multi-method dispatch, then both method dispatch and multiple dispatch will guard the type of the first argument, but the duplication is eliminated, so only one check is done.)</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/20be85843ca5d903e50e3b87c1cefde5#that-goes-in-the-resumption-hole"></a>That goes in the resumption hole</h3> <p>Coming up with good abstractions is difficult, and therein lies much of the challenge of the new dispatch mechanism. Raku has quite a number of different dispatch-like things. However, encoding all of them directly in the virtual machine leads to high complexity, which makes building reliable optimizations (or even reliable unoptimized implementations!) challenging. Thus the aim is to work out a comparatively small set of primitives that allow for dispatches to be “explained” to the virtual machine in such a way that it can deliver decent performance.</p> <p>It’s fairly clear that <code>callsame</code> is a kind of dispatch resumption, but what about the custom <code>proto</code> case and the <code>where</code> clause case? It turns out that these can both be neatly expressed in terms of dispatch resumption too (the <code>where</code> clause case needing one small addition at the virtual machine level, which in time is likely to be useful for other things too). Not only that, but encoding these features in terms of dispatch resumption is also quite direct, and thus should be efficient. Every trick we teach the specializer about doing better with dispatch resumptions can benefit all of the language features that are implemented using them, too.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/20be85843ca5d903e50e3b87c1cefde5#custom-protos"></a>Custom protos</h3> <p>Recall this example:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> proto mean($collection) { $collection.elems == 0 ?? Nil !! {*} } </pre></div> <p>Here, we want to run the body of the <code>proto</code>, and then proceed to the chosen candidate at the point of the <code>{*}</code>. By contrast, when we don’t have a custom <code>proto</code>, we’d like to simply get on with calling the correct <code>multi</code>.</p> <p>To achieve this, I first moved the multi candidate selection logic from the <code>raku-multi</code> dispatcher to the <code>raku-multi-core</code> dispatcher. The <code>raku-multi</code> dispatcher then checks if we have an “onlystar” <code>proto</code> (one that does not need us to run it). If so, it delegates immediately to <code>raku-multi-core</code>. If not, it saves the arguments to the dispatch as the resumption initialization state, and then calls the <code>proto</code>. The <code>proto</code>‘s <code>{*}</code> is compiled into a dispatch resumption. The resumption then delegates to <code>raku-multi-core</code>. Or, in code:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> nqp::dispatch('boot-syscall', 'dispatcher-register', 'raku-multi', # Initial dispatch, only setting up resumption if we need to invoke the # proto. -> $capture { my $callee := nqp::captureposarg($capture, 0); my int $onlystar := nqp::getattr_i($callee, Routine, '$!onlystar'); if $onlystar { # Don't need to invoke the proto itself, so just get on with the # candidate dispatch. nqp::dispatch('boot-syscall', 'dispatcher-delegate', 'raku-multi-core', $capture); } else { # Set resume init args and run the proto. nqp::dispatch('boot-syscall', 'dispatcher-set-resume-init-args', $capture); nqp::dispatch('boot-syscall', 'dispatcher-delegate', 'raku-invoke', $capture); } }, # Resumption means that we have reached the {*} in the proto and so now # should go ahead and do the dispatch. Make sure we only do this if we # are signalled to that it's a resume for an onlystar (resumption kind 5). -> $capture { my $track_kind := nqp::dispatch('boot-syscall', 'dispatcher-track-arg', $capture, 0); nqp::dispatch('boot-syscall', 'dispatcher-guard-literal', $track_kind); my int $kind := nqp::captureposarg_i($capture, 0); if $kind == 5 { nqp::dispatch('boot-syscall', 'dispatcher-delegate', 'raku-multi-core', nqp::dispatch('boot-syscall', 'dispatcher-get-resume-init-args')); } elsif !nqp::dispatch('boot-syscall', 'dispatcher-next-resumption') { nqp::dispatch('boot-syscall', 'dispatcher-delegate', 'boot-constant', nqp::dispatch('boot-syscall', 'dispatcher-insert-arg-literal-obj', $capture, 0, Nil)); } }); </pre></div> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/20be85843ca5d903e50e3b87c1cefde5#two-become-one"></a>Two become one</h3> <p>Deferring to the next candidate (for example with <code>callsame</code>) and trying the next candidate because a <code>where</code> clause failed look very similar: both involve walking through a list of possible candidates. There’s some details, but they have a great deal in common, and it’d be nice if that could be reflected in how multiple dispatch is implemented using the new dispatcher.</p> <p>Before that, a slightly terrible detail about how things work in Rakudo today when we have <code>where</code> clauses. First, the dispatcher does a “trial bind”, where it asks the question: would this signature bind? To do this, it has to evaluate all of the <code>where</code> clauses. Worse, it has to use the slow-path signature binder too, which interprets the signature, even though we can in many cases compile it. If the candidate matches, great, we select it, and then invoke it…which runs the <code>where</code> clauses a second time, as part of the compiled signature binding code. There is nothing efficient about this at all, except for it being by far more efficient on developer time, which is why it happened that way.</p> <p>Anyway, it goes without saying that I’m rather keen to avoid this duplicate work <em>and</em> the slow-path binder where possible as I re-implement this using the new dispatcher. And, happily, a small addition provides a solution. There is an op <code>assertparamcheck</code>, which any kind of parameter checking compiles into (be it type checking, <code>where</code> clause checking, etc.) This triggers a call to a function that gets the arguments, the thing we were trying to call, and can then pick through them to produce an error message. The trick is to provide a way to invoke a routine such that a bind failure, instead of calling the error reporting function, will leave the routine and then do a dispatch resumption! This means we can turn failure to pass <code>where</code> clause checks into a dispatch resumption, which will then walk to the next candidate and try it instead.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/20be85843ca5d903e50e3b87c1cefde5#trivial-vs-non-trivial"></a>Trivial vs. non-trivial</h3> <p>This gets us most of the way to a solution, but there’s still the question of being memory and time efficient in the common case, where there is no resumption and no <code>where</code> clauses. I coined the term “trivial multiple dispatch” for this situation, which makes the other situation “non-trivial”. In fact, I even made a dispatcher called <code>raku-multi-non-trivial</code>! There are two ways we can end up there.</p> <ol class="wp-block-list"><li>The initial attempt to find a matching candidate determines that we’ll have to consider <code>where</code> clauses. As soon as we see this is the case, we go ahead and produce a full list of possible candidates that could match. This is a linked list (see my previous post for why).</li><li>The initial attempt to find a matching candidate finds one that can be picked based purely on argument count and nominal types. We stop there, instead of trying to build a full candidate list, and run the matching candidate. In the event that a <code>callsame</code> happens, we end up in the trivial dispatch resumption handler, which – since this situation is now non-trivial – builds the full candidate list, snips the first item off it (because we already ran that), and delegates to <code>raku-multi-non-trivial</code>.</li></ol> <p>Lost in this description is another significant improvement: today, when there are <code>where</code> clauses, we entirely lose the ability to use the MoarVM multiple dispatch cache, but under the new dispatcher, we store a type-filtered list of candidates at the callsite, and then cheap type guards are used to check it is valid to use.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/20be85843ca5d903e50e3b87c1cefde5#preliminary-results"></a>Preliminary results</h3> <p>I did a few benchmarks to see how the new dispatch mechanism did with a couple of situations known to be sub-optimal in Rakudo today. These numbers do not reflect what is <em>possible</em>, because at the moment the specializer does not have much of an understanding of the new dispatcher. Rather, they reflect the <em>minimal</em> improvement we can expect.</p> <p>Consider this benchmark using a <code>multi</code> with a <code>where</code> clause to recursively implement factorial.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> multi fac($n where $n <= 1) { 1 } multi fac($n) { $n * fac($n - 1) } for ^100_000 { fac(10) } say now - INIT now; </pre></div> <p>This needs some tweaks (and to be run under an environment variable) to use the new dispatcher; these are temporary, until such a time I switch Rakudo over to using the new dispatcher by default:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> use nqp; multi fac($n where $n <= 1) { 1 } multi fac($n) { $n * nqp::dispatch('raku-call', &fac, $n - 1) } for ^100_000 { nqp::dispatch('raku-call', &fac, 10); } say now - INIT now; </pre></div> <p>On my machine, the first runs in 4.86s, the second in 1.34s. Thus under the new dispatcher this runs in little over a quarter of the time it used to – a quite significant improvement already.</p> <p>A case involving <code>callsame</code> is also interesting to consider. Here it is without using the new dispatcher:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> multi fallback(Any $x) { "a$x" } multi fallback(Numeric $x) { "n" ~ callsame } multi fallback(Real $x) { "r" ~ callsame } multi fallback(Int $x) { "i" ~ callsame } for ^1_000_000 { fallback(4+2i); fallback(4.2); fallback(42); } say now - INIT now; </pre></div> <p>And with the temporary tweaks to use the new dispatcher:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> use nqp; multi fallback(Any $x) { "a$x" } multi fallback(Numeric $x) { "n" ~ new-disp-callsame } multi fallback(Real $x) { "r" ~ new-disp-callsame } multi fallback(Int $x) { "i" ~ new-disp-callsame } for ^1_000_000 { nqp::dispatch('raku-call', &fallback, 4+2i); nqp::dispatch('raku-call', &fallback, 4.2); nqp::dispatch('raku-call', &fallback, 42); } say now - INIT now; </pre></div> <p>On my machine, the first runs in 31.3s, the second in 11.5s, meaning that with the new dispatcher we manage it in a little over a third of the time that current Rakudo does.</p> <p>These are both quite encouraging, but as previously mentioned, a majority of multiple dispatches are of the trivial kind, not using these features. If I make the most common case worse on the way to making other things better, that would be bad. It’s not yet possible to make a fair comparison of this: trivial multiple dispatches already receive a lot of attention in the specializer, and it doesn’t yet optimize code using the new dispatcher well. Of note, in an example like this:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> multi m(Int) { } multi m(Str) { } for ^1_000_000 { m(1); m("x"); } say now - INIT now; </pre></div> <p>Inlining and other optimizations will turn this into an empty loop, which is hard to beat. There is one thing we can already do, though: run it with the specializer disabled. The new dispatcher version looks like this:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> use nqp; multi m(Int) { } multi m(Str) { } for ^1_000_000 { nqp::dispatch('raku-call', &m, 1); nqp::dispatch('raku-call', &m, "x"); } say now - INIT now; </pre></div> <p>The results are 0.463s and 0.332s respectively. Thus, the baseline execution time – before the specializer does its magic – is less using the new general dispatch mechanism than it is using the special-case multiple dispatch cache that we currently use. I wasn’t sure what to expect here before I did the measurement. Given we’re going from a specialized mechanism that has been profiled and tweaked to a new general mechanism that hasn’t received such attention, I was quite ready to be doing a little bit worse initially, and would have been happy with parity. Running in 70% of the time was a bigger improvement than I expected at this point.</p> <p>I expect that once the specializer understands the new dispatch mechanism better, it will be able to also turn the above into an empty loop – however, since more iterations can be done per-optimization, this should still show up as a win for the new dispatcher.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/20be85843ca5d903e50e3b87c1cefde5#final-thoughts"></a>Final thoughts</h3> <p>With one relatively small addition, the new dispatch mechanism is already handling most of the Raku multiple dispatch semantics. Furthermore, even without the specializer and JIT really being able to make a good job of it, some microbenchmarks already show a factor of 3x-4x improvement. That’s a pretty good starting point.</p> <p>There’s still a good bit to do before we ship a Rakudo release using the new dispatcher. However, multiple dispatch was the biggest remaining threat to the design: it’s rather more involved than other kinds of dispatch, and it was quite possible that an unexpected shortcoming could trigger another round of design work, or reveal that the general mechanism was going to struggle to perform compared to the more specialized one in the baseline unoptimized, case. So far, there’s no indication of either of these, and I’m cautiously optimistic that the overall design is about right.</p> <h2><a href="https://6guts.wordpress.com/2021/03/15/towards-a-new-general-dispatch-mechanism-in-moarvm/">Towards a new general dispatch mechanism in MoarVM</h2></a> <p><h3>Published by jnthnwrthngtn on 2021-03-15T02:08:42</h3></p> <p>My goodness, it appears I’m writing my first Raku internals blog post in over two years. Of course, two years ago it wasn’t even called Raku. Anyway, without further ado, let’s get on with this shared brainache.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/157fd5bd91a45cf6e62ecf0b9099bfba#what-is-dispatch"></a>What is dispatch?</h3> <p>I use “dispatch” to mean a process by which we take a set of arguments and end up with some action being taken based upon them. Some familiar examples include:</p> <ul class="wp-block-list"><li>Making a method call, such as <code>$basket.add($product, $quantity)</code>. We might traditionally call just <code>$product</code> and <code>$qauntity</code> the arguments, but for my purposes, all of <code>$basket</code>, the method name <code>'add'</code>, <code>$product</code>, and $quantity` are arguments to the dispatch: they are the things we need in order to make a decision about what we’re going to do.</li><li>Making a subroutine call, such as <code>uc($youtube-comment)</code>. Since Raku sub calls are lexically resolved, in this case the arguments to the dispatch are <code>&uc</code> (the result of looking up the subroutine) and <code>$youtube-comment</code>.</li><li>Calling a multiple dispatch subroutine or method, where the number and types of the arguments are used in order to decide which of a set of candidates is to be invoked. This process could be seen as taking place “inside” of one of the above two dispatches, given we have both multiple dispatch subroutines and methods in Raku.</li></ul> <p>At first glance, perhaps the first two seem fairly easy and the third a bit more of a handful – which is sort of true. However, Raku has a number of other features that make dispatch rather more, well, interesting. For example:</p> <ul class="wp-block-list"><li><code>wrap</code> allows us to wrap any <code>Routine</code> (sub or method); the wrapper can then choose to defer to the original routine, either with the original arguments or with new arguments</li><li>When doing multiple dispatch, we may write a <code>proto</code> routine that gets to choose when – or even if – the call to the appropriate candidate is made</li><li>We can use routines like <code>callsame</code> in order to defer to the next candidate in the dispatch. But what does that mean? If we’re in a multiple dispatch, it would mean the next most applicable candidate, if any. If we’re in a method dispatch then it means a method from a base class. (The same thing is used to implement going to the next wrapper or, eventually, to the originally wrapped routine too). And these can be combined: we can wrap a multi method, meaning we can have 3 levels of things that all potentially contribute the next thing to call!</li></ul> <p>Thanks to this, dispatch – at least in Raku – is not always something we do and produce an outcome, but rather a process that we may be asked to continue with multiple times!</p> <p>Finally, while the examples I’ve written above can all quite clearly be seen as examples of dispatch, a number of other common constructs in Raku can be expressed as a kind of dispatch too. Assignment is one example: the semantics of it depend on the target of the assignment and the value being assigned, and thus we need to pick the correct semantics. Coercion is another example, and return value type-checking yet another.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/157fd5bd91a45cf6e62ecf0b9099bfba#why-does-dispatch-matter"></a>Why does dispatch matter?</h3> <p>Dispatch is everywhere in our programs, quietly tieing together the code that wants stuff done with the code that does stuff. Its ubiquity means it plays a significant role in program performance. In the best case, we can reduce the cost to zero. In the worst case, the cost of the dispatch is high enough to exceed that of the work done as a result of the dispatch.</p> <p>To a first approximation, when the runtime “understands” the dispatch the performance tends to be at least somewhat decent, but when it doesn’t there’s a high chance of it being awful. Dispatches tend to involve an amount of work that can be cached, often with some cheap guards to verify the validity of the cached outcome. For example, in a method dispatch, naively we need to walk a linearization of the inheritance graph and ask each class we encounter along the way if it has a method of the specified name. Clearly, this is not going to be terribly fast if we do it on every method call. However, a particular method name on a particular type (identified precisely, without regard to subclassing) will resolve to the same method each time. Thus, we can cache the outcome of the lookup, and use it whenever the type of the invocant matches that used to produce the cached result.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/157fd5bd91a45cf6e62ecf0b9099bfba#specialized-vs-generalized-mechanisms-in-language-runtimes"></a>Specialized vs. generalized mechanisms in language runtimes</h3> <p>When one starts building a runtime aimed at a particular language, and has to do it on a pretty tight budget, the most obvious way to get somewhat tolerable performance is to bake various hot-path language semantics into the runtime. This is exactly how MoarVM started out. Thus, if we look at MoarVM as it stood several years ago, we find things like:</p> <ul class="wp-block-list"><li>Some support for method caching</li><li>A multi-dispatch cache highly tied to Raku’s multi-dispatch semantics, and only really able to help when the dispatch is all about nominal types (so using <code>where</code> comes at a very high cost)</li><li>A mechanism for specifying how to find the actual code handle inside of a wrapping code object (for example, a <code>Sub</code> object has a private attribute in it that holds the low-level code handle identifying the bytecode to run)</li><li>Some limited attempts to allow us to optimize correctly in the case we know that a dispatch will not be continued – which requires careful cooperation between compiler and runtime (or less diplomatically, it’s all a big hack)</li></ul> <p>These are all still there today, however are also all on the way out. What’s most telling about this list is what <em>isn’t</em> included. Things like:</p> <ul class="wp-block-list"><li>Private method calls, which would need a different cache – but the initial VM design limited us to one per type</li><li>Qualified method calls (<code>$obj.SomeType::method-name()</code>)</li><li>Ways to decently optimize dispatch resumption</li></ul> <p>A few years back I started to partially address this, with the introduction of a mechanism I called “specializer plugins”. But first, what is the specializer?</p> <p>When MoarVM started out, it was a relatively straightforward interpreter of bytecode. It only had to be fast enough to beat the Parrot VM in order to get a decent amount of usage, which I saw as important to have before going on to implement some more interesting optimizations (back then we didn’t have the kind of pre-release automated testing infrastructure we have today, and so depended much more on feedback from early adopters). Anyway, soon after being able to run pretty much as much of the Raku language as any other backend, I started on the dynamic optimizer. It gathered type statistics as the program was interpreted, identified hot code, put it into <a href="https://en.wikipedia.org/wiki/Static_single_assignment_form">SSA form</a>, used the type statistics to insert guards, used those together with static properties of the bytecode to analyze and optimize, and produced <em>specialized bytecode</em> for the function in question. This bytecode could elide type checks and various lookups, as well as using a range of internal ops that make all kinds of assumptions, which were safe because of the program properties that were proved by the optimizer. This is called specialized bytecode because it has had a lot of its genericity – which would allow it to work correctly on all types of value that we might encounter – removed, in favor of working in a particular special case that actually occurs at runtime. (Code, especially in more dynamic languages, is generally far more generic in theory than it ever turns out to be in practice.)</p> <p>This component – the specializer, known internally as “spesh” – delivered a significant further improvement in the performance of Raku programs, and with time its sophistication has grown, taking in optimizations such as <a href="https://en.wikipedia.org/wiki/Inline_expansion">inlining</a> and <a href="https://en.wikipedia.org/wiki/Escape_analysis">escape analysis with scalar replacement</a>. These aren’t easy things to build – but once a runtime has them, they create design possibilities that didn’t previously exist, and make decisions made in their absence look sub-optimal.</p> <p>Of note, those special-cased language-specific mechanisms, baked into the runtime to get some speed in the early days, instead become something of a liability and a bottleneck. They have complex semantics, which means they are either opaque to the optimizer (so it can’t reason about them, meaning optimization is inhibited) or they need special casing in the optimizer (a liability).</p> <p>So, back to specializer plugins. I reached a point where I wanted to take on the performance of things like <code>$obj.?meth</code> (the “call me maybe” dispatch), <code>$obj.SomeType::meth()</code> (dispatch qualified with a class to start looking in), and private method calls in roles (which can’t be resolved statically). At the same time, I was getting ready to implement some amount of escape analysis, but realized that it was going to be of very limited utility because assignment had also been special-cased in the VM, with a chunk of opaque C code doing the hot path stuff.</p> <p>But why did we have the C code doing that hot-path stuff? Well, because it’d be too espensive to have every assignment call a VM-level function that does a bunch of checks and logic. Why is that costly? Because of function call overhead and the costs of interpretation. This was all true once upon a time. But, some years of development later:</p> <ul class="wp-block-list"><li>Inlining was implemented, and could eliminate the overhead of doing a function call</li><li>We could compile to machine code, eliminating interpretation overhead</li><li>We were in a position where we had type information to hand in the specializer that would let us eliminate branches in the C code, but since it was just an opaque function we called, there was no way to take this opportunity</li></ul> <p>I solved the assignment problem and the dispatch problems mentioned above with the introduction of a single new mechanism: specializer plugins. They work as follows:</p> <ul class="wp-block-list"><li>The first time we reach a given callsite in the bytecode, we run the plugin. It produces a code object to invoke, along with a set of guards (conditions that have to be met in order to use that code object result)</li><li>The next time we reach it, we check if the guards are met, and if so, just use the result</li><li>If not, we run the plugin again, and stack up a guard set at the callsite</li><li>We keep statistics on how often a given guard set succeeds, and then use that in the specializer</li></ul> <p>The vast majority of cases are monomorphic, meaning that only one set of guards are produced and they always succeed thereafter. The specializer can thus compile those guards into the specialized bytecode and then assume the given target invocant is what will be invoked. (Further, duplicate guards can be eliminated, so the guards a particular plugin introduces may reduce to zero.)</p> <p>Specializer plugins felt pretty great. One new mechanism solved multiple optimization headaches.</p> <p>The new MoarVM dispatch mechanism is the answer to a fairly simple question: what if we get rid of <em>all</em> the dispatch-related special-case mechanisms in favor of something a bit like specializer plugins? The resulting mechanism would need to be a more powerful than specializer plugins. Further, I could learn from some of the shortcomings of specializer plugins. Thus, while they will go away after a relatively short lifetime, I think it’s fair to say that I would not have been in a place to design the new MoarVM dispatch mechanism without that experience.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/157fd5bd91a45cf6e62ecf0b9099bfba#the-dispatch-op-and-the-bootstrap-dispatchers"></a>The dispatch op and the bootstrap dispatchers</h3> <p>All the method caching. All the multi dispatch caching. All the specializer plugins. All the invocation protocol stuff for unwrapping the bytecode handle in a code object. It’s all going away, in favor of a single new dispatch instruction. Its name is, boringly enough, <code>dispatch</code>. It looks like this:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> dispatch_o result, 'dispatcher-name', callsite, arg0, arg1, ..., argN </pre></div> <p>Which means:</p> <ul class="wp-block-list"><li>Use the dispatcher called <code>dispatcher-name</code></li><li>Give it the argument registers specified (the callsite referenced indicates the number of arguments)</li><li>Put the object result of the dispatch into the register <code>result</code></li></ul> <p>(Aside: this implies a new calling convention, whereby we no longer copy the arguments into an argument buffer, but instead pass the base of the register set and a pointer into the bytecode where the register argument map is found, and then do a lookup <code>registers[map[argument_index]]</code> to get the value for an argument. That alone is a saving when we interpret, because we no longer need a loop around the interpreter per argument.)</p> <p>Some of the arguments might be things we’d traditionally call arguments. Some are aimed at the dispatch process itself. It doesn’t really matter – but it <em>is</em> more optimal if we arrange to put arguments that are only for the dispatch first (for example, the method name), and those for the target of the dispatch afterwards (for example, the method parameters).</p> <p>The new bootstrap mechanism provides a small number of built-in dispatchers, whose names start with “boot-“. They are:</p> <ul class="wp-block-list"><li><code>boot-value</code>– take the first argument and use it as the result (the identity function, except discarding any further arguments)</li><li><code>boot-constant</code> – take the first argument and produce it as the result, but also treat it as a constant value that will always be produced (thus meaning the optimizer could consider any pure code used to calculate the value as dead)</li><li><code>boot-code</code> – take the first argument, which must be a VM bytecode handle, and run that bytecode, passing the rest of the arguments as its parameters; evaluate to the return value of the bytecode</li><li><code>boot-syscall</code> – treat the first argument as the name of a VM-provided built-in operation, and call it, providing the remaining arguments as its parameters</li><li><code>boot-resume</code> – resume the topmost ongoing dispatch</li></ul> <p>That’s pretty much it. Every dispatcher we build, to teach the runtime about some other kind of dispatch behavior, eventually terminates in one of these.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/157fd5bd91a45cf6e62ecf0b9099bfba#building-on-the-bootstrap"></a>Building on the bootstrap</h3> <p>Teaching MoarVM about different kinds of dispatch is done using nothing less than the dispatch mechanism itself! For the most part, <code>boot-syscall</code> is used in order to register a dispatcher, set up the guards, and provide the result that goes with them.</p> <p>Here is a minimal example, taken from the dispatcher test suite, showing how a dispatcher that provides the identity function would look:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> nqp::dispatch('boot-syscall', 'dispatcher-register', 'identity', -> $capture { nqp::dispatch('boot-syscall', 'dispatcher-delegate', 'boot-value', $capture); }); sub identity($x) { nqp::dispatch('identity', $x) } ok(identity(42) == 42, 'Can define identity dispatch (1)'); ok(identity('foo') eq 'foo', 'Can define identity dispatch (2)'); </pre></div> <p>In the first statement, we call the <code>dispatcher-register</code> MoarVM system call, passing a name for the dispatcher along with a closure, which will be called each time we need to handle the dispatch (which I tend to refer to as the “dispatch callback”). It receives a single argument, which is a capture of arguments (not actually a Raku-level <code>Capture</code>, but the idea – an object containing a set of call arguments – is the same).</p> <p>Every user-defined dispatcher should eventually use <code>dispatcher-delegate</code> in order to identify another dispatcher to pass control along to. In this case, it delegates immediately to <code>boot-value</code> – meaning it really is nothing except a wrapper around the <code>boot-value</code> built-in dispatcher.</p> <p>The sub <code>identity</code> contains a single static occurrence of the <code>dispatch</code> op. Given we call the sub twice, we will encounter this op twice at runtime, but the two times are very different.</p> <p>The first time is the “record” phase. The arguments are formed into a capture and the callback runs, which in turn passes it along to the <code>boot-value</code> dispatcher, which produces the result. This results in an extremely simple dispatch program, which says that the result should be the first argument in the capture. Since there’s no guards, this will always be a valid result.</p> <p>The second time we encounter the <code>dispatch</code> op, it already has a dispatch program recorded there, so we are in run mode. Turning on a debugging mode in the MoarVM source, we can see the dispatch program that results looks like this:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> Dispatch program (1 temporaries) Ops: Load argument 0 into temporary 0 Set result object value from temporary 0 </pre></div> <p>That is, it reads argument 0 into a temporary location and then sets that as the result of the dispatch. Notice how there is no mention of the fact that we went through an extra layer of dispatch; those have zero cost in the resulting dispatch program.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/157fd5bd91a45cf6e62ecf0b9099bfba#capture-manipulation"></a>Capture manipulation</h3> <p>Argument captures are immutable. Various VM syscalls exist to transform them into new argument captures with some tweak, for example dropping or inserting arguments. Here’s a further example from the test suite:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> nqp::dispatch('boot-syscall', 'dispatcher-register', 'drop-first', -> $capture { my $capture-derived := nqp::dispatch('boot-syscall', 'dispatcher-drop-arg', $capture, 0); nqp::dispatch('boot-syscall', 'dispatcher-delegate', 'boot-value', $capture-derived); }); ok(nqp::dispatch('drop-first', 'first', 'second') eq 'second', 'dispatcher-drop-arg works'); </pre></div> <p>This drops the first argument before passing the capture on to the <code>boot-value</code> dispatcher – meaning that it will return the second argument. Glance back at the previous dispatch program for the identity function. Can you guess how this one will look?</p> <p>Well, here it is:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> Dispatch program (1 temporaries) Ops: Load argument 1 into temporary 0 Set result string value from temporary 0 </pre></div> <p>Again, while in the record phase of such a dispatcher we really do create capture objects and make a dispatcher delegation, the resulting dispatch program is far simpler.</p> <p>Here’s a slightly more involved example:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> my $target := -> $x { $x + 1 } nqp::dispatch('boot-syscall', 'dispatcher-register', 'call-on-target', -> $capture { my $capture-derived := nqp::dispatch('boot-syscall', 'dispatcher-insert-arg-literal-obj', $capture, 0, $target); nqp::dispatch('boot-syscall', 'dispatcher-delegate', 'boot-code-constant', $capture-derived); }); sub cot() { nqp::dispatch('call-on-target', 49) } ok(cot() == 50, 'dispatcher-insert-arg-literal-obj works at start of capture'); ok(cot() == 50, 'dispatcher-insert-arg-literal-obj works at start of capture after link too'); </pre></div> <p>Here, we have a closure stored in a variable <code>$target</code>. We insert it as the first argument of the capture, and then delegate to <code>boot-code-constant</code>, which will invoke that code object and pass the other dispatch arguments to it. Once again, at the record phase, we really do something like:</p> <ul class="wp-block-list"><li>Create a new capture with a code object inserted at the start</li><li>Delegate to the boot code constant dispatcher, which…</li><li>…creates a new capture without the original argument and runs bytecode with those arguments</li></ul> <p>And the resulting dispatch program? It’s this:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> Dispatch program (1 temporaries) Ops: Load collectable constant at index 0 into temporary 0 Skip first 0 args of incoming capture; callsite from 0 Invoke MVMCode in temporary 0 </pre></div> <p>That is, load the constant bytecode handle that we’re going to invoke, set up the args (which are in this case equal to those of the incoming capture), and then invoke the bytecode with those arguments. The argument shuffling is, once again, gone. In general, whenever the arguments we do an eventual bytecode invocation with are a tail of the initial dispatch arguments, the arguments transform becomes no more than a pointer addition.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/157fd5bd91a45cf6e62ecf0b9099bfba#guards"></a>Guards</h3> <p>All of the dispatch programs seen so far have been unconditional: once recorded at a given callsite, they shall always be used. The big missing piece to make such a mechanism have practical utility is guards. Guards assert properties such as the type of an argument or if the argument is definite (<code>Int:D</code>) or not (<code>Int:U</code>).</p> <p>Here’s a somewhat longer test case, with some explanations placed throughout it.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # A couple of classes for test purposes my class C1 { } my class C2 { } # A counter used to make sure we're only invokving the dispatch callback as # many times as we expect. my $count := 0; # A type-name dispatcher that maps a type into a constant string value that # is its name. This isn't terribly useful, but it is a decent small example. nqp::dispatch('boot-syscall', 'dispatcher-register', 'type-name', -> $capture { # Bump the counter, just for testing purposes. $count++; # Obtain the value of the argument from the capture (using an existing # MoarVM op, though in the future this may go away in place of a syscall) # and then obtain the string typename also. my $arg-val := nqp::captureposarg($capture, 0); my str $name := $arg-val.HOW.name($arg-val); # This outcome is only going to be valid for a particular type. We track # the argument (which gives us an object back that we can use to guard # it) and then add the type guard. my $arg := nqp::dispatch('boot-syscall', 'dispatcher-track-arg', $capture, 0); nqp::dispatch('boot-syscall', 'dispatcher-guard-type', $arg); # Finally, insert the type name at the start of the capture and then # delegate to the boot-constant dispatcher. nqp::dispatch('boot-syscall', 'dispatcher-delegate', 'boot-constant', nqp::dispatch('boot-syscall', 'dispatcher-insert-arg-literal-str', $capture, 0, $name)); }); # A use of the dispatch for the tests. Put into a sub so there's a single # static dispatch op, which all dispatch programs will hang off. sub type-name($obj) { nqp::dispatch('type-name', $obj) } # Check with the first type, making sure the guard matches when it should # (although this test would pass if the guard were ignored too). ok(type-name(C1) eq 'C1', 'Dispatcher setting guard works'); ok($count == 1, 'Dispatch callback ran once'); ok(type-name(C1) eq 'C1', 'Can use it another time with the same type'); ok($count == 1, 'Dispatch callback was not run again'); # Test it with a second type, both record and run modes. This ensures the # guard really is being checked. ok(type-name(C2) eq 'C2', 'Can handle polymorphic sites when guard fails'); ok($count == 2, 'Dispatch callback ran a second time for new type'); ok(type-name(C2) eq 'C2', 'Second call with new type works'); # Check that we can use it with the original type too, and it has stacked # the dispatch programs up at the same callsite. ok(type-name(C1) eq 'C1', 'Call with original type still works'); ok($count == 2, 'Dispatch callback only ran a total of 2 times'); </pre></div> <p>This time two dispatch programs get produced, one for <code>C1</code>:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> Dispatch program (1 temporaries) Ops: Guard arg 0 (type=C1) Load collectable constant at index 1 into temporary 0 Set result string value from temporary 0 </pre></div> <p>And another for C2:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> Dispatch program (1 temporaries) Ops: Guard arg 0 (type=C2) Load collectable constant at index 1 into temporary 0 Set result string value from temporary 0 </pre></div> <p>Once again, no leftovers from capture manipulation, tracking, or dispatcher delegation; the dispatch program does a type guard against an argument, then produces the result string. The whole call to <code>$arg-val.HOW.name($arg-val)</code> is elided, the dispatcher we wrote encoding the knowledge – in a way that the VM can understand – that a type’s name can be considered immutable.</p> <p>This example is a bit contrived, but now consider that we instead look up a method and guard on the invocant type: that’s a method cache! Guard the types of more of the arguments, and we have a multi cache! Do both, and we have a multi-method cache.</p> <p>The latter is interesting in so far as both the method dispatch <em>and</em> the multi dispatch want to guard on the invocant. In fact, in MoarVM today there will be two such type tests until we get to the point where the specializer does its work and eliminates these duplicated guards. However, the new dispatcher does not treat the <code>dispatcher-guard-type</code> as a kind of imperative operation that writes a guard into the resultant dispatch program. Instead, it declares that the argument in question must be guarded. If some other dispatcher already did that, it’s idempotent. The guards are emitted once all dispatch programs we delegate through, on the path to a final outcome, have had their say.</p> <p><em>Fun aside:</em> those being especially attentive will have noticed that the dispatch mechanism is used as part of implementing new dispatchers too, and indeed, this ultimately will mean that the specializer can specialize the dispatchers and have them JIT-compiled into something more efficient too. After all, from the perspective of MoarVM, it’s all just bytecode to run; it’s just that some of it is bytecode that tells the VM how to execute Raku programs more efficiently!</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/157fd5bd91a45cf6e62ecf0b9099bfba#dispatch-resumption"></a>Dispatch resumption</h3> <p>A resumable dispatcher needs to do two things:</p> <ol class="wp-block-list"><li>Provide a resume callback as well as a dispatch one when registering the dispatcher</li><li>In the dispatch callback, specify a capture, which will form the resume initialization state</li></ol> <p>When a resumption happens, the resume callback will be called, with any arguments for the resumption. It can also obtain the resume initialization state that was set in the dispatch callback. The resume initialization state contains the things needed in order to continue with the dispatch <em>the first time it is resumed</em>. We’ll take a look at how this works for method dispatch to see a concrete example. I’ll also, at this point, switch to looking at the real Rakudo dispatchers, rather than simplified test cases.</p> <p>The Rakudo dispatchers take advantage of delegation, duplicate guards, and capture manipulations all having no runtime cost in the resulting dispatch program to, in my mind at least, quite nicely factor what is a somewhat involved dispatch process. There are multiple entry points to method dispatch: the normal boring <code>$obj.meth()</code>, the qualified <code>$obj.Type::meth()</code>, and the call me maybe <code>$obj.?meth()</code>. These have common resumption semantics – or at least, they can be made to provided we always carry a starting type in the resume initialization state, which is the type of the object that we do the method dispatch on.</p> <p>Here is the entry point to dispatch for a normal method dispatch, with the boring details of reporting missing method errors stripped out.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # A standard method call of the form $obj.meth($arg); also used for the # indirect form $obj."$name"($arg). It receives the decontainerized invocant, # the method name, and the the args (starting with the invocant including any # container). nqp::dispatch('boot-syscall', 'dispatcher-register', 'raku-meth-call', -> $capture { # Try to resolve the method call using the MOP. my $obj := nqp::captureposarg($capture, 0); my str $name := nqp::captureposarg_s($capture, 1); my $meth := $obj.HOW.find_method($obj, $name); # Report an error if there is no such method. unless nqp::isconcrete($meth) { !!! 'Error reporting logic elided for brevity'; } # Establish a guard on the invocant type and method name (however the name # may well be a literal, in which case this is free). nqp::dispatch('boot-syscall', 'dispatcher-guard-type', nqp::dispatch('boot-syscall', 'dispatcher-track-arg', $capture, 0)); nqp::dispatch('boot-syscall', 'dispatcher-guard-literal', nqp::dispatch('boot-syscall', 'dispatcher-track-arg', $capture, 1)); # Add the resolved method and delegate to the resolved method dispatcher. my $capture-delegate := nqp::dispatch('boot-syscall', 'dispatcher-insert-arg-literal-obj', $capture, 0, $meth); nqp::dispatch('boot-syscall', 'dispatcher-delegate', 'raku-meth-call-resolved', $capture-delegate); }); </pre></div> <p>Now for the resolved method dispatcher, which is where the resumption is handled. First, let’s look at the normal dispatch callback (the resumption callback is included but empty; I’ll show it a little later).</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # Resolved method call dispatcher. This is used to call a method, once we have # already resolved it to a callee. Its first arg is the callee, the second and # third are the type and name (used in deferral), and the rest are the args to # the method. nqp::dispatch('boot-syscall', 'dispatcher-register', 'raku-meth-call-resolved', # Initial dispatch -> $capture { # Save dispatch state for resumption. We don't need the method that will # be called now, so drop it. my $resume-capture := nqp::dispatch('boot-syscall', 'dispatcher-drop-arg', $capture, 0); nqp::dispatch('boot-syscall', 'dispatcher-set-resume-init-args', $resume-capture); # Drop the dispatch start type and name, and delegate to multi-dispatch or # just invoke if it's single dispatch. my $delegate_capture := nqp::dispatch('boot-syscall', 'dispatcher-drop-arg', nqp::dispatch('boot-syscall', 'dispatcher-drop-arg', $capture, 1), 1); my $method := nqp::captureposarg($delegate_capture, 0); if nqp::istype($method, Routine) && $method.is_dispatcher { nqp::dispatch('boot-syscall', 'dispatcher-delegate', 'raku-multi', $delegate_capture); } else { nqp::dispatch('boot-syscall', 'dispatcher-delegate', 'raku-invoke', $delegate_capture); } }, # Resumption -> $capture { ... 'Will be shown later'; }); </pre></div> <p>There’s an arguable cheat in <code>raku-meth-call</code>: it doesn’t actually insert the type object of the invocant in place of the invocant. It turns out that it doesn’t really matter. Otherwise, I think the comments (which are to be found in the real implementation also) tell the story pretty well.</p> <p>One important point that may not be clear – but follows a repeating theme – is that the setting of the resume initialization state is also more of a declarative rather than an imperative thing: there isn’t a runtime cost at the time of the dispatch, but rather we keep enough information around in order to be able to reconstruct the resume initialization state at the point we need it. (In fact, when we are in the run phase of a resume, we don’t even have to reconstruct it in the sense of creating a capture object.)</p> <p>Now for the resumption. I’m going to present a heavily stripped down version that only deals with the <code>callsame</code> semantics (the full thing has to deal with such delights as <code>lastcall</code> and <code>nextcallee</code> too). The resume initialization state exists to seed the resumption process. Once we know we actually do have to deal with resumption, we can do things like calculating the full list of methods in the inheritance graph that we want to walk through. Each resumable dispatcher gets a single storage slot on the call stack that it can use for its state. It can initialize this in the first step of resumption, and then update it as we go. Or more precisely, it can set up a dispatch program that will do this when run.</p> <p>A linked list turns out to be a very convenient data structure for the chain of candidates we will walk through. We can work our way through a linked list by keeping track of the current node, meaning that there need only be a single thing that mutates, which is the current state of the dispatch. The dispatch program mechanism also provides a way to read an attribute from an object, and that is enough to express traversing a linked list into the dispatch program. This also means zero allocations.</p> <p>So, without further ado, here is the linked list (rather less pretty in NQP, the restricted Raku subset, than it would be in full Raku):</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> # A linked list is used to model the state of a dispatch that is deferring # through a set of methods, multi candidates, or wrappers. The Exhausted class # is used as a sentinel for the end of the chain. The current state of the # dispatch points into the linked list at the appropriate point; the chain # itself is immutable, and shared over (runtime) dispatches. my class DeferralChain { has $!code; has $!next; method new($code, $next) { my $obj := nqp::create(self); nqp::bindattr($obj, DeferralChain, '$!code', $code); nqp::bindattr($obj, DeferralChain, '$!next', $next); $obj } method code() { $!code } method next() { $!next } }; my class Exhausted {}; </pre></div> <p>And finally, the resumption handling.</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> nqp::dispatch('boot-syscall', 'dispatcher-register', 'raku-meth-call-resolved', # Initial dispatch -> $capture { ... 'Presented earlier; }, # Resumption. The resume init capture's first two arguments are the type # that we initially did a method dispatch against and the method name # respectively. -> $capture { # Work out the next method to call, if any. This depends on if we have # an existing dispatch state (that is, a method deferral is already in # progress). my $init := nqp::dispatch('boot-syscall', 'dispatcher-get-resume-init-args'); my $state := nqp::dispatch('boot-syscall', 'dispatcher-get-resume-state'); my $next_method; if nqp::isnull($state) { # No state, so just starting the resumption. Guard on the # invocant type and name. my $track_start_type := nqp::dispatch('boot-syscall', 'dispatcher-track-arg', $init, 0); nqp::dispatch('boot-syscall', 'dispatcher-guard-type', $track_start_type); my $track_name := nqp::dispatch('boot-syscall', 'dispatcher-track-arg', $init, 1); nqp::dispatch('boot-syscall', 'dispatcher-guard-literal', $track_name); # Also guard on there being no dispatch state. my $track_state := nqp::dispatch('boot-syscall', 'dispatcher-track-resume-state'); nqp::dispatch('boot-syscall', 'dispatcher-guard-literal', $track_state); # Build up the list of methods to defer through. my $start_type := nqp::captureposarg($init, 0); my str $name := nqp::captureposarg_s($init, 1); my @mro := nqp::can($start_type.HOW, 'mro_unhidden') ?? $start_type.HOW.mro_unhidden($start_type) !! $start_type.HOW.mro($start_type); my @methods; for @mro { my %mt := nqp::hllize($_.HOW.method_table($_)); if nqp::existskey(%mt, $name) { @methods.push(%mt{$name}); } } # If there's nothing to defer to, we'll evaluate to Nil (just don't set # the next method, and it happens below). if nqp::elems(@methods) >= 2 { # We can defer. Populate next method. @methods.shift; # Discard the first one, which we initially called $next_method := @methods.shift; # The immediate next one # Build chain of further methods and set it as the state. my $chain := Exhausted; while @methods { $chain := DeferralChain.new(@methods.pop, $chain); } nqp::dispatch('boot-syscall', 'dispatcher-set-resume-state-literal', $chain); } } elsif !nqp::istype($state, Exhausted) { # Already working through a chain of method deferrals. Obtain # the tracking object for the dispatch state, and guard against # the next code object to run. my $track_state := nqp::dispatch('boot-syscall', 'dispatcher-track-resume-state'); my $track_method := nqp::dispatch('boot-syscall', 'dispatcher-track-attr', $track_state, DeferralChain, '$!code'); nqp::dispatch('boot-syscall', 'dispatcher-guard-literal', $track_method); # Update dispatch state to point to next method. my $track_next := nqp::dispatch('boot-syscall', 'dispatcher-track-attr', $track_state, DeferralChain, '$!next'); nqp::dispatch('boot-syscall', 'dispatcher-set-resume-state', $track_next); # Set next method, which we shall defer to. $next_method := $state.code; } else { # Dispatch already exhausted; guard on that and fall through to returning # Nil. my $track_state := nqp::dispatch('boot-syscall', 'dispatcher-track-resume-state'); nqp::dispatch('boot-syscall', 'dispatcher-guard-literal', $track_state); } # If we found a next method... if nqp::isconcrete($next_method) { # Call with same (that is, original) arguments. Invoke with those. # We drop the first two arguments (which are only there for the # resumption), add the code object to invoke, and then leave it # to the invoke or multi dispatcher. my $just_args := nqp::dispatch('boot-syscall', 'dispatcher-drop-arg', nqp::dispatch('boot-syscall', 'dispatcher-drop-arg', $init, 0), 0); my $delegate_capture := nqp::dispatch('boot-syscall', 'dispatcher-insert-arg-literal-obj', $just_args, 0, $next_method); if nqp::istype($next_method, Routine) && $next_method.is_dispatcher { nqp::dispatch('boot-syscall', 'dispatcher-delegate', 'raku-multi', $delegate_capture); } else { nqp::dispatch('boot-syscall', 'dispatcher-delegate', 'raku-invoke', $delegate_capture); } } else { # No method, so evaluate to Nil (boot-constant disregards all but # the first argument). nqp::dispatch('boot-syscall', 'dispatcher-delegate', 'boot-constant', nqp::dispatch('boot-syscall', 'dispatcher-insert-arg-literal-obj', $capture, 0, Nil)); } }); </pre></div> <p>That’s quite a bit to take in, and quite a bit of code. Remember, however, that this is only run for the record phase of a dispatch resumption. It also produces a dispatch program at the callsite of the <code>callsame</code>, with the usual guards and outcome. Implicit guards are created for the dispatcher that we are resuming at that point. In the most common case this will end up monomorphic or bimorphic, although situations involving nestings of multiple dispatch or method dispatch could produce a more morphic callsite.</p> <p>The design I’ve picked forces resume callbacks to deal with two situations: the first resumption and the latter resumptions. This is not ideal in a couple of ways:</p> <ol class="wp-block-list"><li>It’s a bit inconvenient for those writing dispatch resume callbacks. However, it’s not like this is a particularly common activity!</li><li>The difference results in two dispatch programs being stacked up at a callsite that might otherwise get just one</li></ol> <p>Only the second of these really matters. The reason for the non-uniformity is to make sure that the overwhelming majority of calls, which never lead to a dispatch resumption, incur no per-dispatch cost for a feature that they never end up using. If the result is a little more cost for those using the feature, so be it. In fact, early benchmarking shows <code>callsame</code> with <code>wrap</code> and method calls seems to be up to 10 times faster using the new dispatcher than in current Rakudo, and that’s <em>before</em> the specializer understands enough about it to improve things further!</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/157fd5bd91a45cf6e62ecf0b9099bfba#whats-done-so-far"></a>What’s done so far</h3> <p>Everything I’ve discussed above is implemented, except that I may have given the impression somewhere that multiple dispatch is fully implemented using the new dispatcher, and that is not the case yet (no handling of <code>where</code> clauses and no dispatch resumption support).</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/157fd5bd91a45cf6e62ecf0b9099bfba#next-steps"></a>Next steps</h3> <p>Getting the missing bits of multiple dispatch fully implemented is the obvious next step. The other missing semantic piece is support for <code>callwith</code> and <code>nextwith</code>, where we wish to change the arguments that are being used when moving to the next candidate. A few other minor bits aside, that in theory will get all of the Raku dispatch semantics at least supported.</p> <p>Currently, all standard method calls (<code>$obj.meth()</code>) and other calls (<code>foo()</code> and <code>$foo()</code>) go via the existing dispatch mechanism, not the new dispatcher. Those will need to be migrated to use the new dispatcher also, and any bugs that are uncovered will need fixing. That will get things to the point where the new dispatcher is <em>semantically ready</em>.</p> <p>After that comes performance work: making sure that the specializer is able to deal with dispatch program guards and outcomes. The goal, initially, is to get steady state performance of common calling forms to perform at least as well as in the current <code>master</code> branch of Rakudo. It’s already clear enough there will be some big wins for some things that to date have been glacial, but it should not come at the cost of regression on the most common kinds of dispatch, which have received plenty of optimization effort before now.</p> <p>Furthermore, NQP – the restricted form of Raku that the Rakudo compiler and other bits of the runtime guts are written in – also needs to be migrated to use the new dispatcher. Only when that is done will it be possible to rip out the current method cache, multiple dispatch cache, and so forth from MoarVM.</p> <p>An open question is how to deal with backends other than MoarVM. Ideally, the new dispatch mechanism will be ported to those. A decent amount of it should be possible to express in terms of the JVM’s <code>invokedynamic</code> (and this would all probably play quite well with a Truffle-based Raku implementation, although I’m not sure there is a current active effort in that area).</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/157fd5bd91a45cf6e62ecf0b9099bfba#future-opportunities"></a>Future opportunities</h3> <p>While my current focus is to ship a Rakudo and MoarVM release that uses the new dispatcher mechanism, that won’t be the end of the journey. Some immediate ideas:</p> <ul class="wp-block-list"><li>Method calls on roles need to pun the role into a class, and so method lookup returns a closure that does that and replaces the invocant. That’s a lot of indirection; the new dispatcher could obtain the pun and produce a dispatch program that replaces the role type object with the punned class type object, which would make the per-call cost far lower.</li><li>I expect both the <code>handles</code> (delegation) and <code>FALLBACK</code> (handling missing method call) mechanisms can be made to perform better using the new dispatcher</li><li>The current implementation of <code>assuming</code> – used to curry or otherwise prime arguments for a routine – is not ideal, and an implementation that takes advantage of the argument rewriting capabilities of the new dispatcher would likely perform a great deal better</li></ul> <p>Some new language features may also be possible to provide in an efficient way with the help of the new dispatch mechanism. For example, there’s currently not a reliable way to try to invoke a piece of code, just run it if the signature binds, or to do something else if it doesn’t. Instead, things like the Cro router have to first do a trial bind of the signature, and then do the invoke, which makes routing rather more costly. There’s also the long suggested idea of providing pattern matching via signatures with the <code>when</code> construct (for example, <code>when * -> ($x) {}; when * -> ($x, *@tail) { }</code>), which is pretty much the same need, just in a less dynamic setting.</p> <h3 class="wp-block-heading"><a href="https://gist.github.com/jnthn/157fd5bd91a45cf6e62ecf0b9099bfba#in-closing"></a>In closing…</h3> <p>Working on the new dispatch mechanism has been a longer journey than I first expected. The resumption part of the design was especially challenging, and there’s still a few important details to attend to there. Something like four potential approaches were discarded along the way (although elements of all of them influenced what I’ve described in this post). Abstractions that hold up are really, really, hard.</p> <p>I also ended up having to take a couple of months away from doing Raku work at all, felt a bit crushed during some others, and have been juggling this with the equally important RakuAST project (which will be simplified by being able to assume the presence of the new dispatcher, and also offers me a range of softer Raku hacking tasks, whereas the dispatcher work offers few easy pickings).</p> <p>Given all that, I’m glad to finally be seeing the light at the end of the tunnel. The work that remains is enumerable, and the day we ship a Rakudo and MoarVM release using the new dispatcher feels a small number of months away (and I hope writing that is not tempting fate!)</p> <p>The new dispatcher is probably the most significant change to MoarVM since I founded it, in so far as it sees us removing a bunch of things that have been there pretty much since the start. RakuAST will also deliver the greatest architectural change to the Rakudo compiler in a decade. Both are an opportunity to fold years of learning things the hard way into the runtime and compiler. I hope when I look back at it all in another decade’s time, I’ll at least feel I made more interesting mistakes this time around.</p> <h2><a href="http://brrt-to-the-future.blogspot.com/2021/03/why-bother-with-scripting.html">Why bother with Scripting?</h2></a> <p><h3>Published by Bart Wiegmans on 2021-03-14T14:33:00</h3></p> <p>Many years back, Larry Wall shared his <a href="https://www.perl.com/pub/2007/12/06/soto-11.html/" target="_blank">thesis</a> on the nature of scripting. Since recently even <a href="https://openjdk.java.net/jeps/330">Java</a> gained 'script' support I thought it would be fitting to revisit the topic, and hopefully relevant to the perl and raku language community.<br /></p><p>The weakness of Larry's treatment (which, to be fair to the author, I think is more intended to be enlightening than to be complete) is the contrast of <i>scripting</i> with <i>programming</i>. This contrast does not permit a clear separation because <i>scripts</i> are <i>programs</i>. That is to say, no matter how long or short, scripts are written commands for a machine to execute, and I think that's a pretty decent definition of a program in general.</p><p>A more useful contrast - and, I think, the intended one - is between <i>scripts</i> and <i>other sorts of programs</i>, because that allows us to compare <i>scripting</i> (writing scripts) with 'programming' (writing non-script programs). And to do that we need to know what other sorts of programs there are.</p><p>The short version of that answer is - systems and applications, and a bunch of other things that aren't really relevant to the working programmer, like (embedded) control algorithms, spreadsheets and database queries. (The definition I provided above is very broad, by design, because I don't want to get stuck on boundary questions). Most programmers write applications, some write systems, virtually all write scripts once in a while, though plenty of people who <i>aren't</i> professional programmers also write scripts.</p><p>I think the defining features of applications and systems are, respectively:</p><ul style="text-align: left;"><li>Applications present <i>models</i> to users (for <i>manipulation</i>)</li><li>Systems provide <i>functionality</i> to other programs</li></ul><p>Consider for instance a mail client (like thunderbird) in comparison to a mailer daemon (like sendmail) - one provides an interface to read and write e-mails (the model) and the other provides functionality to send that e-mail to other servers.</p><p>Note that under this (again, broad) definition, <i>libraries</i> are also system software, which makes sense, considering that their users are developers (just as for, say, PostgreSQL) who care about things like performance, reliability, and correctness. Incidentally, libraries as well as 'typical' system software (such as database engines and operating system kernels) tend to be written in languages like C and C++ for much the same reasons.</p><p>What then, are the differences between scripts, applications, and systems? I think the following is a good list:</p><ul style="text-align: left;"><li>Scripts tend to be short, applications in particular can grow very large.<br /></li><li>Scripts tend to be ad-hoc (written for a specific need), applications and systems tend to be designed for a range of use cases. (Very common example: build scripts)<br /></li><li>Scripts tend to run only in a specific environment; in contrast, many applications are designed for a range of devices/clients; many systems have specific requirements but the intention is that they can be setup in multiple distinct environments. <br /></li><li>Because scripts are ad-hoc, short, and environment-dependent, many of software engineering standard best practices don't really apply (and are in fact often disregarded).<br /></li></ul><p>Obviously these distinctions aren't really binary - 'short' versus 'long', 'ad-hoc' versus 'general purpose'  - and can't be used to conclusively settle the question whether something is a script or an application. (If, indeed, that question ever comes up). More important is that for the 10 or so scripts I've written over the past year - some professionally, some not - all or most of these properties held, and I'd be surprised if the same isn't true for most readers. </p><p>And - finally coming at the point that I'm trying to make today - these features point to a specific <i>niche</i> of programs more than to a specific <i>technology</i> (or set of technologies). To be exact, scripts are (mostly) short, custom programs to automate ad-hoc tasks, tasks that are either to specific or too small to develop and distribute another program for.<br /></p><p>This has further implications on the preferred features of a <i>scripting language</i> (taken to mean, a language designed to enable the development of scripts). In particular:</p><ul style="text-align: left;"><li>It should make programs concise. The economic rationalization is that the total expected lifetime value of a script, being ad-hoc and context-dependent, is not very great, so writing it should be cheap, which implies that the script should be short).</li><li>Related to this, the value provided by type systems is generally less than in larger (application) programs, and the value of extensive modelling features (class hierarchies) is similarly low, so many scripting languages have very weak type systems and data modelling features, if they have them at all.</li><li>Interoperation with the environment is on the other hand very important, so I/O features tend to be well-developed. (Contrast C, in which I/O is entirely an afterthought provided by a library).<br /></li><li>It is acceptable to depend on a local environment in implicit ways, since that's what you are going to do anyway.</li><li>It is acceptable to <i>warn</i> on a condition that might've been a fatal error in another programming language.</li><li>In fact, I think that concerns of correctness are often different, meaning relaxed, compared to applications, again because scripters don't necessarily expect their scripts to run on every environment and with every possible input. <br /></li></ul>As an example of the last point - Python 3 requires users to be exact about the encoding of their input, causing <i>all sorts</i> of trouble for unsuspecting scripters when they accidentally try to read ISO-8551 data as UTF-8, or vice versa. Python 2 did not, and for most <i>scripts</i> - not applications - I actually think that is the right choice.<p>This niche doesn't always exist. In computing environments where everything of interest is adequately captured by an application, or which lacks the ability to effectively automate ad-hoc tasks (I'm thinking in particular of Windows before PowerShell), the practice of scripting tends to not develop. Similarily, in a modern 'cloud' environment, where system setup is controlled by a <a href="https://kubernetes.io/">state machine</a> hosted by <a href="https://cloud.google.com/kubernetes-engine" target="_blank">another organization</a>, scripting doesn't really have much of a future.</p><p>To put it another way, scripting only thrives in an environment that has a lot of 'scriptable' tasks; meaning tasks for which there isn't already a pre-made solution available, environments that have powerful facilities available for a script to access, and whose users are empowered to automate those tasks. Such qualities are common on Unix/Linux 'workstations' but rather less so on smartphones and (as noted before) cloud computing environments.</p><p>Truth be told I'm a little worried about that development. I could point to, and expound on, the development and popularity of languages like go and rust, which aren't exactly scripting languages, or the replacement of Javascript with TypeScript, to make the point further, but I don't think that's necessary. At the same time I could point to the development of data science as a discipline to demonstrate that scripting is alive and well (and indeed perhaps more economically relevant than before).</p><p>What should be the conclusion for perl 5/7 and raku? I'm not quite sure, mostly because I'm not quite sure whether the broader perl/raku community would prefer their sister languages to be scripting or application languages. (As implied above, I think the Python community chose that they wanted Python 3 to be an application language, and this was not without consequences to their users). </p><p>Raku adds a number of features common to application languages (I'm thinking of it's powerful type system in particular), continuing a trend that perl 5 arguably pioneered. This is indeed a very powerful strategy - a language can be introduced for scripts and some of those scripts are then extended into applications (or even systems), thereby ensuring its continued usage. But for it to work, a new<i> </i>perl family language must be introduced on its <i>scripting</i> merits, <i>and</i> there must be a plentiful supply of scriptable tasks to automate, some of which - or a combination of which - grow into an application.<br /></p><p>For myself, I would like to see scripting have a bright future. Not just because scripting is the most accessible form of programming, but also because an environment that permits, even requires scripting, is one were not all interesting problems have been solved, one where it's users ask it to do tasks so diverse that there <i>isn't </i>an app for that, yet. One where the true potential of the wonderful devices that surround is can be explored.<br /></p><p>In such a world there might well be a bright future for scripting.<br /></p> <h2><a href="https://6guts.wordpress.com/2020/10/05/taking-a-break-from-raku-core-development/">Taking a break from Raku core development</h2></a> <p><h3>Published by jnthnwrthngtn on 2020-10-05T19:44:26</h3></p> <p>I’d like to thank everyone who voted for me in the recent Raku Steering Council elections. By this point, I’ve been working on the language for well over a decade, first to help turn a language design I found fascinating into a working implementation, and since the Christmas release to make that implementation more robust and performant. Overall, it’s been as fun as it has been challenging – in a large part because I’ve found myself sharing the journey with a lot of really great people. I’ve also tried to do my bit to keep the community around the language kind and considerate. Receiving a vote from around 90% of those who participated in the Steering Council elections was humbling.</p> <p>Alas, I’ve today submitted my resignation to the Steering Council, on personal health grounds. For the same reason, I’ll be taking a step back from Raku core development (Raku, MoarVM, language design, etc.) Please don’t worry too much; I’ll almost certainly be fine. It may be I’m ready to continue working on Raku things in a month or two. It may also be longer. Either way, I think Raku will be better off with a fully sized Steering Council in place, and I’ll be better off without the anxiety that I’m holding a role that I’m not in a place to fulfill.</p> <h1 class="wp-block-heading"><a href="https://gist.github.com/jnthn/0015f339be0c92c56fdc713852bce0be#steering-council"></a></h1> <h2><a href="http://brrt-to-the-future.blogspot.com/2019/03/reverse-linear-scan-allocation-is.html">Reverse Linear Scan Allocation is probably a good idea</h2></a> <p><h3>Published by Bart Wiegmans on 2019-03-21T15:52:00</h3></p> Hi hackers! Today First of all, I want to thank everybody who gave such useful <a href="https://news.ycombinator.com/item?id=19425485" target="_blank">feedback</a> on my last post.  For instance, I found out that the similarity between the expression JIT IR and the Testarossa <a href="https://github.com/eclipse/omr/blob/master/doc/compiler/il/IntroToTrees.md" target="_blank">Trees IR</a> is quite remarkable, and that they have a fix for the problem that is quite different from what I had in mind.<br /> <br /> Today I want to write something about register allocation, however. Register allocation is probably not my favorite problem, on account of being both messy and thankless. It is a messy problem because - aside from being NP-hard to solve optimally - hardware instruction sets and software ABI's introduce all sorts of annoying constraints. And it is a thankless problem because the case in which a good register allocator is useful - for instance, when there's lots of intermediate values used over a long stretch of code - are fairly rare. Much more common are the cases in which either there are trivially sufficient registers, <i>or </i>ABI constraints force a spill to memory anyway (e.g. when calling a function, almost all registers can be overwritten).<br /> <br /> So, on account of this being not my favorite problem, and also because I promised to implement optimizations in the register allocator, I've been researching if there is a way to do better. And what better place to look than one of the fastest dynamic language implementations arround, <a href="http://luajit.org/luajit.html" target="_blank">LuaJIT</a>? So that's what I did, and this post is about what I learned from that.<br /> <br /> Truth be told, LuaJIT is not at all a learners' codebase (and I don't think it's author would claim this). It uses a rather terse style of C and lots and lots of preprocessor macros. I had somewhat gotten used to the style from hacking <a href="https://github.com/MoarVM/dynasm/tree/x64-dynamic-registers" target="_blank">dynasm</a> though, so that wasn't so bad. What was more surprising is that some of the steps in code generation that are distinct and separate in the MoarVM JIT - instruction selection, register allocation and emitting bytecode - were all blended together in LuaJIT. Over multiple backend architectures, too. And what's more - all these steps were done in <i>reverse order</i> - from the end of the program (trace) to the beginning. Now that's interesting...<br /> <br /> I have no intention of combining all phases of code generation like LuaJIT has. But processing the IR in reverse seems to have some interesting properties. To understand why that is, I'll first have to explain how linear scan allocation currently works in MoarVM, and is most commonly <a href="http://web.cs.ucla.edu/~palsberg/course/cs132/linearscan.pdf" target="_blank">described</a>:<br /> <br /> <ol> <li>First, the <b>live ranges </b>of program values are computed. Like the name indicates, these represent the range of the program code in which a value is both defined and may be used. Note that for the purpose of register allocation, the notion of a value shifts somewhat. In the expression DAG IR, a value is the result of a single computation. But for the purposes of register allocation, a value includes all its copies, as well as values computed from different conditional branches. This is necessary because when we actually start allocating registers, we need to know when a value is no longer in use (so we can reuse the register) and how long a value will remain in use -</li> <li>Because a value may be computed from distinct conditional branches, it is necessary to compute the <b>holes</b><i> </i>in the live ranges. Holes exists because if a value is defined in both sides of a conditional branch, the range will cover both the earlier (in code order) branch and the later branch - but from the start of the later branch to its definition that value doesn't actually exist. We need this information to prevent the register allocator from trying to spill-and-load a nonexistent value, for instance.</li> <li>Only then can we allocate and assign the actual registers to instructions. Because we might have to <b>spill</b><b style="font-style: italic;"> </b>values to memory, and because values now can have multiple definitions, this is a somewhat subtle problem. Also, we'll have to resolve all architecture specific register requirements in this step.</li> </ol> <div> In the MoarVM register allocator, there's a fourth step and a fifth step. The fourth step exists to ensure that instructions conform to x86 two-operand form (Rather than return the result of an instruction in a third register, x86 reuses one of the input registers as the output register. E.g. all operators are of the form <span style="font-family: Courier New, Courier, monospace;">a = op(a, b)</span>  rather than <span style="font-family: Courier New, Courier, monospace;">a = op(b, c)</span>. This saves on instruction encoding space). The fifth step inserts instructions that are introduced by the third step; this is done so that each instruction has a fixed address in the stream while the stream is being processed.</div> <div> <br /></div> <div> Altogether this is quite a bit of complexity and work, even for what is arguably the simplest correct global register allocation algorithm. So when I started thinking of the reverse linear scan algorithm employed by LuaJIT, the advantages became clear:</div> <div> <ul> <li>In LuaJIT, the IR maintains its SSA form - there is only a single definition of a value. This means that when iterating in reverse order, computing the live range becomes trivial. When we first encounter a use of a value, then by definition that is the last use. And when we encounter a definition, that is the only and single definition, and we can release the register.  So there's no need to compute the live range in advance of allocation.</li> <li>Furthermore, rather than <i>merging</i> the values of multiple branches into the same live range, each value on either side becomes an individual live range. As a result, the live range of a value never has a hole, further simplifying code.</li> <li>LuaJIT uses register <i>hints</i> to indicate which registers could best be picked for a specific value. This is often determined by how a value is used (e.g., the divisor in a <span style="font-family: Courier New, Courier, monospace;">div</span> instruction must be in the <span style="font-family: Courier New, Courier, monospace;">rcx</span> register). If the preferred register can't be allocated, the register allocator inserts code to move it to the right place where needed. Having hints can be expected to greatly reduce the need for such code.</li> </ul> <div> There are downsides as well, of course. Not knowing exactly how long a value will be live while processing it may cause the algorithm to make worse choices in which values to spill. But I don't think that's really a great concern, since figuring out the best possible value is practically impossible anyway, and the most commonly cited heuristic - evict the value that is live furthest in the future, because this will release a register over a longer range of code, reducing the chance that we'll need to evict again - is still available. (After all, we <i>do</i> always know the last use, even if we don't necessarily know the first definition).</div> </div> <div> <br /></div> <div> Altogether, I'm quite excited about this algorithm; I think it will be a real simplification over the current implementation. Whether that will work out remains to be seen of course. I'll let you know!</div> <h2><a href="http://brrt-to-the-future.blogspot.com/2019/03/something-about-ir-optimization.html">Something about IR optimization</h2></a> <p><h3>Published by Bart Wiegmans on 2019-03-17T06:23:00</h3></p> Hi hackers! Today I want to write about optimizing IR in the MoarVM JIT, and also a little bit about IR design itself.<br /> <br /> One of the (major) design goals for the expression JIT was to have the ability to optimize code over the boundaries of individual MoarVM instructions. To enable this, the expression JIT first expands each VM instruction into a graph of lower-level operators. Optimization then means pattern-matching those graphs and replacing them with more efficient expressions.<br /> <br /> As a running example, consider the <span style="font-family: "courier new" , "courier" , monospace;">idx</span> operator. This operator takes two inputs (<span style="font-family: "courier new" , "courier" , monospace;">base</span> and <span style="font-family: "courier new" , "courier" , monospace;">element</span>) and a constant parameter <span style="font-family: "courier new" , "courier" , monospace;">scale</span> and computes <span style="font-family: "courier new" , "courier" , monospace;">base+element*scale</span>. This represents one of the operands of an  'indexed load' instruction on x86, typically used to process arrays. Such instructions allow one instruction to be used for what would otherwise be two operations (computing an address and loading a value). However, if the element of the <span style="font-family: "courier new" , "courier" , monospace;">idx</span> operator is a constant, we can replace it instead with the <span style="font-family: "courier new" , "courier" , monospace;">addr</span> instruction, which just adds a constant to a pointer. This is an improvement over <span style="font-family: "courier new" , "courier" , monospace;">idx</span> because we no longer need to load the value of element into a register. This saves both an instruction and valuable register space.<br /> <br /> Unfortunately this optimization introduces a bug. (Or, depending on your point of view, brings an existing bug out into the open). The expression JIT code generation process selects instructions for subtrees (tile) of the graph in a bottom-up fashion. These instructions represent the value computed or work performed by that subgraph. (For instance, a tree like <span style="font-family: "courier new" , "courier" , monospace;">(load (addr ? 8) 8)</span> becomes <span style="font-family: "courier new" , "courier" , monospace;">mov ?, qword [?+8]</span><span style="font-family: inherit;">; the question marks are filled in during register allocation). Because an instruction is always represents a <i>tree</i>, and because the graph is an arbitrary <i>directed acyclic graph</i>, the code generator projects that graph as a tree by visiting each operator node only once. So each value is computed once, and that computed value is reused by all later references.</span><br /> <span style="font-family: inherit;"><br /></span> It is worth going into some detail into why the expression graph is not a tree. Aside from transformations that might be introduced by optimizations (e.g. common subexpression elimination), a template may introduce a value that has multiple references via the <span style="font-family: "courier new" , "courier" , monospace;">let:</span> pseudo-operator. See for instance the following (simplified) template:<br /> <br /> <span style="font-family: "courier new" , "courier" , monospace;">(let: (($foo (<span style="color: red;">load</span> (local))))</span><br /> <span style="font-family: "courier new" , "courier" , monospace;">    (add <span style="color: red;">$foo</span> (sub <span style="color: red;">$foo</span> (const 1))))</span><br /> <span style="font-family: "courier new" , "courier" , monospace;"><br /></span> <table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody> <tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgSrA73CFVJNjTtip0-9OR3cDZLU3EICP5pkQEDqQiUpmK04BSDWv4xoBMk1JXxqSGLoCUMSTe4kNaPlFgIsVlor7c4XpHEi-rajShraLgiiqoLUPMseLPgSS2v1ugG-Up6O7SsJmvR9c/s1600/tree-1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="130" data-original-width="493" height="105" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgSrA73CFVJNjTtip0-9OR3cDZLU3EICP5pkQEDqQiUpmK04BSDWv4xoBMk1JXxqSGLoCUMSTe4kNaPlFgIsVlor7c4XpHEi-rajShraLgiiqoLUPMseLPgSS2v1ugG-Up6O7SsJmvR9c/s400/tree-1.png" width="400" /></a></td></tr> <tr><td class="tr-caption" style="text-align: center;">Both ADD and SUB refer to the same LOAD node</td></tr> </tbody></table> <div class="separator" style="clear: both; text-align: center;"> </div> <span style="font-family: "courier new" , "courier" , monospace;"><br /></span> <br /> In this case, both references to <span style="font-family: "courier new" , "courier" , monospace;">$foo</span> point directly to the <i>same</i> <span style="font-family: "courier new" , "courier" , monospace;">load</span> operator. Thus, the graph is not a tree. Another case in which this occurs is during linking of templates into the graph. The output of an instruction is used, if possible, directly as the input for another instruction. (This is the primary way that the expression JIT can get rid of unnecessary memory operations). But there can be multiple instructions that use a value, in which case an operator can have multiple references. Finally, instruction operands are inserted by the compiler and these can have multiple references as well.<br /> <br /> If each operator is visited only once during code generation, then this may introduce a problem when combined with another feature - conditional expressions. For instance, if two branches of a conditional expression both refer to the same value (represented by name <span style="font-family: "courier new" , "courier" , monospace;">$foo</span>) then the code generator will only emit code to compute its value when it encounters the <i>first</i> reference. When the code generator encounters <span style="font-family: "courier new" , "courier" , monospace;">$foo</span> for the second time in the other branch, <i>no code will be emitted.</i> This means that in the second branch, <span style="font-family: "courier new" , "courier" , monospace;">$foo</span> will effectively have no defined value (because the code in the first branch is never executed), and wrong values or memory corruption is then the predictable result.<br /> <br /> This bug has always existed for as long as the expression JIT has been under development, and in the past the solution has been not to write templates which have this problem. This is made a little easier by a feature the <span style="font-family: "courier new" , "courier" , monospace;">let:</span> operator, in that it inserts a <span style="font-family: "courier new" , "courier" , monospace;">do</span> operator which orders the values that are declared to be computed before the code that references them. So that this is in fact non-buggy:<br /> <span style="font-family: "courier new" , "courier" , monospace;"><br /></span> <span style="font-family: "courier new" , "courier" , monospace;">(let: (($foo (load (local))) # code to compute $foo is emitted here</span><br /> <span style="font-family: "courier new" , "courier" , monospace;">  (if (...)  </span><br /> <span style="font-family: "courier new" , "courier" , monospace;">    (add $foo (const 1)) # $foo is just a reference</span><br /> <span style="font-family: "courier new" , "courier" , monospace;">    (sub $foo (const 2)) # and here as well</span><br /> <br /> <table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody> <tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEil8wV6lSZdDoULUkrk6JawqzLjBL0gODGAh1rEIPN-cnjtHg1IQju4iLzmuAVPmzPIyVY2CnotdPIKQgrtVkrI9YOHSDHX7uZnCSbihzm503nkomghlk3ceVccZSze4M5tUYeqweKgF8E/s1600/tree-2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="264" data-original-width="613" height="171" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEil8wV6lSZdDoULUkrk6JawqzLjBL0gODGAh1rEIPN-cnjtHg1IQju4iLzmuAVPmzPIyVY2CnotdPIKQgrtVkrI9YOHSDHX7uZnCSbihzm503nkomghlk3ceVccZSze4M5tUYeqweKgF8E/s400/tree-2.png" width="400" /></a></td></tr> <tr><td class="tr-caption" style="text-align: center;">The DO node is inserted for the LET operator. It ensures that the value of the LOAD node is computed before the reference in either branch</td></tr> </tbody></table> <br /> <br /> Alternatively, if a value <span style="font-family: "courier new" , "courier" , monospace;">$foo</span> is used in the condition of the if operator, you can also be sure that it is available in both sides of the condition.<br /> <br /> All these methods rely on the programmer being able to predict when a value will be first referenced and hence evaluated. An optimizer breaks this by design. This means that if I want the JIT optimizer to be successful, my options are:<br /> <br /> <ol> <li>Fix the optimizer so as to not remove references that are critical for the correctness of the program</li> <li>Modify the input tree so that such references are either copied or moved forward</li> <li>Fix the code generator to emit code for a value, if it determines that an earlier reference is not available from the current block.</li> </ol> <div> In other words, I first need to decide where this bug really belongs - in the optimizer, the code generator, or even the IR structure itself. The weakness of the expression IR is that expressions don't really impose a particular order. (This is unlike the spesh IR, which is instruction-based, and in which every instruction has a 'previous' and 'next' pointer). Thus, there really isn't a 'first' reference to a value, before the code generator introduces the concept. This is property is in fact quite handy for optimization (for instance, we can evaluate operands in whatever order is best, rather than being fixed by the input order) - so I'd really like to preserve it. But it also means that the property we're interested in - a value is computed before it is used in, in all possible code flow paths - isn't really expressible by the IR. And there is no obvious <i>local invariant</i> that can be maintained to ensure that this bug does not happen, so any correctness check may have to check the entire graph, which is quite impractical.</div> <div> <br /></div> <div> I hope this post explains why this is such a tricky problem! I have some ideas for how to get out of this, but I'll reserve those for a later post, since this one has gotten quite long enough. Until next time!</div> <h2><a href="http://brrt-to-the-future.blogspot.com/2019/01/a-short-posts-about-types-and.html">A short post about types and polymorphism</h2></a> <p><h3>Published by Bart Wiegmans on 2019-01-14T13:34:00</h3></p> Hi all. I usually write somewhat long-winded posts, but today I'm going to try and make an exception. Today I want to talk about the expression template language used to map the high-level MoarVM instructions to low-level constructs that the JIT compiler can easily work with:<br /> <br /> This 'language' was designed back in 2015 subject to three constraints:<br /> <ul> <li>It should make it easy to develop 'templates' for MoarVM instructions, so we can map the ~800 or so different instructions supported by the interpreter to something the JIT compiler can work with.</li> <li>It should be simple to process and analyze; specifically, it should be suitable as input to the instruction selection process (the tiler).</li> <li>It should be simple to implement, both from the frontend (meaning the perl program that compiles a template file to a C header) and the backend (meaning the C code that combines templates into the IR that is compiled).</li> </ul> Recently I've been working on adding support for floating point operations, and  this means working on the type system of the expression language. Because floating point instructions operate on a distinct set of registers from integer instructions, a floating point operator is not interchangeable with an integer (or pointer) operator.<br /> <br /> This type system is enforced in two ways. First, by the template compiler, which attempts to check if you've used all operands correctly. This operates during development, which is convenient. Second, by instruction selection, as there will simply not be any instructions available that have the wrong combinations of types. Unfortunately, that happens at runtime, and such errors so annoying to debug that it motivated the development of the first type checker.<br /> <br /> However, this presents two problems. One of the advantages of the expression IR is that, by virtue of having a small number of operators, it is fairly easy to analyze. Having a distinct set of operators for each type would undo that. But more importantly, there are several MoarVM instructions that are <i>generic</i>, i.e. that operate on integer, floating point, and pointer values. (For example, the <span style="font-family: "courier new" , "courier" , monospace;">set</span>, <span style="font-family: "courier new" , "courier" , monospace;">getlex</span> and <span style="font-family: "courier new" , "courier" , monospace;">bindlex</span> instructions are generic in this way). This makes it impossible to know whether its values will be integers, pointers, or floats.<br /> <br /> This is no problem for the interpreter since it can treat values as bags-of-bits (i.e., it can simply copy the <span style="font-family: "courier new" , "courier" , monospace;">union MVMRegister</span> type that holds all values of all supported types). But the expression JIT works differently - it assumes that it can place any value in a register, and that it can reorder and potentially skip storing them to memory. (This saves work when the value would soon be overwritten anyway). So we need to know what register class that is, and we need to have the correct <i>operators</i> to manipulate a value in the right register class.<br /> <br /> To summarize, the problem is:<br /> <ul> <li>We need to know the type of each value, both to ensure we use the correct instructions and the right registers.</li> <li>There are several cases in which we don't really know (for the template) what type each value has.</li> </ul> There are two ways around this, and I chose to use both. First, we know as a fact for each local or lexical value in a MoarVM frame (subroutine) what type it should have. So even a generic operator like <span style="font-family: "courier new" , "courier" , monospace;">set</span> can be resolved to a specific type at runtime, at which point we can select the correct operators. Second, we can introduce generic operators of our own. This is possible so long as we can select the correct instruction for an operator based on the types of the operands.<br /> <br /> For instance, the <span style="font-family: "courier new" , "courier" , monospace;">store</span> operator takes two operands, an address and a value. Depending on the type of the value (<span style="font-family: "courier new" , "courier" , monospace;">reg</span> or <span style="font-family: "courier new" , "courier" , monospace;">num</span>), we can always select the correct instruction (<span style="font-family: "courier new" , "courier" , monospace;">mov</span> or <span style="font-family: "courier new" , "courier" , monospace;">movsd</span>). It is however <i>not</i> possible to select different instructions for the<span style="font-family: inherit;"> <span style="font-family: "courier new" , "courier" , monospace;">load</span> operator based on the type required, because instruction selection works from the bottom up. So we need a special <span style="font-family: "courier new" , "courier" , monospace;">load_num</span> operator, but a <span style="font-family: "courier new" , "courier" , monospace;">store_num</span> operator is not necessary.<span style="font-family: inherit;"> </span>And this is true for a <i>lot</i> more operators than I had initially thought. For instance, aside from the (naturally generic) <span style="font-family: "courier new" , "courier" , monospace;">do</span> and <span style="font-family: "courier new" , "courier" , monospace;">if</span> operators, all arithmetic operators and comparison operators </span>are generic in this way.<br /> <br /> I realize that, despite my best efforts, this has become a rather long-winded post anyway.....<br /> <br /> Anyway. For the next week, I'll be taking a slight detour, and I aim to generalize the two-operand form conversion that is necessary on x86. I'll try to write a blog about it as well, and maybe it'll be short and to the point. See you later! <h2><a href="http://perlgeek.de/blog-en/perl-6/2018-perl6-coding-contest-seeking-task-makers.html">Perl 6 Coding Contest 2019: Seeking Task Makers</h2></a> <p><h3>Published by Moritz Lenz on 2018-11-10T23:00:01</h3></p>  <p>I want to revive <a href="http://strangelyconsistent.org/blog/masaks-perl-6-coding-contest">Carl Mäsak's Coding Contest</a> as a <a href="https://gist.github.com/moritz/6f06ec7153ce396ac2734c2616d8b075">crowd-sourced contest</a>.</p> <p>The contest will be in four phases:</p> <ul> <li>development of tasks</li> <li>public contest, where you can solve the tasks</li> <li>public commenting on solutions</li> <li>grading and awarding of prizes</li> </ul> <p>For the first phase, development of tasks, I am looking for volunteers who come up with coding tasks collaboratively. Sadly, these volunteers, including myself, will be excluded from participating in the second phase.</p> <p>I am looking for tasks that ...</p> <ul> <li>are well-worded</li> <li>have a (private) implementation that shows it is solvable, preferably in under 500 lines of code</li> <li>have a few public tests to help the participants, and a more tests tests that help the graders and commenters later on</li> </ul> <p>This is non-trivial, so I'd like to have others to discuss things with, and to come up with some more tasks.</p> <p>If you want to help with task creation, please send an email to <a href="/cdn-cgi/l/email-protection#1875776a716c6236747d7662587f75797174367b7775"><span class="__cf_email__" data-cfemail="44292b362d303e6a28212a3e042329252d286a272b29">[email protected]</span></a>, stating your intentions to help, and your freenode IRC handle (optional).</p> <p>There are other ways to help too:</p> <ul> <li>You can pledge a prize (some ideas: <a href="https://commaide.com/">Comma IDE</a> subscription, books, Camelia plushies, Amazon vouchers)</li> <li>You can help design a small website for the contest</li> <li>You can help iron out the exact rules for the contest</li> <li>... or any other way that I didn't think of :-)</li> </ul> <p>In these cases you can use the same email address to contact me, or use IRC (<code>moritz</code> on freenode) or <a href="https://twitter.com/nogoodnickleft">twitter</a>.</p> <h2><a href="https://stmuk.wordpress.com/2017/08/30/swiss-perl-workshop-2017/">Swiss Perl Workshop 2017</h2></a> <p><h3>Published by stmuk on 2017-08-30T17:48:17</h3></p> <p><img data-attachment-id="25" data-permalink="https://stmuk.wordpress.com/cropped-lake_castle1-jpeg/" data-orig-file="https://stmuk.wordpress.com/wp-content/uploads/2017/08/cropped-lake_castle1.jpeg" data-orig-size="1440,1080" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="cropped-lake_castle1.jpeg" data-image-description="" data-image-caption="" data-medium-file="https://stmuk.wordpress.com/wp-content/uploads/2017/08/cropped-lake_castle1.jpeg?w=300" data-large-file="https://stmuk.wordpress.com/wp-content/uploads/2017/08/cropped-lake_castle1.jpeg?w=1024" class="alignnone size-full wp-image-25" src="https://stmuk.wordpress.com/wp-content/uploads/2017/08/cropped-lake_castle1.jpeg?w=1100" alt="cropped-lake_castle1.jpeg" srcset="https://stmuk.wordpress.com/wp-content/uploads/2017/08/cropped-lake_castle1.jpeg 1440w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/cropped-lake_castle1.jpeg?w=150&h=113 150w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/cropped-lake_castle1.jpeg?w=300&h=225 300w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/cropped-lake_castle1.jpeg?w=768&h=576 768w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/cropped-lake_castle1.jpeg?w=1024&h=768 1024w" sizes="(max-width: 1440px) 100vw, 1440px" /></p> <p><span style="font-weight:400;">After a perilous drive up a steep, narrow, winding road from Lake Geneva we arrived at an attractive Alpine village (Villars-sur-Ollon) to meet with fellow Perl Mongers in a small restaurant. There followed much talk and a little clandestine drinking of exotic spirits including Swiss whisky. The following morning walking to the conference venue there was an amazing view of mountain ranges. On arrival I failed to operate the Nespresso machine which I later found was due to it simply being off. Clearly software engineers should never try to use hardware. At least after an evening of drinking. </span></p> <p><span style="font-weight:400;">Wendy’s stall was piled high with swag including new Bailador (Perl 6 dancer like framework) stickers, a Shadowcat booklet about Perl 6 and the new O’Reilly “Thinking in Perl 6″. Unfortunately she had sold out of Moritz’s book “Perl 6 Fundamentals” (although there was a sample display copy present). Thankfully later that morning I discovered I had a £3 credit on Google Play Books so I bought the ebook on my phone.</span></p> <p><span style="font-weight:400;">The conference started early with Damian Conway’s Three Little Words. These were “has”, “class” and “method” from Perl 6 which he liked so much that he had added them to Perl 5 with his “Dios” – “Declarative Inside-Out Syntax” module. PPI wasn’t fast enough so he had to replace it with a 50,000 character regex PPR. Practical everyday modules mentioned included Regexp::Optimizer and Test::Expr. If the video doesn’t appear shortly on youtube a version of his talk dating from a few weeks earlier is available at </span><a href="https://www.youtube.com/watch?v=ob6YHpcXmTg"><span style="font-weight:400;">https://www.youtube.com/watch?v=ob6YHpcXmTg</span></a></p> <p><span style="font-weight:400;">Jonathan Worthington returned with his Perl 6 talk on “How does deoptimization help us go faster?” giving us insight into why Perl 6 was slow at the Virtual Machine level (specifically MoarVM). Even apparently simple and fast operations like indexing an array were slow due to powerful abstractions, late binding and many levels of Multiple Dispatch. In short the flexibility and power of such an extensible language also led to slowness due to the complexity of code paths. The AST optimizer helped with this at compile time but itself took time and it could be better to do this at a later compile time (like Just In Time). Even with a simple program reading lines from a file it was very hard to determine statically what types were used (even with type annotations) and whether it was worth optimizing (since the file could be very short).</span></p> <p><span style="font-weight:400;">The solution to these dynamic problems was also dynamic but to see what was happening needed cheap logging of execution which was passed to another thread. This logging is made visible by setting the environment variable MVM_SPESH_LOG to a filename. Better tooling for this log would be a good project for someone.</span></p> <p><span style="font-weight:400;">For execution planning we look for hot (frequently called) code, long blocks of bytecode (slow to run) and consider how many types are used (avoiding “megamorphic” cases with many types which needs many versions of code). Also analysis of the code flow between different code blocks and SSA. Mixins made the optimization particularly problematic.</span></p> <p><span style="font-weight:400;">MoarVM’s Spesh did statistical analysis of the code in order to rewrite it in faster, simpler ways. Guards (cheap check for things like types) were placed to catch cases where it got it wrong and if these were triggered (infrequently) it would deoptimize as well, hence the counterintuitive title since “Deoptimization enables speculation” The slides are at </span><a href="http://jnthn.net/papers/2017-spw-deopt.pdf"><span style="font-weight:400;">http://jnthn.net/papers/2017-spw-deopt.pdf</span></a><span style="font-weight:400;"> with the video at </span><a href="https://www.youtube.com/watch?v=3umNn1KnlCY"><span style="font-weight:400;">https://www.youtube.com/watch?v=3umNn1KnlCY</span></a><span style="font-weight:400;"> The older and more dull witted of us (including myself) might find the latter part of the video more comprehensible at 0.75 Youtube speed.</span></p> <p><span style="font-weight:400;">After a superb multi-course lunch (the food was probably the best I’d had at any Perl event) we returned promptly to hear Damian talk of “Everyday Perl 6”. He pointed out that it wasn’t necessary to code golf obfuscated extremes of Perl 6 and that the average Perl 5 programmer would see many things simpler in Perl 6. Also a rewrite from 5 to 6 might see something like 25% fewer lines of code since 6 was more expressive in syntax (as well as more consistent) although performance problems remained (and solutions in progress as the previous talk had reminded us).</span></p> <p><span style="font-weight:400;">Next Liz talked of a “gross” (in the numerical sense of 12 x 12 rather than the American teen sense) of Perl 6 Weeklies as she took us down memory lane to 2014 (just about when MoarVM was launched and when unicode support was poor!) with some selected highlights and memories of Perl 6 developers of the past (and hopefully future again!). Her talk was recorded at </span><a href="https://www.youtube.com/watch?v=418QCTXmvDU"><span style="font-weight:400;">https://www.youtube.com/watch?v=418QCTXmvDU</span></a></p> <p><img data-attachment-id="46" data-permalink="https://stmuk.wordpress.com/newton/" data-orig-file="https://stmuk.wordpress.com/wp-content/uploads/2017/08/newton.jpg" data-orig-size="3264,2448" data-comments-opened="1" data-image-meta="{"aperture":"2.2","credit":"","camera":"iPhone 6","caption":"","created_timestamp":"1503676921","copyright":"","focal_length":"4.15","iso":"50","shutter_speed":"0.03030303030303","title":"","orientation":"1","latitude":"46.294861111111","longitude":"7.0555111111111"}" data-image-title="newton" data-image-description="" data-image-caption="" data-medium-file="https://stmuk.wordpress.com/wp-content/uploads/2017/08/newton.jpg?w=300" data-large-file="https://stmuk.wordpress.com/wp-content/uploads/2017/08/newton.jpg?w=1024" class="alignnone size-full wp-image-46" src="https://stmuk.wordpress.com/wp-content/uploads/2017/08/newton.jpg?w=1100" alt="newton" srcset="https://stmuk.wordpress.com/wp-content/uploads/2017/08/newton.jpg 3264w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/newton.jpg?w=150&h=113 150w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/newton.jpg?w=300&h=225 300w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/newton.jpg?w=768&h=576 768w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/newton.jpg?w=1024&h=768 1024w" sizes="(max-width: 3264px) 100vw, 3264px" /></p> <p><span style="font-weight:400;">Cal then spoke of Perl 6 maths which he thought was good with its Rats and FatRats but not quite good enough and his ideas of fixing it. On the following day he showed us he had started some TDD work on TrimRats. He also told us that Newton’s Method wasn’t very good but generated a pretty fractal. See </span><a href="https://www.youtube.com/watch?v=3na_Cx-anvw"><span style="font-weight:400;">https://www.youtube.com/watch?v=3na_Cx-anvw</span></a></p> <p><span style="font-weight:400;">Lee spoke about how to detect Perl 5 memory leaks with various CPAN modules and his examples are at </span><a href="https://github.com/leejo/Perl_memory_talk"><span style="font-weight:400;">https://github.com/leejo/Perl_memory_talk</span></a></p> <p><span style="font-weight:400;">The day finished with Lightning Talks and a barbecue at givengain — a main sponsor.</span></p> <p>On the second day I noticed the robotic St Bernards dog in a tourist shop window had come to life.</p> <p><img data-attachment-id="44" data-permalink="https://stmuk.wordpress.com/dog1/" data-orig-file="https://stmuk.wordpress.com/wp-content/uploads/2017/08/dog1.gif" data-orig-size="600,338" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="dog1" data-image-description="" data-image-caption="" data-medium-file="https://stmuk.wordpress.com/wp-content/uploads/2017/08/dog1.gif?w=300" data-large-file="https://stmuk.wordpress.com/wp-content/uploads/2017/08/dog1.gif?w=600" class="alignnone size-full wp-image-44" src="https://stmuk.wordpress.com/wp-content/uploads/2017/08/dog1.gif?w=1100" alt="dog1" srcset="https://stmuk.wordpress.com/wp-content/uploads/2017/08/dog1.gif 600w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/dog1.gif?w=150&h=85 150w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/dog1.gif?w=300&h=169 300w" sizes="(max-width: 600px) 100vw, 600px" /></p> <p><span style="font-weight:400;">Damian kicked off the talks with my favourite of his talks, “Standing on the Shoulders of Giants”, starting with the Countess of Lovelace and her Bernoulli number program. This generated a strange sequence with many zeros. The Perl 6 version since it used rational numbers not floating point got the zeros right whereas the Perl 5 version initially suffered from floating point rounding errors (which are fixable).</span></p> <p><span style="font-weight:400;">Among other things he showed us how to define a new infix operator in Perl 6. He also showed us a Perl 6 sort program that looked exactly like LISP even down to the Lots of Irritating Superfluous Parentheses. I think this was quicksort (he certainly showed us a picture of Sir Tony Hoare at some point). Also a very functional (Haskell-like) equivalent with heavy use of P6 Multiple Dispatch. Also included was demonstration of P6 “before” as a sort of typeless/multi-type comparison infix. Damian then returned to his old favourite of Quantum Computing.</span></p> <p><span style="font-weight:400;">My mind and notes got a bit jumbled at this point but I particularly liked the slide that explained how factorisation could work by observing the product of possible inputs since this led to a collapse that revealed the factors. To do this on RSA etc., of course, needs real hardware support which probably only the NSA and friends have (?). Damian’s code examples are at </span><a href="http://www.bit.do/Perl6SOG"><span style="font-weight:400;">http://www.bit.do/Perl6SOG</span></a><span style="font-weight:400;"> with an earlier version of his talk at </span><a href="https://www.youtube.com/watch?v=Nq2HkAYbG5o"><span style="font-weight:400;">https://www.youtube.com/watch?v=Nq2HkAYbG5o</span></a><span style="font-weight:400;"> Around this point there was a road race of classic cars going on outside up the main road into the village and there were car noises in the background that strangely were more relaxing than annoying.</span></p> <p><img loading="lazy" data-attachment-id="57" data-permalink="https://stmuk.wordpress.com/2017/08/30/swiss-perl-workshop-2017/file_000/" data-orig-file="https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000.jpeg" data-orig-size="3264,2448" data-comments-opened="1" data-image-meta="{"aperture":"2.2","credit":"","camera":"iPhone 6","caption":"","created_timestamp":"1503771021","copyright":"","focal_length":"4.15","iso":"32","shutter_speed":"0.004739336492891","title":"","orientation":"1","latitude":"46.294783333333","longitude":"7.0559888888889"}" data-image-title="File_000" data-image-description="" data-image-caption="" data-medium-file="https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000.jpeg?w=300" data-large-file="https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000.jpeg?w=1024" class="alignnone size-full wp-image-57" src="https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000.jpeg?w=1100" alt="File_000" srcset="https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000.jpeg 3264w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000.jpeg?w=150&h=113 150w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000.jpeg?w=300&h=225 300w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000.jpeg?w=768&h=576 768w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000.jpeg?w=1024&h=768 1024w" sizes="(max-width: 3264px) 100vw, 3264px" /></p> <p><span style="font-weight:400;">After Quantum Chaos Paul Johnson brought us all back down to ground with an excellent practical talk on modernising legacy Perl 5 applications based on his war stories. Hell, of course, is “Other People’s Code”, often dating from Perl’s early days and lacking documentation and sound engineering.</span></p> <p><span style="font-weight:400;">Often the original developers had long since departed or, in the worse cases, were still there. Adding tests and logging (with stack traces) were particularly useful. As was moving to git (although its steep learning curve meant mentoring was needed) and handling CPAN module versioning with pinto. Many talks had spoken of the Perl 6 future whereas this spoke of the Perl 5 past and present and the work many of us suffer to pay the bills. It’s at </span><a href="https://www.youtube.com/watch?v=4G5EaUNOhR0"><span style="font-weight:400;">https://www.youtube.com/watch?v=4G5EaUNOhR0</span></a></p> <p><img loading="lazy" data-attachment-id="63" data-permalink="https://stmuk.wordpress.com/2017/08/30/swiss-perl-workshop-2017/file_000-1/" data-orig-file="https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000-1.jpeg" data-orig-size="3264,2448" data-comments-opened="1" data-image-meta="{"aperture":"2.2","credit":"","camera":"iPhone 6","caption":"","created_timestamp":"1503747734","copyright":"","focal_length":"4.15","iso":"50","shutter_speed":"0.03030303030303","title":"","orientation":"1","latitude":"46.294861111111","longitude":"7.0555111111111"}" data-image-title="File_000 (1)" data-image-description="" data-image-caption="" data-medium-file="https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000-1.jpeg?w=300" data-large-file="https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000-1.jpeg?w=1024" class="alignnone size-full wp-image-63" src="https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000-1.jpeg?w=1100" alt="File_000 (1)" srcset="https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000-1.jpeg 3264w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000-1.jpeg?w=150&h=113 150w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000-1.jpeg?w=300&h=225 300w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000-1.jpeg?w=768&h=576 768w, https://stmuk.wordpress.com/wp-content/uploads/2017/08/file_000-1.jpeg?w=1024&h=768 1024w" sizes="(max-width: 3264px) 100vw, 3264px" /></p> <p><span style="font-weight:400;">Jonathan then spoke of reactive distributed software. A distributed system is an async one where “Is it working?” means “some of it is working but we don’t know which bits”. Good OO design is “tell don’t ask” — you tell remote service to do something for you and not parse the response and do it yourself thus breaking encapsulation. This is particularly important in building well designed distributed systems since otherwise the systems are less responsive and reliable. Reactive (async) works better for distributed software than interactive (blocking or sync).</span></p> <p><span style="font-weight:400;">We saw a table that used a Perl 6 promise for one value and a supply for many values for reactive (async) code and the equivalent (one value) and a Perl 6 Seq for interactive code. A Supply could be used for pub/sub and the Observer Pattern. A Supply could either be live (like broadcast TV) or, for most Perl 6 supplies, on-demand (like Netflix). Then samples of networking (socket) based code were discussed including a web client, web server and SSH::LibSSH (async client bindings often very useful in practical applications like port forwarding)</span></p> <p><a href="https://github.com/jnthn/p6-ssh-libssh"><span style="font-weight:400;">https://github.com/jnthn/p6-ssh-libssh</span></a></p> <p><span style="font-weight:400;">Much of the socket code had a pattern of “react { whenever {” blocks with “whenever” as a sort of async loop.He then moved on from sockets to services (using a Supply pipeline) and amazed us by announcing the release of “cro”, a microservices library that even supports HTTP/2 and Websockets, at </span><a href="http://mi.cro.services"><span style="font-weight:400;">http://mi.cro.services</span></a><span style="font-weight:400;">/. This is installable using Perl 6 by “zef install –/test cro”.</span></p> <p><span style="font-weight:400;">Slides at </span><a href="http://jnthn.net/papers/2017-spw-sockets-services.pdf"><span style="font-weight:400;">http://jnthn.net/papers/2017-spw-sockets-services.pdf</span></a><span style="font-weight:400;"> and video at </span><a href="https://www.youtube.com/watch?v=6CsBDnTUJ3A"><span style="font-weight:400;">https://www.youtube.com/watch?v=6CsBDnTUJ3A</span></a></p> <p><span style="font-weight:400;">Next Lee showed Burp Scanner which is payware but probably the best web vulnerabilities scanner. I wondered if anyone had dare run it on ACT or the hotel’s captive portal.</span></p> <p><span style="font-weight:400;">Wendy did some cheerleading in her “Changing Image of Perl”. An earlier version is at </span><a href="https://www.youtube.com/watch?v=Jl6iJIH7HdA"><span style="font-weight:400;">https://www.youtube.com/watch?v=Jl6iJIH7HdA</span></a></p> <p><span style="font-weight:400;">Sue’s talk was “Spiders, Gophers, Butterflies” although the latter were mostly noticeably absent. She promises me that a successor version of the talk will use them more extensively. Certainly any Perl 6 web spidering code is likely to fit better on one slide than the Go equivalent.</span></p> <p><span style="font-weight:400;">During the lightning talks Timo showed us a very pretty Perl 6 program using his SDL2::Raw to draw an animated square spiral with hypnotic colour cycling type patterns. Also there was a talk by the author about </span><a href="https://bifax.org/bif/"><span style="font-weight:400;">https://bifax.org/bif/</span></a><span style="font-weight:400;">— a distributed bug tracking system (which worked offline like git).</span></p> <p><span style="font-weight:400;">Later in the final evening many of us ate and chatted in another restaurant where we witnessed a dog fight being narrowly averted and learnt that Wendy didn’t like Perl 5’s bless for both technical and philosophical reasons.</span></p> <h2><a href="http://perlgeek.de/blog-en/perl-6/2017-020-ten-years.html">My Ten Years of Perl 6</h2></a> <p><h3>Published by Moritz Lenz on 2017-08-08T22:00:01</h3></p>  <p>Time for some old man's reminiscence. Or so it feels when I realize that I've spent more than 10 years involved with the Perl 6 community.</p> <h2>How I Joined the Perl 6 Community</h2> <p>It was February 2007.</p> <p>I was bored. I had lots of free time (crazy to imagine that now...), and I spent some of that answering (Perl 5) questions on perlmonks. There was a category of questions where I routinely had no good answers, and those were related to threads. So I decided to play with threads, and got frustrated pretty quickly.</p> <p>And then I remember that a friend in school had told me (about four years earlier) that there was this Perl 6 project that wanted to do concurrency <em>really</em> well, and even automatically parallelize some stuff. And this was some time ago, maybe they had gotten anywhere?</p> <p>So I searched the Internet, and found out about Pugs, a Perl 6 compiler written in Haskell. And I wanted to learn more, but some of the links to the presentations were dead. I <a href="https://irclog.perlgeek.de/perl6/2007-02-04#i_-55445">joined the #perl6 IRC channel to report the broken link</a>.</p> <p>And within three minutes I got a "thank you" for the report, the broken links were gone, <em>and</em> I had an invitation for a commit bit to the underlying SVN repo.</p> <p>I stayed.</p> <h2>The Early Days</h2> <p>Those were they wild young days of Perl 6 and Pugs. Audrey Tang was pushing Pugs (and Haskell) very hard, and often implemented a feature within 20 minutes after somebody mentioned it. Things were unstable, broken often, and usually fixed quickly. No idea was too crazy to be considered or even implemented.</p> <p>We had bots that evaluated Perl 6 and Haskell code, and gave the result directly on IRC. There were lots of cool (and sometimes somewhat frightening) automations, for example for inviting others to the SVN repo, to the shared hosting system (called feather), for searching SVN logs and so on. Since git was still an obscure and very unusable, people tried to use <a href="https://en.wikipedia.org/wiki/SVK">SVK</a>, an attempt to implement a decentralized version control system on top of of the SVN protocol.</p> <p>Despite some half-hearted attempts, I didn't really make inroads into compiler developments. Having worked with neither Haskell nor compilers before proved to be a pretty steep step. Instead I focused on some early modules, documentation, tests, and asking and answering questions. When the IRC logger went offline for a while, I wrote my own, which is still in use today.</p> <p>I felt at home in that IRC channel and the community. When the community asked for mentors for the Google Summer of Code project, I stepped up. The project was a revamp of the Perl 6 test suite, and to prepare for mentoring task, I decided to dive deeper. That made me the maintainer of the test suite.</p> <h2>Pet Projects</h2> <p>I can't recount a full history of Perl 6 projects during that time range, but I want to reflect on some projects that I considered my pet projects, at least for some time.</p> <ul> <li>In early 2008 I started to contribute some patches to <a href="https://perl6.org/">perl6.org</a>. Over time I became frustrated with the outdated content on many Perl 6 websites, so I created perl6-projects.org, which only contained links to up-to-date resources. Over time, people with an actual sense of design contribute a nice layout. Later we switched perl6.org over to the content from perl6-projects.org. Its structure today is still mostly that of a link collection.</li> <li>In September 2008 I started with the <a href="https://perlgeek.de/en/article/5-to-6">Perl 5 to 6</a> blog series, which was the de facto resource for learning Perl 6 for some years.</li> <li>In 2009 I wrote <a href="https://github.com/moritz/json/">JSON::Tiny</a>, the first pure-Perl 6 JSON parser. I mostly did it to become familiar with grammars, and because the book "Real World Haskell" also developed a JSON module.</li> <li>May 2010 kicked off a program to make <a href="https://perlgeek.de/blog-en/perl-6/contribute-now-announce.html">contributing to Perl 6 more accessible</a>.</li> <li>September 2010: the Pugs SVN repository became unmaintainable. It had accumulated the official Perl 6 test suite, some documentation, modules, at least one IRC bot and a proverbial metric ton of other stuff. In coordination with #perl6, I split it up into a collection of git repositories and put them on GitHub. A decision we haven't regretted yet. </li> <li>In 2012 and 2016 I was part of the organization team of the German Perl Workshop in Erlangen and Nürnberg, which both featured several Perl 6 talks.</li> <li>2012 saw the birth of <a href="https://docs.perl6.org/">the documentation project</a>. Initially I was nearly the sole contributor, and I'm very happy that, over time, others have taken over.</li> <li>Between 2014 and 2015, our community server "feather" EOLed, so the Perl 6 community <a href="https://perlgeek.de/blog-en/perl-6/2014-community-server-funding.html">funded a new server</a>, which my <a href="https://www.noris.de/">employer, noris network AG</a> kindly hosts for free in their data center in Nürnberg, Germany.</li> <li>In 2016 I started writing <a href="https://leanpub.com/perl6">Perl 6 by Example</a>, which Apress later adopted and published as <a href="https://www.apress.com/us/book/9781484228982">Perl 6 Fundamentals</a>.</li> <li>Currently I'm working on a treatise on <a href="https://leanpub.com/perl6regex">Perl 6 Regexes and Grammars</a>, which Apress will also publish.</li> </ul> <p>It is not quite clear from this (very selected) timeline, but my Perl 6 related activity dropped around 2009 or 2010. This is when I started to work full time, moved in with my girlfriend (now wife), and started to plan a family.</p> <h2>Relationships</h2> <p>The technologies and ideas in Perl 6 are fascinating, but that's not what kept me. I came for the technology, but stayed for the community.</p> <p>There were and are many great people in the Perl 6 community, some of whom I am happy to call my friends. Whenever I get the chance to attend a Perl conference, workshop or hackathon, I find a group of Perl 6 hackers to hang out and discuss with, and generally have a good time.</p> <p>Four events stand out in my memory. In 2010 I was invited to the Open Source Days in Copenhagen. I missed most of the conference, but spent a day or two with (if memory serve right) Carl Mäsak, Patrick Michaud, Jonathan Worthington and Arne Skjærholt. We spent some fun time trying to wrap our minds around macros, the intricacies of human and computer language, and Japanese food. (Ok, the last one was easy). Later the same year, I attended my first YAPC::EU in Pisa, and met most of the same crowd again -- this time joined by Larry Wall, and over three or four days. I still fondly remember the Perl 6 hallway track from that conference. And 2012 I flew to Oslo for a Perl 6 hackathon, with a close-knit, fabulous group of Perl 6 hackers. Finally, the <a href="http://blogs.perl.org/users/liz/2012/08/as-some-of-you-may.html">Perl Reunification Summit</a> in the beautiful town of Perl in Germany, which brought together Perl 5 and Perl 6 hackers in a very relaxed atmosphere.</p> <p>For three of these four events, different private sponsors from the Perl and Perl 6 community covered travel and/or hotel costs, with their only motivation being meeting folks they liked, and seeing the community and technology flourish.</p> <h2>The Now</h2> <p>The Perl 6 community has evolved a lot over the last ten years, but it is still a very friendly and welcoming place. There are lots of "new" folks (where "new" is everybody who joined after me, of course :D), and a surprising number of the old guard still hang around, some more involved, some less, all of them still very friendly and supportive</p> <h2>The Future</h2> <p>I anticipate that my family and other projects will continue to occupy much of my time, and it is unlikely that I'll be writing another Perl 6 book (after the one about regexes) any time soon. But the Perl 6 community has become a second home for me, and I don't want to miss it.</p> <p>In the future, I see myself supporting the Perl 6 community through infrastructure (community servers, IRC logs, running IRC bots etc.), answering questions, writing a blog article here and there, but mostly empowering the "new" guard to do whatever they deem best.</p> <h2><a href="http://perlgeek.de/blog-en/perl-6/2017-018-perl6-fundamentals-available-for-purchase.html">Perl 6 Fundamentals Now Available for Purchase</h2></a> <p><h3>Published by Moritz Lenz on 2017-07-21T22:00:01</h3></p>  <p>After about nine months of work, my book <a href="https://www.apress.com/us/book/9781484228982">Perl 6 Fundamentals</a> is now available for purchase on <a href="https://www.apress.com/us/book/9781484228982">apress.com</a> and <a href="http://www.springer.com/us/book/9781484228982">springer.com</a>.</p> <p>The ebook can be purchased right now, and comes in the epub and PDF formats (with watermarks, but DRM free). The print form can be pre-ordered from <a href="https://www.amazon.com/dp/1484228987">Amazon</a>, and will become ready for shipping in about a week or two.</p> <p>I will make a copy of the ebook available for free for everybody who purchased an earlier version, "Perl 6 by Example", from LeanPub.</p> <p>The book is aimed at people familiar with the basics of programming; prior Perl 5 or Perl 6 knowledge is not required. It features a practical example in most chapters (no mammal hierarchies or <code>class Rectangle</code> inheriting from <code>class Shape</code>), ranging from simple input/output and text formatting to plotting with python's matplotlib libraries. Other examples include date and time conversion, a Unicode search tool and a directory size visualization.</p> <p>I use these examples to explain subset of Perl 6, with many pointers to more documentation where relevant. Perl 6 topics include the basic lexicographic structure, testing, input and output, multi dispatch, object orientation, regexes and grammars, usage of modules, functional programming and interaction with python libraries through <code>Inline::Python</code>.</p> <p>Let me finish with Larry Wall's description of this book, quoted from his foreword:</p> <blockquote> It's not just a reference, since you can always find such materials online. Nor is it just a cookbook. I like to think of it as an extended invitation, from a well-liked and well-informed member of our circle, to people like you who might want to join in on the fun. Because joy is what's fundamental to Perl. The essence of Perl is an invitation to love, and to be loved by, the Perl community. It's an invitation to be a participant of the gift economy, on both the receiving and the giving end. </blockquote> <h2><a href="http://perlgeek.de/blog-en/perl-6/2017-018-losing-named-and-orientation.html">The Loss of Name and Orientation</h2></a> <p><h3>Published by Moritz Lenz on 2017-07-10T22:00:01</h3></p>  <p>The <a href="https://perl6.party/post/The-Hot-New-Language-Named-Rakudo">Perl 6 naming debate has started again</a>. And I guess with good reason. Teaching people that Perl 6 is <i>a</i> Perl, but not <i>the</i> Perl requires too much effort. Two years ago, I didn't believe. Now you're reading a tired man's words.</p> <p>I'm glad that this time, we're not discussing giving up the "Perl" brand, which still has very positive connotations in my mind, and in many other minds as well.</p> <p>And yet, I can't bring myself to like "Rakudo Perl 6" as a name. There are two vary shallow reasons for that: Going from two syllables, "Perl six", to five of them, seems a step in the wrong direction. And two, I remember the days when the name was pretty young, and people would misspell it all the time. That seems to have abated, though I don't know why.</p> <p>But there's also a deeper reason, probably sentimental old man's reason. I remember the days when Pugs was actively developed, and formed the center of a vibrant community. When kp6 and SMOP and all those weird projects were around. And then, just when it looked like there was only a single compiler was around, Stefan O'Rear conjured up niecza, almost single-handedly, and out of thin air. Within months, it was a viable Perl 6 compiler, that people on #perl6 readily recommended.</p> <p>All of this was born out of the vision that Perl 6 was a language with no single, preferred compiler. Changing the language name to include the compiler name means abandoning this vision. How can we claim to welcome alternative implementations when the commitment to one compiler is right in the language name?</p> <p>However I can't weigh this loss of vision against a potential gain in popularity. I can't decide if it's my long-term commitment to the name "Perl 6" that makes me resent the new name, or valid objections. The lack of vision mirrors my own state of mind pretty well.</p> <p>I don't know where this leaves us. I guess I must apologize for wasting your time by publishing this incoherent mess.</p> <h2><a href="http://perlgeek.de/blog-en/perl-6/2017-017-living-on-the-bleading-edge.html">Living on the (b)leading edge</h2></a> <p><h3>Published by Moritz Lenz on 2017-06-24T22:00:01</h3></p>  <p>Perl 6 is innovative in many ways, and sometimes we don't fully appreciate all the implications, for good or for bad.</p> <p>There's one I stumbled upon recently: The use of fancy Unicode symbols for built-in stuff. In this case: the `.gist` output of Match objects. For example</p> <pre> <span class="synSpecial">my</span> <span class="synStatement">token</span> word {<span class="synConstant"> </span><span class="synSpecial">\w+</span><span class="synConstant"> </span>} <span class="synIdentifier">say</span> <span class="synSpecial">'</span><span class="synConstant">abc=def</span><span class="synSpecial">'</span> <span class="synStatement">~~</span> <span class="synSpecial">/<</span>word<span class="synSpecial">></span><span class="synConstant"> </span><span class="synSpecial">'</span><span class="synConstant">=</span><span class="synSpecial">'</span><span class="synConstant"> </span><span class="synSpecial"><</span>word<span class="synSpecial">>/</span><span class="synStatement">;</span> </pre> produces this output: <pre> ｢abc=def｣ word => ｢abc｣ word => ｢def｣ </pre> <p>And that's where the problems start. In my current <a href="https://leanpub.com/perl6regex">quest to write a book on Perl 6 regexes</a>, I noticed that the PDF that LeanPub generates from my Markdown sources don't correctly display those pesky ｢｣ characters, which are <pre> $ uni -c ｢｣｢ - U+0FF62 - HALFWIDTH LEFT CORNER BRACKET ｣ - U+0FF63 - HALFWIDTH RIGHT CORNER BRACKET </pre> <p>When I copied the text from the PDF and pasted into my editor, they showed up correctly, which indicates that the characters are likely missing from the monospace font.</p> <p>The toolchain allows control over the font used for displaying code, so I tried all the monospace fonts that were available. I tried them in alphabetical order. Among the earlier fonts I tried was Deja Vu Sans Mono, which I use in my terminal, and which hasn't let me down yet. No dice. I arrived at <a href"https://www.google.com/get/noto/">Noto</a>, a font designed to cover all Unicode codepoints. And it didn't work either. So it turns out these two characters are part of some Noto Sans variants, but not of the monospace font.</p> <p>My terminal, and even some font viewers, use some kind of fallback where they use glyphs from other fonts to render missing characters. The book generation toolchain does not.</p> <p>The Google Group for Leanpub was <a href="https://groups.google.com/forum/#!topic/leanpub/bqVazrui-O4">somewhat helpful:</a> if I could recommend an Open Source mono space font that fit my needs, they'd likely include it in their toolchain.</p> <p>So I searched and searched, learning more about fonts than I wanted to know. My circle of geek friends came up with several suggestions, one of them being <a href="https://github.com/be5invis/Iosevka">Iosevka</a>, which actually contains those characters. So now I wait for others to step up, either for LeanPub to include that font, or for <a href="https://github.com/googlei18n/noto-fonts/issues/939">the Noto maintainers to create a monospace variant of those characters</a> (and then LeanPub updating their version of the font).</p> <p>And all of that because Perl 6 was being innovative, and used two otherwise little-used characters as delimiters, in an attempt to avoid collisions between delimiters and content.</p> <p>(In the mean time I've replaced the two offending characters with ones that look similar. It means the example output is technically incorrect, but at least it's readable).</p> <h2><a href="https://blogs.perl.org/users/steve_mynott/2017/01/rakudo-star-past-present-and-future.html">Rakudo Star: Past Present and Future</h2></a> <p><h3>Published by Steve Mynott on 2017-01-02T14:07:31</h3></p> <p>At YAPC::EU 2010 in Pisa I received a business card with "Rakudo Star" and the<br /> date July 29, 2010 which was the date of the first release -- a week earlier<br /> with a countdown to 1200 UTC. I still have mine, although it has a tea stain<br /> on it and I refreshed my memory over the holidays by listening again to Patrick<br /> Michaud speaking about the launch of Rakudo Star (R*):</p> <p><a href="https://www.youtube.com/watch?v=MVb6m345J-Q">https://www.youtube.com/watch?v=MVb6m345J-Q<br /> </a></p> <p>R* was originally intended as first of a number of distribution releases (as<br /> opposed to a compiler release) -- useable for early adopters but not initially production<br /> Quality. Other names had been considered at the time like Rakudo Beta (rejected as<br /> sounding like "don't use this"!) and amusingly Rakudo Adventure Edition.<br /> Finally it became Rakudo Whatever and Rakudo Star (since * means "whatever"!).</p> <p>Well over 6 years later and we never did come up with a better name although there<br /> was at least one IRC conversation about it and perhaps "Rakudo Star" is too<br /> well established as a brand at this point anyway. R* is the Rakudo compiler, the main docs, a module installer, some modules and some further docs.</p> <p>However, one radical change is happening soon and that is a move from panda to<br /> zef as the module installer. Panda has served us well for many years but zef is<br /> both more featureful and more actively maintained. Zef can also install Perl<br /> 6 modules off CPAN although the CPAN-side support is in its early days. There<br /> is a zef branch (pull requests welcome!) and a tarball at:</p> <p><a href="http://pl6anet.org/drop/rakudo-star-2016.12.zef-beta2.tar.gz">http://pl6anet.org/drop/rakudo-star-2016.12.zef-beta2.tar.gz<br /> </a></p> <p>Panda has been patched to warn that it will be removed and to advise the use of<br /> zef. Of course anyone who really wants to use panda can reinstall it using zef<br /> anyway.</p> <p>The modules inside R* haven't changed much in a while. I am considering adding<br /> DateTime::Format (shown by ecosystem stats to be widely used) and<br /> HTTP::UserAgent (probably the best pure perl6 web client library right now).<br /> Maybe some modules should also be removed (although this tends to be more<br /> controversial!). I am also wondering about OpenSSL support (if the library is<br /> available).</p> <p>p6doc needs some more love as a command line utility since most of the focus<br /> has been on the website docs and in fact some of these changes have impacted<br /> adversely on command line use, eg. under Windows cmd.exe "perl 6" is no longer<br /> correctly displayed by p6doc. I wonder if the website generation code should be<br /> decoupled from the pure docs and p6doc command line (since R* has to ship any<br /> new modules used by the website). p6doc also needs a better and faster search<br /> (using sqlite?). R* also ships some tutorial docs including a PDF generated from perl6intro.com.<br /> We only ship the English one and localisation to other languages could be<br /> useful.</p> <p>Currently R* is released roughly every three months (unless significant<br /> breakage leads to a bug fix release). Problems tend to happen with the<br /> less widely used systems (Windows and the various BSDs) and also with the<br /> module installers and some modules. R* is useful in spotting these issues<br /> missed by roast. Rakudo itself is still in rapid development. At some point a less frequently<br /> updated distribution (Star LTS or MTS?) will be needed for Linux distribution<br /> packagers and those using R* in production). There are also some question<br /> marks over support for different language versions (6.c and 6.d).</p> <p>Above all what R* (and Rakudo Perl 6 in general) needs is more people spending<br /> more time working on it! JDFI! Hopefully this blog post might<br /> encourage more people to get involved with github pull requests. </p> <p><a href="https://github.com/rakudo/star">https://github.com/rakudo/star<br /> </a><br /> Feedback, too, in the comments below is actively encouraged.</p> <p><br /> </p> <h2><a href="https://blogs.perl.org/users/steve_mynott/2016/11/rakudo-star-201611-release-candidate.html">Rakudo Star 2016.11 Release Candidate </h2></a> <p><h3>Published by Steve Mynott on 2016-11-20T14:01:22</h3></p> <p>There is a Release Candidate for Rakudo Star 2016.11 (currently RC2) available at</p> <p><a href="http://pl6anet.org/drop/">http://pl6anet.org/drop/<br /> </a><br /> This includes binary installers for Windows and Mac.</p> <p>Usually Star is released about every three months but last month's release didn't include a Windows installer so there is another release.</p> <p>I'm hoping to release the final version next weekend and would be grateful if people could try this out on as many systems as possible. </p> <p>Any feedback email steve *dot* mynott *at* gmail *dot* com</p> <p>Full draft announce at</p> <p><a href="https://github.com/rakudo/star/blob/master/docs/announce/2016.11.md">https://github.com/rakudo/star/blob/master/docs/announce/2016.11.md<br /> </a></p> <h2><a href="http://stevemynott.blogspot.com/2016/10/there-is-release-candidate-for-rakudo.html">Rakudo Star 2016.10 Release Candidate</h2></a> <p><h3>Published by Steve on 2016-10-16T06:10:00</h3></p> There is a Release Candidate for Rakudo Star 2016.10 (currently RC0) available at<br /> <br /> <div class="p1"> <span class="s1"><a href="http://pl6anet.org/drop/">http://pl6anet.org/drop/</a></span></div> <div class="p1"> <br /></div> <div class="p1"> This should be quite a bit faster than previous releases and work better on OpenBSD/FreeBSD than the previous release.</div> <div class="p1"> <br /></div> <div class="p1"> It also features "prove6" which is now used by Panda -- removing a run-time dependency on Perl 5. Although it still needs Perl 5 to build.</div> <div class="p1"> <br /></div> <div class="p1"> I'm hoping to release the final version next weekend (Oct 21st) and would be grateful if people could try this out on as many systems as possible (eg. exotic systems like Solaris-like ones and Windows!) </div> <div class="p1"> <br /></div> <div class="p1"> Full draft announce at</div> <div class="p1"> <a href="https://www.blogger.com/goog_1218437599"><br /></a></div> <div class="p1"> <a href="https://github.com/rakudo/star/blob/master/docs/announce/2016.10.md">https://github.com/rakudo/star/blob/master/docs/announce/2016.10.md</a></div> <div class="p1"> <br /></div> <div class="p1"> Note compiling under Windows is possible using the gcc which comes with Strawberry Perl and gmake running under cmd.exe.  Further instructions will be added (thanks to Christopher for feedback).<br /> <br /> Any feedback email steve *underscore* mynott *at* gmail *dot* com</div> <h2><a href="https://blogs.perl.org/users/steve_mynott/2016/08/you-wouldnt-believe-what-i-saw-at-yapceu.html">You Wouldn't BELIEVE what I saw at YAPC::EU!!!</h2></a> <p><h3>Published by Steve Mynott on 2016-08-28T18:57:57</h3></p> <p>We turned up in Cluj via Wizz Air to probably one of the best pre YAPC parties ever located on three levels on the rooftop of Evozon‎’s plush city centre offices. We were well supplied with excellent wine, snacks and the local Ursus beer and had many interesting conversations with old friends. </p> <p>On the first day Tux spoke about his Text::CSV modules for both Perl 5 and 6 on the first day and I did a short talk later in the day on benchmarking Perl 6. Only Nicholas understood my trainspotter joke slide with the APT and Deltic! Sadly my talk clashed with Lee J talking about Git which I wanted to see so I await the youtube version! Jeff G then spoke about Perl 6 and parsing languages such as JavaScript. Sadly I missed Leon T’s Perl 6 talk which I also plan on watching on youtube. Tina M gave an excellent talk on writing command line tools. She also started the lightning talks with an evangelical talk about how tmux was better than screen. Geoffrey A spoke about configuring sudo to run restricted commands in one directory which seemed a useful technique to me. Dave C continued his conference tradition of dusting off his Perl Vogue cover and showing it again. The age of the image was emphasised by the amazingly young looking mst on it. And Stefan S ended with a call for Perl unification. </p> <p>The main social event was in the courtyard of the main museum off the central square with free food and beer all evening and an impressive light show on the slightly crumbling facade. There were some strange chairs which resembled cardboard origami but proved more comfortable than they looked when I was finally able to sit in one. The quality of the music improved as the evening progressed (or maybe the beer helped) I was amazed to see Perl Mongers actually dancing apparently inspired by the younger Cluj.pm members. </p> <p>Day Two started with Sawyer’s State of the Velociraptor‎ which he had, sensibly, subcontracted to various leading lights of the Perl Monger community. Sue S (former London.pm leader) was up first with a short and sweet description of London.pm. Todd R talked about Houston.pm. Aaron Crane spoke about the new improved friendlier p5p. Tina about Berlin.pm and the German Perl community site she had written back in the day. This new format worked very well and it was obvious Perl Mongers groups could learn much from each other. Max M followed with a talk about using Perl and ElasticSearch to index websites and documents and Job about accessibility.</p> <p>1505 had, from the perspective of London.pm, one of the most unfortunate scheduling clashes at YAPC::EU ever, with three titans of London.pm (all former leaders) battling for audience share. I should perhaps tread carefully here lest bias become apparent but the heavyweight Sue Spence was, perhaps treacherously, talking about Go in the big room and Dave Cross and Tom talking about Perl errors and HTML forms respectively in the other rooms. This momentous event should be reproducible by playing all three talks together in separate windows once they are available.</p> <p>Domm did a great talk on Postgres which made me keen to use this technology again. André W described how he got Perl 6 running on his Sailfish module phone while Larry did a good impression of a microphone stand. I missed most of Lance Wick’s talk but the bit I caught at the end made me eager to watch the whole thing.</p> <p>Guinevere Nell gave a fascinating lightning talk about agent based economic modelling. Lauren Rosenfield spoke of porting (with permission) a “Python for CS” book to perl 6. Lukas Mai described his journey from Perl to Rust. Lee J talked about photography before Sue encouraged people to break the London.pm website. Outside the talk rooms on their stall Liz and Wendy had some highly cool stuffed toy Camelia butterflies produced by the Beverly Hills Teddy Bear Company and some strange “Camel Balls” bubblegum. At the end of the day Sue cat herded many Mongers to eat at the Enigma Steampunk Bar in central Cluj with the cunning ploy of free beer money (recycled from the previous year’s Sherry money).</p> <p>The third day started with Larry’s Keynote in which photographs of an incredible American house “Fallingwater” and Chinese characters (including “arse rice”) featured heavily. Sweth C gave a fast and very useful introduction to swift. Nicholas C then confused a room of people for an hour with a mixture of real Perl 5 and 6 and an alternative timeline compete with T shirts. The positive conclusion was that even if the past had been different the present isn’t likely to have been much better for the Perl language family than it is now! Tom spoke about Code Review and Sawyer about new features in Perl 5.24. Later I heard Ilya talk about running Perl on his Raspberry PI Model B and increasing the speed of his application very significantly to compensate for its low speed! And we finished with lightning talks where we heard about the bug tracker OTRS (which was new to me), Job spoke about assistive tech and Nine asked us to ask our bosses for money for Perl development amongst several other talks. We clapped a lot in thanks, since this was clearly a particularly well organised YAPC::EU (due to Amalia and her team!) and left to eat pizza and fly away the next day. Some stayed to visit a salt mine (which looked most impressive from the pictures!) and some stayed longer due to Lufthansa cancelling their flights back!</p> <p><br /> </p> <h2><a href="https://blogs.perl.org/users/steve_mynott/2016/03/german-perl-workshop-2016.html">German Perl Workshop 2016</h2></a> <p><h3>Published by Steve Mynott on 2016-03-15T15:36:12</h3></p> <p>The meeting first night was in a large beer bar in the centre of Nuremberg. <br /> We went back to the Best Western to find a certain exPumpkin already resident in the bar.<br /> Despite several of the well named Bitburgers we managed to arrive at the<br /> conference venue on time the following morning. Since my knowledge of German was<br /> limited to a C grade 'O' Level last century my review talks will be mostly<br /> limited to English talks. Apologies in advance to those giving German talks<br /> (not unreasonable considering the country). Hopefully other blog posts will<br /> cover these. </p> <p>Masak spoke about the dialectic between planning (like physics) and chaos (like<br /> biology) in software development. </p> <p><a href="http://masak.org/carl/gpw-2016-domain-modeling/talk.pdf">http://masak.org/carl/gpw-2016-domain-modeling/talk.pdf</a></p> <p>Tobias gave a good beginners guide to Perl 6 in German and I was able to follow<br /> most of the slides since I knew more Perl 6 than German and even learnt a thing<br /> or two.</p> <p>After lunch Stefan told us he was dancing around drunk and naked on the turn of<br /> the 2000s and also about communication between Perl 6 and Perl 5 and back again<br /> via his modules Inline::Perl5 (from Perl 6) -- the most important take away<br /> being that "use Foo::Bar:from<Perl5>" can be used from Perl 6 and "use<br /> Inline::Perl6" from Perl 5. The modules built bridges like those built in the<br /> old school computer game "Lemmings".<br /> <a href="http://niner.name/talks/Perl%205%20and%20Perl%206%20-%20a%20great%20team/Perl%205%20and%20Perl%206%20-%20a%20great%20team.odp"><br /> http://niner.name/talks/Perl%205%20and%20Perl%206%20-%20a%20great%20team/Perl%205%20and%20Perl%206%20-%20a%20great%20team.odp</a></p> <p>Max told us (in German) about his Dancer::SearchApp search<br /> engine which has based on Elastic Search but I was able to follow along on the<br /> English version of his slides on the web. </p> <p><a href="http://corion.net/talks/dancer-searchapp/dancer-searchapp.en.html">http://corion.net/talks/dancer-searchapp/dancer-searchapp.en.html</a></p> <p>Sue got excited about this. Tina showed us some slides in Vim and her module<br /> to add command line tab completion to script arguments using zsh and bash. I<br /> wondered whether some of her code could be repurposed to add fish shell man<br /> page parsing autocompletion to zsh. She also had a good lightening talk about<br /> Ingy's command line utility for github.<br /> <a href="https://github.com/perlpunk/myslides/tree/master/app-spec"><br /> https://github.com/perlpunk/myslides/tree/master/app-spec</a></p> <p>Second day started early with Moritz talking about Continuous Delivery which<br /> could mean just delivering to a staging server. He was writing a book about it<br /> at deploybook.com with slides at:</p> <p><a href="https://deploybook.com/talks/gpw2016-continuous-delivery.pdf">https://deploybook.com/talks/gpw2016-continuous-delivery.pdf</a></p> <p>Salve wanted us to write elegant code as a reply to the Perl Jam guy at CCC in<br /> a self confessed "rant".</p> <p>Sawyer described writing Ref::Util to optimise things like "ref $foo" in a<br /> Hardcore Perl 5 XS/Core talk and Masak told us about his little 007 language<br /> written in Perl 6 as a proof of concept playroom for future Perl 6 extended<br /> macro support and demonstrated code written over lunch in support of this.</p> <p><a href="http://masak.org/carl/gpw-2016-big-hairy-yaks/talk.pdf">http://masak.org/carl/gpw-2016-big-hairy-yaks/talk.pdf</a></p> <p>Stefan gave a great talk about CURLI and explained the complexity of what was<br /> intended.</p> <p><a href="http://niner.name/talks/A%20look%20behind%20the%20curtains%20-%20module%20loading%20in%20Perl%206/Module%20loading%20in%20Perl%206.pdf">http://niner.name/talks/A%20look%20behind%20the%20curtains%20-%20module%20loading%20in%20Perl%206/Module%20loading%20in%20Perl%206.pdf</a></p> <p>I gave my talk on "Simple Perl 6 Fractals and Concurrency" on Friday. It<br /> started badly with AV issues my side but seemed well received. It was useful<br /> speaking with people about it and I managed to speed things up *after* the talk<br /> and I should have new material for a 2.0 version. </p> <p>There were very good talks on extracting data from PDFs and writing JSON apis.<br /> <a href="https://github.com/mickeyn/PONAPI"><br /> https://github.com/mickeyn/PONAPI </a></p> <p>looked very interesting and would have saved me much coding at a recent job.</p> <p>There were some great lightening talks at th end of the day. Sawyer wanted<br /> people to have English slides and gave his talk in Hebrew to stress this.<br /> Things ended Friday night with great food and beer in a local bar.<br /> </p> <h2><a href="https://blogs.perl.org/users/steve_mynott/2016/02/fosdem-2016.html">FOSDEM 2016</h2></a> <p><h3>Published by Steve Mynott on 2016-02-02T19:33:44</h3></p> <p>To me It seemed a particularly good FOSDEM for both for Perl5/6 and<br /> other talks although very crowded as usual and I didn't see the usual<br /> *BSD or Tor stalls. I was stuck by the statistic that there were<br /> about 500 speakers from many thousands of people so of the order of<br /> one speaker per tens of attendees which is very high.</p> <p>Videos are already starting to appear at</p> <p><a href="http://video.fosdem.org/2016/">http://video.fosdem.org/2016/</a></p> <p>On Saturday I started with Poettering and systemd which was a keynote<br /> and perhaps a little disappointing since he usually is a better<br /> speaker and the audio was a little indistinct. systemd had won being<br /> used by all distros except gentoo and slackware. They were now working<br /> on a dns resolver component which supported DNSSEC although in<br /> practice validating signed zone files would slow down browsing and<br /> currently only 2% of websites had it activated. He didn't mention<br /> strong criticisms of its security by crypto experts such as DJB.</p> <p>The most amusing talk was Stark's retro running of Postgres on<br /> NetBSD/VAX which exposed some obscure OS bugs and was livened up by a<br /> man in an impressive Postgres Elephant costume appearing. We later<br /> spoke to Mr Elephant who said he was both blind and very hot at the<br /> time. I then went to the Microkernel room to hear about GNU/Hurd<br /> progress from Thibault since this room is usually "OPEN" and he's an<br /> excellent speaker. I noticed even this obscure room was quite crowded<br /> as compared with previous years so I'd guess total attendees this year<br /> were high. He stressed the advantages of running device drivers in<br /> userspace as allowing more user "freedom" to mount fs etc. without<br /> root and improving kernel stability since the drivers could crash and<br /> restart without bringing down the kernel. In previous years he had<br /> talked of his DDE patches allowing linux 2.6 hardware drivers on Hurd<br /> and this year he was using the NetBSD Rump kernel under Hurd to add<br /> sound support with USB support promised. His demo was RMS singing his<br /> song on his Hurd laptop. The irony was he needed to use BSD code on a<br /> GNU/BSD/Hurd system to do it! There had been some work on X86-64 Hurd<br /> but it wasn't there yet since he needed more help from the community.<br /> I then saw some lightening talks (actually 20 mins long) including a<br /> good one on C refactoring.</p> <p>The Perl dinner on Saturday night featured the usual good food and<br /> conversation and the devroom was on Sunday. Ovid spoke about Perl 6<br /> and its advantages (such as being able to perform maths on floats<br /> correctly). I had a python guy sitting next to me who admitted he had<br /> never been to a Perl talk before so that was a success in reaching<br /> someone new. Will Braswell spoke next about his "Rperl" compiler<br /> which translated his own quite restricted subset (no regexps yet and<br /> no $_) of Perl 5 line by line into C++ in order to run some of the<br /> language shootups benchmarks (a graphical animation of planetary<br /> motion) at increased speed. I'd not seen Will before and he was an<br /> excellent speaker who left me more impressed than I'd expected and I<br /> hope he gets to YAPC::EU in the summer. I saw some non-Perl stuff<br /> next for variety including a good one on the Go debugger Delve which<br /> was aware of the go concurrency and could be used as a basic REPL. I<br /> returned to Perl to see Bart explain some surprisingly simple X86-64<br /> assembly language to do addition and ROT13 which he interfaced with<br /> Perl 6 using NativeCall (although it stuck me that the<br /> CPAN P5NCI module on Perl 5 would have also worked). <br /> Again an excellent talk and a good start to the a<br /> run of some of the best Perl talks I'd ever seen. Stevan Little's talk<br /> was one of the his most amusing ever and perl wasn't really dead.<br /> Sawyer also did an excellent promotion of Perl 5 targeted at the<br /> people who maybe hadn't used it since the early 2000s explaining what<br /> had changed. Liz finished with her autobiographical account of Perl<br /> development and some nice short Perl 6 examples. We all ate again in<br /> the evening together my only regrets being I'd missed the odd talk or<br /> two (which I should be able to watch on video).</p> <h2><a href="http://stevemynott.blogspot.com/2016/01/fosdem-2016-perl-dev-room-lineup.html">FOSDEM 2016 Perl Dev Room Lineup</h2></a> <p><h3>Published by Steve on 2016-01-09T13:32:00</h3></p> FOSDEM is a free two day conference in Brussels, Belgium on Jan 30th and 31st 2016.<br /> <br /> The FOSDEM 2016 schedule for the Perl Dev Room on the second day (the Sunday)  has now been announced at<br /> <br /> <a href="https://fosdem.org/2016/schedule/track/perl/">https://fosdem.org/2016/schedule/track/perl/</a><br /> <br /> From a Perl 6 perspective it includes  Ovid's "<i>Perl 6 for those who hate Perl</i>", Daisuke Maki on "<i>Crust --  Perl6 Port of Plack</i>", Jeffrey Goff on <i>Perl 6 Grammars</i>, Bart Wiegmans talks about AMD64 assembly language programming and MoarVM, Stevan Little's "<i>Perl is not dead,... it got better</i>" and lastly Elizabeth Mattijsen finishes with "<i>Perl 6 -- The end of the beginning</i>". <h2><a href="http://jdv79.blogspot.com/2015/10/perl6-and-cpan-metacpan-status-as-of.html">Perl6 and CPAN: MetaCPAN Status as of 2015-10-09</h2></a> <p><h3>Published by jdv on 2015-10-09T07:04:00</h3></p> <p>MetaCPAN, like the rest of "CPAN", was built assuming the sole context of Perl5. Which is cool until we want to use it for Perl6 and avoid the troubles associated with different namespaces, dist mgmt, etc... To largely avoid and more easily handle these issues for MetaCPAN it's been suggested that we have separate instances. The existing Perl5 instance only needs to be changed to ignore Perl6 distributions. There has already been some breakage because it didn't ignore a Perl6 dist of mine which exists in the Perl5 world:( And the new Perl6 instance will do just the opposite and only look at Perl6 distributions.</p> <p>In contrast, and relatedly, on CPAN we've designated a special spot for Perl6 distributions in order to keep them separate from the Perl5 dists. This reserved place is a Perl6 subdir in an author's dir (/author/id/*/*/*/Perl6/). Any dists in or under that spot on the fs will be considered a Perl6 dist; valid or invalid. So this is where the Perl6 MetaCPAN will look and the Perl5 instance will not.</p> <p>Current development is being done on these temporary branches:</p> <ul> <li>https://github.com/jdv/metacpan-web</li> <li>https://github.com/jdv/cpan-api</li> </ul> <p>And the main dev instance is running on hack.p6c.org. The web end is at http://hack.p6c.org:5001 and the api is at http://hack.p6c.org:5000.</p> <p>So far the idea has been to iterate on the aforementioned branches and instance until we have something that works sufficiently well. At that point we'll tidy up the branches and submit them for merging. Shortly after that time the hope is that we'll be able to stand up the official Perl6 instance.</p> <p>The list of requirements for being adequately cooked is:</p> <ol> <li>track Perl6 CPAN dists and ignore Perl5 dists</li> <li>import a Perl6 distribution</li> <li>index a Perl6 distribution for search</li> <li>render pod6 documentation</li> <li>do Perl6 syntax highlighting</li> </ol> <p>All of these have been hacked in and are at various degrees of completeness. Next up is testing and fixing bugs until nothing major is left. To that end I've recently loaded up the dev instance with all the distributions from modules.perl6.org. The dist files were generated, very hackily, with https://github.com/jdv/cpan-api/blob/master/test_p6_eco_to_p6_cpan.pl. I also just loaded them all under one user, mine, for simplicity. That load looks like it has problems of its own as well as revealing a bunch of issues. So in the coming days I hope to get that all sorted out.</p> <h2><a href="http://jdv79.blogspot.com/2015/10/perl6-and-cpan.html">Perl6 and CPAN</h2></a> <p><h3>Published by jdv on 2015-10-08T13:31:00</h3></p> <p>In the Perl5 world, just in case anyone is unaware, CPAN is a major factor. Its basically the hub of the Perl5 world.</p> <p>What I am referring to here as CPAN is not just the mirrored collection of 32K+ distributions. Its the ecosystem that's built up around that collection. This ecosystem has many parts, some more important than others depending on who you talk to, but the most important parts to me are:</p> <ul> <li>the mirrored distribution collection - all the Perl5 modules in one place</li> <li>pause (pause.perl.org) - module upload and mgmt services</li> <li>metacpan (metacpan.org) - online search, docs, and other services</li> <li>cpan testers (http://cpantesters.org/) - CPAN smoking</li> <li>module installers - Perl5 has cpanm and others - Perl6 has panda and others</li> </ul> <p>These are the 5 aspects of "CPAN" that I'd like to see happen for Perl6. One way to get that would be to write the whole thing from scratch in Perl6. While it may sound cool in some sort of dogfoody and/or bootstrappy kind of way to some, it sounds like a lot of work to me and we're a bit strapped for developer resources. Another way would be to add support for Perl6 to the existing CPAN bits. The hope there being, primarily, that it'd be a lot less work. The latter approach is what I've been working on lately. And if we want to refactor ourselves off the Perl5 bits in the future we can take our time doing it; later.</p> At this time we have: <ul> <li>basic pause support</li> <li>a metacpan prototype</li> </ul> <p>So we can publish Perl6 distributions to CPAN and search that collection. Well, sort of on that last bit. The metacpan prototype instance is not currently tracking CPAN. Its actually been loaded up with Perl6 distributions from the Perl6 module ecosystem (modules.perl6.org) for testing. But hopefully soon we'll have an official Perl6 metacpan instance, separate from the Perl5 instance, that will track CPAN's Perl6 content as it should.</p> What we need next is: <ul> <li>completion of basic metacpan Perl6 support</li> <li>at least one Perl6 module installer to support CPAN (probably panda)</li> <li>Perl6 integration with cpan testers</li> </ul> <p>If anyone is interested in working on any of this stuff please stop by #perl6 on freenode. If nobody else is able to help you I'll (jdv79) do my best.</p> <h2><a href="http://stevemynott.blogspot.com/2015/09/a-little-glrer-revision-1-glr-great.html"></h2></a> <p><h3>Published by Steve on 2015-09-11T09:43:00</h3></p> <h2> A Little GLRer (revision 1)</h2> The GLR (Great List Refactor) radically changed the way lists worked in Rakudo (an implementation of Perl). <br /> <br /> This blog post is a list of some one-liners to show differences between the old (pre-glr) rakudo and the new (glr) rakudo intended to aid understanding and porting of modules.<br /> <br /> Note this was done for self-education and may contain errors or things which may change.  <p> Thanks to those on Freenode IRC/perl6 for help. <p> Further corrections and expansions welcome either on iRC via pull request to <a href="https://github.com/stmuk/glr-html">https://github.com/stmuk/glr-html</a><br /> <br /> <br />  <table border="1" cellpadding="2" cellspacing="2" width="100%"> <tbody> <tr> <td valign="top"><br> </td> <td valign="top"><b> pre  GLR<b><br> </td> <td valign="top"><b>GLR</b><br> </td> </tr> <tr> <td valign="top"><b> LIST IS NOW PARCEL </b> </td> <tt> <td valign="top"> <pre>> say (1,2,3).WHAT</pre> <pre>(Parcel)</pre> </td> <td valign="top"> <pre>> say (1,2,3).WHAT</pre> <pre>(List)</pre> </td> </tt> </tr> <tr> <td valign="top"><b> LACK OF IMPLICIT LIST FLATTENING </b> </td> <tt> <td valign="top"> <pre>> my @array = 1,(2,3),4</pre> <pre>1 2 3 4</pre> <pre>> @array.elems</pre> <pre>4</pre> </td> <td valign="top"> <pre>my @array = 1,(2,3),4</pre> <pre>[1 (2 3) 4]</pre> <pre>> @array.elems</pre> <pre>3</pre> <h4>to flatten</h4> <pre>> my @list := 1, [2, 3], 4</pre> <pre>(1 [2 3] 4)</pre> <pre>> dd @list.flat.list</pre> <pre>(1, 2, 3, 4)</pre> <h4>or</h4> <pre>> my @array = (1,(2,3),4).flat</pre> <pre>[1 2 3 4]</pre> <h4>or more complex structures (jnthn++)</h4> <pre>say gather [[[[["a", "b"], "c"], "a"], "d"], "e"].deepmap(*.take)</pre> </td> </tt> </tr> <tr> <td valign="top"><b> .lol METHOD REMOVED </b> </td> <tt> <td valign="top"> <pre>> dd (1,2,3).lol</pre> <pre>(1; 2; 3)</pre> </td> <td valign="top"> </td> </tt> </tr> <tr> <td valign="top"><b> SINGLE ARG RULE </b> </td> <tt> <td valign="top"> <pre>> dd (1,)</pre> <pre>(1,)</pre> <pre>> dd [1,]</pre> <pre>$ = [1]</pre> <pre>> dd [[1,]]</pre> <pre>$ = [[1]]</pre> </td> <td valign="top"> <pre>> dd (1,)</pre> <pre>(1)</pre> <pre>> dd [1,]</pre> <pre>[1]</pre> <pre>> dd [[1],]</pre> <pre>[[1],]</pre> </td> </tt> </tr> <tr> <td valign="top"><b> LIST NOW IMMUTABLE </b> </td> <tt> <td valign="top"> <pre>> my @array = 1,2,3</pre> <pre>1 2 3</pre> <pre>> @array.shift</pre> <pre>1</pre> <pre>> dd @array</pre> <pre>@array = [2, 3]<></pre> </td> <td valign="top"> <pre>> my @list := 1,2,3</pre> <pre>(1 2 3)</pre> <pre>> @list.shift</pre> <pre>Method 'shift' not found for invocant of class 'List'</pre> <pre>> @list[0]</pre> <pre>1</pre> <pre>> dd @list</pre> <pre>(1, 2, 3)</pre> </td> </tt> </tr> <tr> <td valign="top"><b> ARRAY IS MUTABLE AND A SUBCLASS OF LIST </b> </td> <tt> <td valign="top"> </td> <td valign="top"> <pre>> my @array = 1,2,3</pre> <pre>[1 2 3]</pre> <pre>> @array[0]=0</pre> <pre>0</pre> <pre>> dd @array</pre> <pre>@array = [0, 2, 3]</pre> <pre>>say (Array).^mro</pre> <pre>((Array) (List) (Cool) (Any) (Mu))</pre> </td> </tt> </tr> <tr> <td valign="top"><b> SLIP SUBCLASS OF LIST </b> </td> <tt> <td valign="top"> </td> <td valign="top"> <pre>> my @a = 1, (2, 3).Slip, 4</pre> <pre>[1 2 3 4]</pre> <pre>> my $slip = slip(2,3)</pre> <pre>(2 3)</pre> <pre>> dd $slip</pre> <pre>Slip $slip = $(2, 3)</pre> <pre>> my @array = 1,$slip,4</pre> <pre>[1 2 3 4]</pre> <pre>> (1,$(2,3),4)</pre> <pre>(1 (2 3) 4)</pre> <pre>> (1,|(2,3),4)</pre> <pre>(1 2 3 4)</pre> </td> </tt> </tr> <tr> <td valign="top"><b> SEQUENCE </b> </td> <tt> <td valign="top"> </td> <td valign="top"> <pre>> my $grep = (1..4).grep(*>2); dd $grep>>.Int;</pre> <pre>(3, 4)</pre> <pre>> dd $grep>>.Int;</pre> <pre>This Seq has already been iterated, and its values consumed</pre> <pre>in block <unit> at <unknown file>:1</pre> <h4>prevent consumption</h4> <pre>> my $grep = (1..4).grep(*>2); my $cache=$grep.cache</pre> <pre>(3 4)</pre> <pre>> say $cache>>.Int</pre> <pre>(3 4)</pre> <pre>> say $cache>>.Int</pre> <pre>(3 4)</pre> <pre>> my @array = 1,(2,3),4</pre> <pre>[1 (2 3) 4]</pre> <pre>> dd @array.flat</pre> <pre>(1, $(2, 3), 4).Seq</pre> <pre>> dd @array.flat.list</pre> <pre>(1, $(2, 3), 4)</pre> </td> </tt> </tr> </tbody> </table> <h2><a href="http://stevemynott.blogspot.com/2015/09/we-came-down-to-granada-on-tuesday.html">YAPC::EU 2015</h2></a> <p><h3>Published by Steve on 2015-09-05T08:42:00</h3></p> We came down to Granada on Tuesday night and (after missing the pre-conference meeting with its free pizza) made our way to the Amsterdam Bar with its massive selection of bottled import beers and rather bizarre nut and soft sweets tapas.<br /> <br /> Wednesday morning we made our way to the venue.  The conference topic was Art and Engineering and the venue a particularly arty looking university science building with a large Foucault pendulum inside and "Bombes de Vapor" (steam engines and the like) outside.  The Arabic art influenced T shirts were the most stylish since the Pisa ones and the seats in the main hall were the most comfortable YAPC seats ever.<br /> <br /> <br /> <br /> <br /> <div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQQa3jJBh5fOv3kJ7ohni8p3TdByDun4oRR6T2NQRZfZoD5UznBp1ROb4ZyaowI-VnnHzlJG8qlGlWsm7bdFDJY0UmALwYyGlW_LZtmvJWI565NrGC4MuX0BV86R0kYBE8FyxqPUX1mf8/s1600/IMG_0334.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQQa3jJBh5fOv3kJ7ohni8p3TdByDun4oRR6T2NQRZfZoD5UznBp1ROb4ZyaowI-VnnHzlJG8qlGlWsm7bdFDJY0UmALwYyGlW_LZtmvJWI565NrGC4MuX0BV86R0kYBE8FyxqPUX1mf8/s320/IMG_0334.jpg" width="320" /></a></div> <br /> <br /> I first saw Leon Timmermans gave some good advice about how to contribute to Perl 5 core even if you didn't know the odd C89 plus macros language in which it was written.  It was followed by Bart (brrt) Wiegmans speaking about the Just In Time (JIT) compiler for MoarVM -- perl6's main VM -- in a quite high level talk so we were spared the scary details (which I later noticed included s-expressions).  Kang-min (gugod) Liu spoke about Booking's search engine which he couldn't show us and how he indexed his email (which he could). <br /> <br /> <a href="https://github.com/gugod/Mailsheep">https://github.com/gugod/Mailsheep</a><br /> <br /> The main conference dinner of tapas was that evening around the pool of a four star hotel with constant glass refills.  Thankfully noone fell in.  More sadly we learnt Jeff Goff had been injured earlier and was in hospital.<br /> <br /> Next day started with Sawyer X's State of the [Art] Velociraptor which was upbeat and positive stressing the benefits of community.  Upasana spoke about Moose meta objects and Leonerd bravely fought AV issues to speak about how perl5 sort of resembled scheme a little bit. <br /> <br /> <a href="https://t.co/9KeXNzSSlJ">https://t.co/9KeXNzSSlJ</a> <br /> <br /> At the end of day Xavier Noria, currently a ruby programmer, spoke about how much he missed Perl since many things (like docs) were better.<br /> <br /> Next day I got up at silly o'clock to hear Art School dropout Stevan Little compare his former subject with programming with some interesting details about painting techniques.  Kerstin Puschke talked about RabbitMQ including some live code examples using Perl 5.<br /> <br /> <a href="https://speakerdeck.com/titanoboa/decouple-all-the-things-asynchronous-messaging-keeps-it-simple">https://speakerdeck.com/titanoboa/decouple-all-the-things-asynchronous-messaging-keeps-it-simple</a><br /> <br /> <div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBjVe9P7sSjOEr-gt62sEroLujTbWNC3nqQJXmRD83N-EYlqrNzXNpA6Ox-J57xx51mtESuujpee9L_IbFcGBfTCT6gllNEqSV6pf1ViA_RiZbsUaMTC0NP_3Cww0BUqhiWh4eCHhswu4/s1600/IMG_0320.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBjVe9P7sSjOEr-gt62sEroLujTbWNC3nqQJXmRD83N-EYlqrNzXNpA6Ox-J57xx51mtESuujpee9L_IbFcGBfTCT6gllNEqSV6pf1ViA_RiZbsUaMTC0NP_3Cww0BUqhiWh4eCHhswu4/s320/IMG_0320.jpg" width="320" /></a></div> <br /> <br /> Domm told us about his image uploading Perl 6 script<br /> <br /> <a href="http://domm.plix.at/talks/2015_granada_potd">http://domm.plix.at/talks/2015_granada_potd</a><br /> <br /> which uploaded pics to twitter including one of his audience.<br /> <br /> <a href="http://domm.plix.at/potd/2015-09-04.html">http://domm.plix.at/potd/2015-09-04.html</a><br /> <br /> Gabor talked us through a minimal port of Dancer to Perl 6 called "Bailador" (which is part of Rakudo Star).<br /> <br /> <a href="http://perl6maven.com/bailador-app-in-a-module">http://perl6maven.com/bailador-app-in-a-module</a><br /> <br /> perl6maven.com actually uses perl6 in production!<br /> <br /> Herbert Breunung spoke about Functional Perl 6 using a particularly garish slide deck.  John Lightsey did a line by line audit of an old version of Module::Signature to point out some security issues.  Liz did Jonathan's Parallelism, Concurrency, and Asynchrony in Perl 6 since the original author sadly couldn't make it.  At least one thing had changed in the week since I last heard the talk!<br /> <br /> Finally a long haired Larry compared Perl 5 and 6 with Tolkien's Hobbit and Lord of the Rings respectively  and sang a bit. Two out of the three big ticket items for Perl 6.0 were done and things were looking good for a Long Expected Christmas Party.  This was a truly great keynote and went down a storm.<br /> <br /> Some of the final lightening talks were particularly good with one even given in rapid Japanese.  To finish off Friday night Sue organised a "Sherry Tasting" visit to a local tapas restaurant which also included much consumption of the local beer Alhambra 1925.  A large number of mongers turned up to effectively take over the whole place.  Some also stayed up all night playing card games <h2><a href="http://stevemynott.blogspot.com/2015/08/perl6-grammars-for-beginners-talk.html">Perl6 Grammars for Beginners Talk</h2></a> <p><h3>Published by Steve on 2015-08-14T07:35:00</h3></p> I gave a beginners level talk about Perl6 grammars to a meeting of London Perlmongers on Aug 13th talking about <b>App::p6tags (</b>Generate editor tags for perl6). <br /> <span class="edit-link js-details-target"></span><br /> <a href="https://github.com/stmuk/p6-app-p6tags">https://github.com/stmuk/p6-app-p6tags</a><br /> <br /> <br /> Sides are at<br /> <a href="https://drive.google.com/file/d/0B731WtD7iMvMbThKOEszellTTjA/view?usp=sharing">https://drive.google.com/file/d/0B731WtD7iMvMbThKOEszellTTjA/view?usp=sharing</a><br /> <br /> And video on youtube. <br /> <br /> <div class="separator" style="clear: both; text-align: center;"> <iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/BQfP5o_0YzY/0.jpg" src="https://www.youtube.com/embed/BQfP5o_0YzY?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div> <br /> <br /> <br /> <br /> <h2><a href="http://blog.brentlaabs.com/2015/05/three-tales-of-second-system-syndrome.html">Three Tales of Second System Syndrome</h2></a> <p><h3>Published by Brent Laabs on 2015-05-03T19:57:00</h3></p> <br /> In the last decade, three major scripting languages embarked on project to produce a major revision to each language: Perl 6, Python 3, and PHP 6. Despite surface similarities, such as the problem of Unicode support, each language ended up on a radically different track.<br /> <br /> With the Perl 6.0.0 release officially coming this year, it's a good time to reflect on how we got to this point, and to start thinking about what comes after the release.<br /> <br /> <h2 id="php-6"> PHP 6</h2> So -- and I can't believe I'm writing this -- let's see if we can learn something from PHP. Andi Gutmans, who is now the CEO of Zend Technologies, <a href="http://www.itjungle.com/tlb/tlb021208-story09.html">gave an interview</a> back in February 2008. In it, he said, <br /> <blockquote> So we are anticipating a long rollout cycle for PHP 6, and we did not want to take the same route that the Perl project did, with project contributors still working on Perl 6 I think six years later. People make fun of Microsoft, but take a look at Perl 6. . . .</blockquote> <br /> To which Andy Lester of PerlBuzz <a href="https://web.archive.org/web/20140802224335/http://perlbuzz.com/2008/02/tell-us-how-to-do-it-andi.html">replied</a>:<br /> <blockquote> Sure, PHP 6 may have a shorter release cycle than Perl 6 has, but at the end of it all, we'll have Perl 6, and you'll still have PHP. <br /> <br /> Just sayin'. <br /> xoxo,<br /> Andy</blockquote> So how did those predictions work out? Well, after a little over six years of development, we discovered that we were <a href="https://philsturgeon.uk/php/2014/07/23/neverending-muppet-debate-of-php-6-v-php-7/">never going to see a PHP 6 at all</a>. Having seen how long Perl 6 had taken, and how long PHP 6 was taking, the number 6 is associated with failure. So they cancelled PHP 6 and voted to change the name to PHP 7. Problem solved! No, really, this is some of the actual reasoning given by people on the 6 to 7 RFC. (Someone should tell the ES6 folks before the curse strikes our browsers!)<br /> <br /> But the main intent of the renumbering was to justify a much reduced scope of new features for the next major version of PHP. <a href="http://www.phpclasses.org/blog/post/242-PHP-7-Features-and-Release-Date.html">PHP 7 is slated to add</a>:<br /> <ul> <li>"Huge Performance Improvements" to the Zend engine (HHVM already pretty speedy)</li> <li><strike>JIT to the Zend engine (but already available in HHVM)</strike></li> <li>Abstract Syntax Tree (AST) generation</li> <li>Asynchronous IO and functions</li> <li>Standalone Multi-threading Web Server (HHVM) <ul> <li>Which is cool if you want a language to provide its own server, I guess </li> </ul> </li> </ul> <b>EDIT</b>: People both here and on <a href="https://news.ycombinator.com/item?id=9484939">Hacker News</a> have pointed out that this is the above feature list was from a bad source, and that much of PHP 6 was incorporated into 5.3.  See the <a href="https://blog.engineyard.com/2015/what-to-expect-php-7">better summary of PHP 7 features</a>, including generator improvements, and new operators like <span style="background-color: #eeeeee;"><span style="font-family: "Courier New",Courier,monospace;">??</span></span>.  However, much of the same analysis still applies -- the end result was very few backwards incompatible changes, not the major revision promised with major Unicode improvements.<br /> <br /> <h2 id="perl-6"> Perl 6</h2> Meanwhile Perl 6, which has taken 15 years to get to the 6.0.0 release slated for <i>this</i> Christmas.  I'm sure that there were some embarrassing quotes about when it's going to be done, but that was so long ago, I'll just link to <a href="http://blogs.perl.org/users/steven_haryanto/2013/03/dont-wait-for-perl-6-to-build-your-product-on.html">this post forecasting</a> that Perl 6 will be ready for production in 2027.<br /> <br /> As it now stands, Perl 6 comes with this <a href="http://perl6.org/compilers/features">set of new features</a>:<br /> <ul> <li>A real type system (not just type hints as in PHP) everywhere <ul> <li>The ability to continue ignoring types most short scripting code</li> <li>The ability to use static type checks to catch errors</li> <li>Native types (C strings, unsigned ints, etc.) unlock new performance potential</li> <li>Meta-object programming is available</li> </ul> </li> <li>A sane native function calling interface</li> <li>Rakudo Perl 6 runs on multiple virtual machines (JVM, MoarVM), with more backends planned (Javascript) <ul> <li>Can compile P6 to bytecode or an AST</li> <li>Takes advantage of VM's JIT</li> </ul> </li> <li>Complete syntax refactoring <ul> <li>Fully backwards compatible with the use of Inline::Perl5</li> <li>Consistent syntax throughout</li> </ul> </li> <li>Native Unicode handling, with NFG (grapheme) strings by default</li> <li>Hygienic macros <ul> <li>As I said above, AST</li> </ul> </li> <li>Regexes have evolved to Grammars, a first class language <ul> <li>PCRE are no longer Perl-compatible (except in Perl 5 mode), but are instead much easier to read</li> </ul> </li> <li>Easy to use concurrency <ul> <li>Some operators will autothread</li> <li>Junction types for set operations</li> </ul> </li> <li>Module versioning to ensure that even if a module completely changes its API, your code will not break. If you declare a version, of course.</li> </ul> Honestly, there are a whole lot more of these features. This even excludes things that have already made back into the Perl 5 core, like subroutine signatures and smartmatching. And these are all things that are working <i>today</i>.<br /> <br /> The eerie thing is that Andy's flippant prediction came true. At the end of it, we have Perl 6, and they still have the same old PHP. Let me repeat that: <i>we have Perl 6</i>. It works, it will get a major release this year, and it is going to come with many more features than originally promised.<br /> <br /> Still, Perl 6 has had its share of doubters. Some people proposed, actually seriously, that <a href="http://blogs.perl.org/users/ovid/2013/02/perl-7.html">Perl 5 should leapfrog ahead to Perl 7</a> with the next version, and Perl 6 can go on calling itself that if it wants. Right. While this idea was rejected by the general Perl community, PHP actually skipped a version a year later. I guess it's another example of PHP stealing the worst ideas from Perl.<br /> <br /> <h2 id="python-3"> Python 3</h2> The Python group, on the other hand, has tried to stay mostly on the side of sanity. Python 3 introduced a much smaller set of breaking changes, in order to keep updates rolling out. It was introduced, well, six years ago in early 2009.<br /> <br /> <a href="https://docs.python.org/release/3.1.2/whatsnew/3.0.html">New features of Python 3</a> included:<br /> <ul> <li>Sane Unicode handling <ul> <li>A breaking change that allowed all of the other breaking changes to happen</li> </ul> </li> <li>Various name changes for style consistency</li> <li>Automatically loading C modules when available.</li> <li>Refactor of exceptions</li> <li>Support for ancient OSes dropped</li> <li>Old functions removed, along with generally bad APIs</li> <li>Statement form <code>print</code> removed in favor of function <code>print()</code>, ostensibly to make a consistent API <span style="font-size: xx-small;">but really just to mess with people</span>.</li> </ul> So how's that working out? The latest version of python preinstalled on my fully updated MacBook is 2.7.6. At least Ubuntu gives me 3.4.0 — Apple is well known to be crap at updating OSS. But you'd think someone at Apple would have cared in six years would have cared enough to throw python3 in the XCode monster download; after all, Python does not have the <a href="https://news.ycombinator.com/item?id=3559990">kiss of death</a> known as the GPLv3 license.<br /> <br /> The flip side of availability is developer adoption; this story isn't much better. If you look at <a href="https://blog.newrelic.com/2014/01/21/python-3-adoption-web-apps/">statistics from a last year</a> and <a href="http://lwn.net/Articles/640181/">this month</a>, Python 3 adoption rates are abysmal. Hell, even 23% of people inside the Python community <a href="https://wiki.python.org/moin/2.x-vs-3.x-survey">still think Python 3 was a mistake</a>. Despite obvious improvements, it's still considered a tough sell.<br /> <br /> <h2 id="second-deployment-syndrome"> Second Deployment Syndrome</h2> So the takeaway from all of this is that <a href="http://en.wikipedia.org/wiki/The_Mythical_Man-Month#The_second-system_effect">Second System Syndrome</a> is a real problem, but not the only problem. Successfully executing major revisions to a language is difficult, but getting widespread <i>adoption</i> is just as difficult with breaking changes. <i>Second Deployment Syndrome</i> can be just as hard to face as making the new system in the first place.<br /> <br /> So we have three software communities that took radically different approaches to building a second system. PHP is a <a href="http://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/">complete zoo of awful design</a>, begging to be tamed. Yet the PHP community effectively voted to give up, and only offer incremental change that doesn't address PHP 6's number one issue of Unicode support. The Python folks, bless their hearts, made a smaller set of achievable changes, implemented it in 3 years, and shipped the damn thing. And despite the truly useful improvements, only a few people came.<br /> <br /> Perl decided to stick to its vision of "break all the things once", and it's taken 15 long years. That's almost as long as the HTML 5 spec. Over this time, the design has continued to evolve, incorporating more modern needs like easily multithreaded code that would have otherwise been missed. Although the complaint of "no final spec" is common, it has been learned the hard way that the spec is the very last thing that should be finalized.<br /> <br /> It's easy to naively say that 15 years is a ridiculous amount of development time, but it's obvious from looking at second systems for the other scripting languages, Perl 6 was never going to complete such a major transition in less than a decade. What's still unclear is whether this transition is going to work out for Perl.<br /> <br /> Nearly everyone who tries Perl 6 from a Perl 5 background likes it immensely, which is usually followed up by a "can this not be so <i>slow</i>?" Optimization is still getting there, just not prematurely. In general, reception has been a net positive. And unlike the breaking changes introduced in the other languages, Inline::Perl5 allows multiple versions of Perl to coexist in the same program.<br /> <br /> Will this be enough? It's too early to tell. Perl 5 is going to last another 5 years at the minimum, if not forever, munging text output by a shell script written by a programmer from generations ago. Perl 6 will have an uphill battle with Perl 5 for ubiquity, legacy code, and language familiarity.<br /> <br /> Adoption rate is the next big challenge facing Perl 6. There is a very real possibility that six years from now, Perl 5 will still be the dominant form of an ever shrinking faction of Perl users. After all, Python might be in the <a href="http://en.wikipedia.org/wiki/TIOBE_index">same boat</a> right now. Perl needs to reverse an already existing downward trend, at least partially brought on by how frakking long Perl 6 took in the first place.<br /> <br /> The best advice I can see for ensuring Perl 6's success is for Perl developers to <b>start writing code in Perl 6</b>. I mean now; it's definitely stable enough. Every module available within a year of release is going to be a major argument for people to try the new version. Getting Shit Done can win a lot of arguments.<br /> <br /> After that, it's going to be a tough slog. Is it deployed enough places to distribute code in? Is there enough code written in it to deploy to more places? Package managers like apt and Homebrew are going to help with bootstrapping the user base, but to win Perl 6 going to have to get that killer app.<br /> <br /> So for now, it's a giant gamble. In poker terms, Python 3 called, PHP 6 folded, and Perl 6 went all-in. It just might be possible that Perl 6's crazy long development process can produce the best-adopted second system around, if people decide that the overwhelming improvements are worth the hassle of upgrading.<br /> <br /> I'll let you know how that went in six years. <h2><a href="https://www.nntp.perl.org/group/perl.perl6.announce/2015/05/msg726.html">Parrot 7.4.0 "Festive Amazon" released! by Bruce Gray</h2></a> <p><h3>Published on 2015-05-20T14:59:53</h3></p> On behalf of the Parrot team, I'm proud to announce Parrot 7.4.0, also known <br/>as "Festive Amazon". Parrot (http://parrot.org/) is a virtual machine aimed <br/>at running all dynamic languages. <br/> <br/>Parrot 7.4.0 is available on Parrot's FTP site <br/>(ftp://ftp.parrot.org/pub/parrot/releases/devel/7.4.0/), or by following the <br/>download instructions at http://parrot.org/download. For those who would like <br/>to develop on Parrot, or help develop Parrot itself, we recommend using Git to <br/>retrieve the source code to get the latest and best Parrot code. <br/> <br/>Parrot 7.4.0 News: <br/> - Documentation <br/> + Many minor corrections <br/> - Community <br/> + Coverity scans to resume RSN. <br/> <br/> <br/>The SHA256 message digests for the downloadable tarballs are: <br/>b191da72e668c5bd97e1792a1b5d8fe66713819066f6a2f5eef2e9bc21d92968 parrot-7.4.0.tar.gz <br/>724868f94bf7d45ba5cda29b041b18fc7cbcd2fe5196455cc3882c2f99a84f4b parrot-7.4.0.tar.bz2 <br/> <br/>Many thanks to all our contributors for making this possible, and our sponsors <br/>for supporting this project. Our next scheduled release is at 16 Jun 2015. <br/> <br/>Enjoy! <br/> <br/> <h2><a href="https://www.nntp.perl.org/group/perl.perl6.announce/2015/04/msg725.html">Parrot 7.3.0 release announcement by Reini Urban</h2></a> <p><h3>Published on 2015-04-21T17:58:08</h3></p> On behalf of the Parrot team, I'm proud to announce Parrot 7.3.0, also<br/>known as "Peach-faced Lovebird".<br/>It is a supported release with a stable API until 7.6.0 end of July 2015.<br/>Parrot (http://parrot.org/) is a virtual machine aimed at running all<br/>dynamic languages.<br/><br/>Parrot 7.3.0 is available on Parrot's FTP site<br/>(ftp://ftp.parrot.org/pub/parrot/releases/supported/7.3.0/), or by following the<br/>download instructions at http://parrot.org/download. For those who<br/>would like to develop on Parrot, or help develop Parrot itself, we<br/>recommend using Git to retrieve the source code to get the latest and<br/>best Parrot code.<br/><br/>Parrot 7.3.0 News:<br/> - Build<br/> + Fixed windows link regression from 7.0.2 with cl.exe. #1203<br/> + Fixed rlimit compilation for OpenBSD<br/> - Tests<br/> + Relaxed the common GC stress test and re-add the JSON.nqp variant.<br/><br/><br/>The SHA256 message digests for the downloadable tarballs are:<br/>23d2f59a0399a63a835087a192bede02a25f21fbcf5e42ed113b4c6dcdbea6b1<br/>parrot-7.3.0.tar.gz<br/>a40a6a21965ead120ceee7ac98e3b0ba2edbdfa2a5d8637ace91dcc7991373f2<br/>parrot-7.3.0.tar.bz2<br/><br/>Many thanks to all our contributors for making this possible, and our<br/>sponsors for supporting this project. Our next scheduled release is<br/>at 19 May 2015.<br/><br/>Enjoy!<br/> <h2><a href="https://www.nntp.perl.org/group/perl.perl6.announce/2015/03/msg724.html">Parrot 7.2.0 "Blue-crowned racquet-tail" released! by Bruce Gray</h2></a> <p><h3>Published on 2015-03-19T06:06:01</h3></p> This is the bright candlelit room where the life-timers are <br/> stored—shelf upon shelf of them, squat hourglasses, one for every <br/> living person, pouring their fine sand from the future into the past. <br/> The accumulated hiss of the falling grains makes the room roar like <br/> the sea. <br/> <br/> This is the owner of the room, stalking through it with a preoccupied air. <br/> His name is Death. <br/> <br/> But not any Death. This is the Death whose particular sphere of <br/> operations is, well, not a sphere at all, but the Discworld, which is <br/> flat and rides on the back of four giant elephants who stand on the <br/> shell of the enormous star turtle Great A’Tuin, and which is bounded by <br/> a waterfall that cascades endlessly into space. <br/> <br/> Scientists have calculated that the chance of anything so patently <br/> absurd actually existing are millions to one. <br/> <br/> But magicians have calculated that million-to-one chances crop up nine <br/> times out of ten. <br/> <br/> -- "Mort", GNU Terry Pratchett <br/> <br/>On behalf of the Parrot team, I'm proud to announce Parrot 7.2.0, also known <br/>as "Blue-crowned racquet-tail". Parrot (http://parrot.org/) is a virtual machine aimed <br/>at running all dynamic languages. The blue-crowned racket-tail (Prioniturus discurus) <br/>is a parrot found on all the larger islands of the Philippines not starting with "P". <br/> <br/>Parrot 7.2.0 is available on Parrot's FTP site <br/>(ftp://ftp.parrot.org/pub/parrot/releases/devel/7.2.0/), or by following the <br/>download instructions at http://parrot.org/download. For those who would like <br/>to develop on Parrot, or help develop Parrot itself, we recommend using Git to <br/>retrieve the source code to get the latest and best Parrot code. <br/> <br/>Parrot 7.2.0 News: <br/> - Build <br/> + Fix warning on Win32 (with cl.exe) when `link` is not explicitly set. <br/> <br/> <br/>The SHA256 message digests for the downloadable tarballs are: <br/>f4792fc1a82040dd855f73890de6fa26759aa62f4b4ad1aa468597592b7bf3bf parrot-7.2.0.tar.gz <br/>74e5821155eaf29d7c1655fd3b5b90a84afe23361318242947c50f59da5918e1 parrot-7.2.0.tar.bz2 <br/> <br/>Many thanks to all our contributors for making this possible, and our sponsors <br/>for supporting this project. Our next scheduled release is at 21 Apr 2015. <br/> <br/>Enjoy! <br/> <br/> <h2><a href="http://pmthium.com/2015/02/suspending-rakudo-parrot/">Suspending Rakudo support for Parrot</h2></a> <p><h3>Published by pmichaud on 2015-02-16T15:47:37</h3></p> <p>At <a href="https://fosdem.org/2015/">FOSDEM 2015</a>, Larry <a href="https://fosdem.org/2015/schedule/event/get_ready_to_party/">announced</a> that there will likely be a <a href="http://perl6.org/">Perl 6</a> release candidate in 2015, possibly around the September timeframe. What we’re aiming for is concurrent publication of a language specification that has been implemented and tested in at least one usable compilation environment — i.e., <a href="http://rakudo.org/">Rakudo Perl 6</a>.</p> <p>So, for the rest of 2015, we can expect the Rakudo development team to be highly focused on doing only those things needed to prepare for the Perl 6 release later in the year. And, from previous planning and discussion, we know that there are three major areas that need work prior to release: the <a title="APW2014 and the Rakudo Great List Refactor" href="http://pmthium.com/2014/10/apw2014/">Great List Refactor</a> (GLR), Native Shaped Arrays (NSA), and Normalization Form Grapheme (NFG).</p> <p>…which brings us to <a href="http://parrot.org/">Parrot</a>. Each of the above items is made significantly more complicated by Rakudo’s ongoing support for Parrot, either because Parrot lacks key features needed for implementation (NSA, NFG) or because a lot of special-case code is being used to maintain adequate performance (lists and GLR).</p> <p>At present most of the current userbase has switched over to <a href="http://moarvm.com/">MoarVM</a> as the backend, for a multitude of reasons. And more importantly, there currently aren’t any Rakudo or NQP developers on hand that are eager to tackle these problems for Parrot.</p> <p>In order to better focus our limited resources on the tasks needed for a Perl 6 language release later in the year, we’re expecting to suspend Rakudo’s support for the Parrot backend sometime shortly after the 2015.02 release.</p> <p>Unfortunately the changes that need to be made, especially for the GLR, make it impractical to simply leave existing Parrot support in place and have it continue to work at a “degraded” level. Many of the underlying assumptions will be changing. It will instead be more effective to (re)build the new systems without Parrot support and then re-establish Parrot as if it is a new backend VM for Rakudo, following the techniques that were used to create JVM, MoarVM, and other backends for Rakudo.</p> <p>NQP will continue to support Parrot as before; none of the Rakudo refactorings require any changes to NQP.</p> <p>If there are people that want to work on refactoring Rakudo’s support for Parrot so that it’s more consistent with the other VMs, we can certainly point them in the right direction. For the GLR this will mainly consists of migrating parrot-specific code from Rakudo into NQP’s APIs. For the NSA and NFG work, it will involve developing a lot of new code and feature capabilities that Parrot doesn’t possess.</p> <h2><a href="https://www.nntp.perl.org/group/perl.perl6.announce/2015/02/msg723.html">Announce: Rakudo Star Release 2015.01 by Moritz Lenz</h2></a> <p><h3>Published on 2015-02-07T23:23:53</h3></p> # Announce: Rakudo Star Release 2015.01<br/><br/>## A useful, usable, "early adopter" distribution of Perl 6<br/><br/>On behalf of the Rakudo and Perl 6 development teams, I'm happy to<br/>announce the January 2015 release of "Rakudo Star", a useful and usable<br/>distribution of Perl 6. The tarball for the January 2015 release is<br/>available from <http://rakudo.org/downloads/star/>.<br/><br/>This Rakudo Star release comes with support for the MoarVM<br/>backend (all module tests pass on supported platforms) along with<br/>experimental support for the JVM backend (some module tests fail).<br/>Three shipped modules are known to fail on Parrot (zavolaj (NativeCall),<br/>jsonrpc and doc)<br/><br/>In the Perl 6 world, we make a distinction between the language<br/>("Perl 6") and specific implementations of the language such as<br/>"Rakudo Perl". This Star release includes [release 2015.01.1] of the<br/>[Rakudo Perl 6 compiler], version 7.0.1 of the [Parrot Virtual<br/>Machine], version 2015.01 of [MoarVM], plus various modules,<br/>documentation, and other resources collected from the Perl 6<br/>community.<br/><br/>[release 2015.01.1]:<br/> https://github.com/rakudo/rakudo/blob/nom/docs/announce/2015.01.md<br/>[Rakudo Perl 6 compiler]: http://github.com/rakudo/rakudo<br/>[Parrot Virtual Machine]: http://parrot.org<br/>[MoarVM]: http://moarvm.org/<br/><br/>Some of the new compiler features added to this release include:<br/><br/>+ Many improvements to Java interop for the JVM backend<br/>+ New simple way of creating an object hash: :{}<br/>+ Substitution now supports assignment meta-op, e.g. s[\d+] += 2<br/>+ Many memory and CPU optimizations<br/>+ Supply.for deprecated in favour of Supply.from-list<br/><br/>Changes to modules included in Rakudo Star:<br/><br/>- [Bailador](https://github.com/tadzik/Bailador) handles POST and URL<br/>params separately<br/>- [DBIish](https://github.com/perl6/DBIish) has improved error reporting<br/>on SQLite<br/>- [doc](https://github.com/perl6/doc) ships with much more documentation<br/>- [panda](https://github.com/tadzik/panda) has a new command `installdeps`<br/>- [Pod::To::HTML](https://github.com/perl6/Pod-To-HTML) now supports<br/>callbacks for code areas<br/><br/>Parrot support will likely be suspended or dropped from future Rakudo<br/>and Rakudo<br/>Star releases, starting with the February or March releases.<br/><br/>In the next Rakudo Star release, modules `Math::RungeKutta` and<br/>`Math::Model`<br/>will likely be dropped. They can still be installed with `panda`.<br/><br/>In future, the `nqp::` namespace willl only be available after a declaration<br/>like `use nqp;'.<br/><br/>There are some key features of Perl 6 that Rakudo Star does not yet<br/>handle appropriately, although they will appear in upcoming releases.<br/>Some of the not-quite-there features include:<br/><br/> * advanced macros<br/> * threads and concurrency (in progress for the JVM and MoarVM backend)<br/> * Unicode strings at levels other than codepoints<br/> * interactive readline that understands Unicode<br/> * non-blocking I/O (in progress for the JVM and MoarVM backend)<br/> * much of Synopsis 9 and 11<br/><br/>There is an online resource at <http://perl6.org/compilers/features><br/>that lists the known implemented and missing features of Rakudo's<br/>backends and other Perl 6 implementations.<br/><br/>In many places we've tried to make Rakudo smart enough to inform the<br/>programmer that a given feature isn't implemented, but there are many<br/>that we've missed. Bug reports about missing and broken features are<br/>welcomed at <<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="93e1f2f8e6f7fcf1e6f4d3e3f6e1ffbdfce1f4">[email protected]</a>>.<br/><br/>See <http://perl6.org/> for links to much more information about<br/>Perl 6, including documentation, example code, tutorials, reference<br/>materials, specification documents, and other supporting resources. A<br/>draft of a Perl 6 book is available as docs/UsingPerl6-draft.pdf in<br/>the release tarball.<br/><br/>The development team thanks all of the contributors and sponsors for<br/>making Rakudo Star possible. If you would like to contribute, see<br/><http://rakudo.org/how-to-help>, ask on the <<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="314154435d071c525e5c41585d5443714154435d1f5e4356">[email protected]</a>><br/>mailing list, or join us on IRC \#perl6 on freenode.<br/> <h2><a href="https://www.nntp.perl.org/group/perl.perl6.announce/2015/01/msg722.html">Parrot 7.0.2 Hotfix released by Reini Urban</h2></a> <p><h3>Published on 2015-01-29T14:02:38</h3></p> We detected and fixed two bugs and regressions from 6.10.0 which<br/>failed to build parrot on Microsoft Windows with Microsoft Visual<br/>Studio C++.<br/><br/>- Wrong function ptr cast on win64<br/> https://github.com/parrot/parrot/issues/1190<br/>- Wrong SAL annotations on msvc cl < 16.00<br/> https://github.com/parrot/parrot/issues/1192<br/><br/>Other minor changes in this hotfix:<br/>- Optimize away ExtUtils::Command on posix systems. #1177<br/>- Fix cpu config values for gcc_cmpxchg to include atomic/gcc_x86.o on amd64.<br/> Harmonized the cpu config keys, no $platform_has_$feature<br/> keys anymore, just HAS_$PLATFORM_$feature. #1173<br/>- Improved msvc configuration from a mingw perl. #1191<br/><br/>Parrot is a virtual machine aimed at running all dynamic languages.<br/>Parrot 7.0.2 is available on Parrot's FTP site, or by following the<br/>download instructions. For those who want to hack on Parrot or<br/>languages that run on top of Parrot, we recommend our organization<br/>page on GitHub, or you can go directly to the official Parrot Git repo<br/>on Github<br/><br/>To clone the Parrot Git repo into a directory called 'parrot', use the<br/>following:<br/> git clone git://github.com/parrot/parrot.git<br/><br/>If you want it to be in a directory other than 'parrot', then just<br/>give that as a second argument to clone:<br/> git clone git://github.com/parrot/parrot.git parrot_foo<br/><br/>The SHA256 message digests for the downloadable tarballs are:<br/>1a8e9e203ad8ac92c89422603d1603fa821e957aa3a9ae57420c737d93c55213<br/>parrot-7.0.2.tar.gz<br/>2cc8dc1eada38bb6328bf1f4648bd5e01e000b415f984b7ad6b5b6c123a15ac9<br/>parrot-7.0.2.tar.bz2<br/><br/>Thanks to all our contributors for making this possible, and our<br/>sponsors for supporting this project. Our next scheduled release is at<br/>17 Feb 2015.<br/>Enjoy!<br/>-- <br/>Reini Urban<br/>http://cpanel.net/ http://www.perl-compiler.org/<br/> <h2><a href="http://pmthium.com/2014/10/apw2014/">APW2014 and the Rakudo Great List Refactor</h2></a> <p><h3>Published by pmichaud on 2014-10-15T15:01:55</h3></p> <p>This past weekend I attended the <a href="http://act.useperl.at/apw2014/">2014 Austrian Perl Workshop and Hackathon</a> in Salzburg, which turned out to be an excellent way for me to catch up on recent changes to <a href="http://perl6.org/">Perl 6</a> and <a href="http://rakudo.org/">Rakudo</a>. I also wanted to participate directly in discussions about the Great List Refactor, which has been a longstanding topic in Rakudo development.</p> <p>What exactly is the “Great List Refactor” (GLR)? For several years Rakudo developers and users have identified a number of problems with the existing implementation of list types — most notably performance. But we’ve also observed the need for user-facing changes in the design, especially in generating and flattening lists. So the term GLR now encompasses all of the list-related changes that seem to want to be made.</p> <p>It’s a significant (“great”) refactor because our past experience has shown that small changes in the list implementation often have far-reaching effects. Almost any bit of rework of list fundamentals requires a fairly significant refactor throughout much of the codebase. This is because lists are so fundamental to how Perl 6 works internally, just like the object model. So, as the number of things that are desirable to fix or change has grown, so has the estimated size of the GLR effort, and the need to try to achieve it “all at once” rather than piecemeal.</p> <p>The pressure to make progress on the GLR has been steadily increasing, and APW2014 was significant in that a lot of the key people needed for that would be in the same location. Everyone I’ve talked to agrees that APW2014 was a smashing success, and I believe that we’ve now resolved most of the remaining GLR design issues. The rest of this post will describe that.</p> <p>This is an appropriate moment to recognize and thank the people behind the APW effort. The organizers did a great job. The Techno-Z and ncm.at venues were fantastic locations for our meetings and discussions, and I especially thank <a href="http://ncm.at/">ncm.at</a>, <a href="http://www.techno-z.at/">Techno-Z</a>, <a href="http://www.yelsterdigital.com/">yesterdigital</a>, and <a href="http://vienna.pm.org/">vienna.pm</a> for their generous support in providing venues and food at the event.</p> <p>So, here’s my summary of GLR issues where we were able to reach significant progress and consensus.</p> <h2>You are now leaving flatland</h2> <p>(Be sure to visit our gift shop!)</p> <p>Much of the GLR discussion at APW2014 concerned flattening list context in Perl 6. Over the past few months and years Perl 6 has slowly but steadily reduced the number of functions and operators that flatten by default. In fact, a very recent (and profound) change occurred within the last couple of months, when the <code>.[]</code> subscript operator for Parcels switched from flattening to non-flattening. To illustrate the difference, the expression</p> <pre>(10,(11,12,13),(14,15)).[2]</pre> <p>previously would flatten out the elements to return 12, but now no longer flattens and produces <code>(14,15)</code>. As a related consequence, <code>.elems</code> no longer flattens either, changing from 6 to 3.</p> <p>Unfortunately, this change created a inconsistency between Parcels and Lists, because <code>.[]</code> and <code>.elems</code> on Lists continued to flatten. Since programmers often don’t know (or care) when they’re working with a Parcel or a List, the inconsistency was becoming a significant pain point. Other inconsistencies were increasing as well: some methods like <code>.sort</code>, <code>.pick</code>, and <code>.roll</code> have become non-flattening, while other methods like <code>.map</code>, <code>.grep</code>, and <code>.max</code> continue to flatten. There’s been no really good guideline to know or decide which should do which.</p> <p>Flattening behavior is great when you want it, which is a lot of the time. After all, that’s what Perl 5 does, and it’s a pretty popular language. But once a list is flattened it’s hard to get the original structure if you wanted that — flattening discards information.</p> <p>So, after many animated discussions, review of lots of code snippets, and seeking some level of consistency, the consensus on Perl 6 flattening behavior seems to be:</p> <div class="lispace"> <ul> <li>List assignment and the <code>[ ]</code> array constructor are unchanged; they continue to flatten their input elements. (Arrays are naturally flat.)</li> <li>The for statement is unchanged. <code>for @a,@b { ... }</code> flattens <code>@a,@b</code> and applies the block to each element of <code>@a</code> followed by each element of <code>@b</code>. Note that flattening can easily be suppressed by itemization, thus <code>for @a, $@b { ... }</code> flattens <code>@a</code> but does all of <code>@b</code> in a single iteration.</li> <li>Method calls tend to not flatten their invocant. This most impacts <code>.map</code>, <code>.grep</code>, and <code>.first</code>… the programmer will have to use <code>.flat.grep</code> and <code>.flat.first</code> to flatten the list invocant. Notably, <code>.map</code> will no longer flatten its invocant — a significant change — but we’re introducing <code>.for</code> as a shortcut for <code>.flat.map</code> to preserve a direct isomorphism with the <code>for</code> statement.There’s ongoing conjecture of creating an operator or syntax for flattening, likely a postfix of some sort, so that something like <code>.|grep</code> would be a convenient alternative to <code>.flat.grep</code>, but it doesn’t appear that decision needs to be made as part of the GLR itself.</li> <li>Argument lists continue to depend on the context in which they are bound: flattening for slurpy parameters, top-level itemizing for slice parameters, and non-flattening (or deferred flattening) for Positionals.</li> <li>The above two points produce a general guideline that method call invocants are generally not-flattened, while function call arguments are more likely to be. <pre>((1,2), 3, (4,5)).map({...}) # iterates over three elements map {...}, ((1,2),3,(4,5)) # iterates over five elements (@a, @b, @c).pick(1) # picks one of three arrays pick 1, @a, @b, @c # flatten arrays and pick one element </pre> </li> <li>We think it will be very difficult to have a guideline that applies 100% of the time — there will be a few exceptions to the rule but they should generally feel natural.</li> <li>The flattening behavior of operators continues to be specific to each operator — some will flatten, others will not. Fortunately, any flattening behavior should be grouped by precdence level, is generally dwimmy, and there are easy ways to use contextualizers to quickly switch to the behavior you want.</li> </ul> <h2>United Parcel Severance</h2> <p>As a result of improvements in flattening consistency and behavior, it appears that we can eliminate the Parcel type altogether. There was almost unanimous agreement and enthusiasm at this notion, as having both the Parcel and List types is quite confusing.</p> <p>Parcel was originally conceived for Perl 6 as a “hidden type” that programmers would rarely encounter, but it didn’t work out that way in practice. It’s nice that we may be able to hide it again — by eliminating it altogether. <img src="https://s.w.org/images/core/emoji/2.2.1/72x72/1f642.png" alt="🙂" class="wp-smiley" style="height: 1em; max-height: 1em;" /></p> <p>Thus <code>infix:<,></code> will now create Lists directly. It’s likely that comma-Lists will be immutable, at least in the initial implementation. Later we may relax that restriction, although immutability also provides some optimization benefits, and Jonathan points out that may help to implement fixed-size Arrays.</p> <p>Speaking of optimization, eliminating Parcel may be a big boost to performance, since Rakudo currently does a fair bit of converting Parcels to Lists and vice-versa, much of which goes away if everything is a List.</p> <h2>A few more times around the (loop) blocks</h2> <p>During a dinner discussion Jonathan reminded me that Synopsis 4 has all of the looping constructs as list generators, but Rakudo really only implements <code>for</code> at the moment. He also pointed out that if the loop generators are implemented, many functions that currently use <code>gather/take</code> could potentially use a loop instead, and this could be much more performant. After thinking on it a bit, I think Jonathan is on to something. For example, the code for <code>IO::Handle.lines()</code> currently does something like:</p> <pre>gather { until not $!PIO.eof { $!ins = $!ins + 1; take self.get; } }</pre> <p>With a lazy <code>while</code> generator, it could be written as</p> <pre>(while not $!PIO.eof { $!ins++; self.get });</pre> <p>This is lazily processed, but doesn’t involve any of the exception or continuation handling that <code>gather/take</code> requires. And since <code>while</code> might choose to not be strictly lazy, but <code>lines()</code> definitely should be, we may also use the <code>lazy</code> statement prefix:</p> <pre>lazy while not $!PIO.eof { $!ins++; self.get };</pre> <p>The <code>lazy</code> prefix tells the list returned from the <code>while</code> that it’s to generate as lazily as it possibly can, only returning the minimum number of elements needed to satisfy each request.</p> <p>So as part of the GLR, we’ll implement the lazy list forms of all of the looping constructs (<code>for</code>, <code>while</code>, <code>until</code>, <code>repeat</code>, <code>loop</code>). In the process I also plan to unify them under a single <code>LoopIter</code> type, which can avoid repetition and be heavily optimized.</p> <p>This new loop iterator pattern should also make it possible to improve performance of <code>for</code> statements when performed in sink context. Currently <code>for</code> statements always generate calls to <code>.map</code>, passing the body of the loop as a closure. But in sink context the block of a <code>for</code> statement could potentially be inlined. This is the way blocks in most other loops are currently generated. Inlining the block of the body could greatly increase performance of <code>for</code> loops in sink context (which are quite common).</p> <p>Many people are aware of the problem that constructs such as <code>for</code> and <code>map</code> aren’t “consuming” their input during processing. In other words, if you’re doing <code>.map</code> on a temporary list containing a million elements, the entire list stays around until all have been processed, which could eat up a lot of memory.</p> <p>Naive solutions to this problem just don’t work — they carry lots of nasty side effects related to binding that led us to design immutable Iterators. We reviewed a few of them at the hackathon, and came back to the immutable Iterator we have now as the correct one. Part of the problem is that the current implementation is a little “leaky”, so that references to temporary objects hang around longer than we’d like and these keep the “processed” elements alive. The new implementation will plug some of the leaks, and then some judicious management of temporaries ought to take care of the rest.</p> <h2>I’ve got a sinking feeling…</h2> <p>In the past year much work has been done to improve sink context to Rakudo, but I’ve never felt the implementation we have now is what we really want. For one, the current approach bloats the codegen by adding a call to <code>.sink</code> after every sink-context statement (i.e., most of them). Also, this only handles sink for the object returned by a Routine — the Routine itself has no way of knowing it’s being called in sink context such that it could optimize what it produces (and not bother to calculate or return a result).</p> <p>We’d really like each Routine to know when it’s being called in sink context. Perl 5 folks will instantly say “Hey, that’s <code>wantarray</code>!”, which we long ago determined <a href="http://faq.perl6.org/#want">isn’t generally feasible in Perl 6</a>.</p> <p>However, although a generalized <code>wantarray</code> is still out of reach, we <em>can</em> provide it for the limited case of detecting sink contexts that we’re generating now, since those are all statically determined. This means a Routine can check if it’s been called in sink context, and use that to select a different codepath or result. Jonathan speculates that the mechanism will be a flag in the callsite, and I further speculate the Routine will have a macro-like keyword to check that flag.</p> <p>Even with detecting context, we still want any objects returned by a Routine to have <code>.sink</code> invoked on them. Instead of generating code for this after each sink-level statement, we can do it as part of the general return handler for Routines; a Routine in sink context invokes <code>.sink</code> on the object it would’ve otherwise returned to the caller. This directly leads to other potential optimizations: we can avoid <code>.sink</code> on some objects altogether by checking their type, and the return handler probably doesn’t need to do any decontainerizing on the return value.</p> <p>As happy as I am to have discovered this way to pass sink context down into Routines, please don’t take this as opening an easy path to lots of other wantarray-like capabilities in Perl 6. There may be others, and we can look for them, but I believe sink context’s static nature (as well as the fact that a false negative generally isn’t harmful) makes it quite a special case.</p> <h2>The value of consistency</h2> <p>One area that has always been ambiguous in the Synopses is determining when various contextualizing methods must return a copy or are allowed to return <code>self</code>. For example, if I invoke <code>.values</code> on a List object, can I just return <code>self</code>, or must I return a clone that can be modified without affecting the original? What about <code>.list</code> and <code>.flat</code> on an already-flattened list?</p> <p>The ultra-safe answer here is probably to always return a copy… but that can leave us with a lot of (intermediate) copies being made and lying around. Always returning <code>self</code> leads to unwanted action-at-a-distance bugs.</p> <p>After discussion with Larry and Jonathan, I’ve decided that true contextualizers like <code>.list</code> and <code>.flat</code> are allowed to return <code>self</code>, but other method are generally obligated to return an independent object. This seems to work well for all of the methods I’ve considered thus far, and may be a general pattern that extends to contextualizers outside of the GLR.</p> <h2>Now it’s just a SMOPAD</h2> <p>(small matter of programming and documentation)</p> <p>The synopses — especially Synopsis 7 — have always been problematic in describing how lists work in Perl 6. The details given for lists have often been conjectural ideas that quickly prove to epic fail in practice. The last major list implementation was done in Summer 2010, and Synopsis 7 was supposed to be updated to reflect this design. However, the ongoing inconsistencies (that have led to the GLR) really precluded any meaningful update to the synopses.</p> <p>With the progress recently made at APW2014, I’m really comfortable about where the Great List Refactor is leading us. It won’t be a trivial effort; there will be significant rewrite and refactor of the current Rakudo codebase, most of which will have to be done in a branch. And of course we’ll have to do a lot of testing, not only of the Perl 6 test suite but also the impact on the module ecosystem. But now that much of the hard decisions have been made, we have a roadmap that I hope will enable most of the GLR to be complete and documented in the synopses by Thanksgiving 2014.</p> <p>Stay tuned.</p> </div> <h2><a href="http://blog.brentlaabs.com/2013/06/camelia-at-age-13-perl-6-on-jvm-debuts.html">Camelia at Age 13: Perl 6 on the JVM Debuts</h2></a> <p><h3>Published by Brent Laabs on 2013-06-22T01:23:00</h3></p> Perl 6 is now thirteen years old.  And she's very much a teenager in attitude, self-confident yet still growing up.  This contrasts with Javascript, which emerged from Brendan Eich's head, fully-formed like Athena -- but that only shared the Zeus-sized headaches with everyone until JQuery came along.<br /> <br /> But Camelia, as she is fondly referred to by fans of Perl 6, is growing up fast.  Both too fast, and not fast enough.  To some of the community, the prospect of major changes to the language is scary.  Perl 6 is trying all of these crazy new things -- invariant sigils, metaoperators, grammars. She's even doing subroutine signatures, because "all of her friends are doing it".<br /> <br /> They can't stay little children forever, you know.<br /> <br /> And teenagers are liable to do surprising things.  So it was, that this week we announced Rakudo Perl 6 now <a href="https://github.com/rakudo/rakudo/blob/nom/docs/announce/2013.06.md">runs on the Java Virtual Machine</a> (JVM).  It's not perfect yet, but 62% of the files in the spectest pass as of yesterday.  Given the rate things are progressing, I'm sure it's already at a higher pass percent.<br /> <br /> And yet, I'm sure there is no small number of you whose first thought about Perl on the JVM was "Heresy!".<br /> <br /> There are certainly good reasons to support this point of view.  Startup times are horrible at this early stage, still O(seconds), and much of that is JVM's overhead.  It has well known security issues.  And of course the major implementation is controlled by <span style="font-family: "Courier New",Courier,monospace;">$corporation</span>, who just wants to make money off it.  And why would we want to abandon open-source VMs?<br /> <br /> Still, there are plenty of good reasons for the port.  One is that the JVM is ubiquitous, and many devices have a copy pre-installed.  Most of Java's stability issues have been dealt with, and it serves Perl's competitors well enough through Jython and JRuby.  And it is well-documented, with bazillions of libraries (more than fooilions, anyway).  So we can finally address the longstanding desires of the community in things like sockets and threading, because we can tell the difference between our mistakes and those of the VM.<br /> <br /> Instead of thinking of Perl 5 as a "sister language", I like to think of it as Camelia's father instead.  A father that might be kind of upset that she brought "that kind of language" into our house.  But she has a mind of her own, and I'm afraid that this won't be the only boyfriend she'll be bringing home.  There is a <a href="http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/pmurias/9002">GSoC grant</a> to build a Javascript backend to Rakudo.  And <a href="https://github.com/sorear/niecza">Niecza Perl 6</a> already uses the .NET Runtime.<br /> <br /> However, Perl 6 is not abandoning open-source and C language implementations, either.  The <a href="http://6guts.wordpress.com/2013/05/31/moarvm-a-virtual-machine-for-nqp-and-rakudo/">announcement of MoarVM</a> shows that Perl 6 developers plan to develop a lightweight VM specifically for NQP and Rakudo.  C functions will be directly callable within Perl itself, with the <a href="https://github.com/jnthn/zavolaj/">NativeCall interface</a>. <br /> <br /> Now, if Parrot <a href="http://lists.parrot.org/pipermail/parrot-dev/2013-June/007489.html">fly off on its own course</a>, that's Parrot's call.  You know how these teenage relationships go -- this could end up in a blow-up on <a href="http://www.reddit.com/r/perl">the quad</a>, or just as easily turn into hot makeup coding.  What, you didn't think I was going to say something that would <a href="http://blog.schwern.net/2013/05/15/yapcna-2013-withdrawal/">violate the CoC</a>, did you?<br /> <br /> But Perl 6 is not done growing yet.  Camelia, like other teenagers, cares about fashion and wishes she had better threads.  And, once we get tuits, this is pretty high priority.  Because any modern language needs to live in the multi-core reality.  This is something that we can still design around,  that may not have recieved the same care ten years ago.  Many threading features are already baked into the language, like hyper operators and async blocks.<br /> <br /> So I view the debut of the JVM port as Rakudo's real début, as with a debutante.  A treceañera, if you will.  I guess, given that she's 13, maybe it's a Bar Mitzvah -- except that she's not a boy, she's a butterfly.  But this is a chance acknowledge Perl 6's presence in the language scene.  Of course, these coming-of-age ceremonies don't mean the teenager is truly grown up yet.  <br /> <br /> But grow up she will, and faster than some of you might like.  Perl 6 is rebellious, and changes things that those in her father's Perl 5 community don't understand.  But if you talk to the pumpkings, they sincerly hope that Camelia doesn't turn out exactly like her father.<br /> <br /> After all, do we want keep the ithreads model?  Do we want to modules that dig into the compiler internals like XS does?  Perl 5 isn't perfect, we are just accustomed to its particular idiosyncrasies.<br /> <br /> But for all that Perl 6 is different, she still loves her father.  We still have sigils, classes, @_ in subs (if you still want it), P5-style regexes, modules, and TIMTOWTDI.  It's still Perl.  Moreover, there are at least two efforts to run Perl 5 code in Perl 6 -- the <a href="https://github.com/rakudo-p5/v5">v5 grammar/compiler</a> in Perl 6, and access to libperl from MoarVM.  So the sky isn't falling on compatibility.<br /> <br /> Nor is the other extreme true: Perl 6 development is in fact moving forward at a fast pace.  We know that Perl 6 is late.  Very late.  So late, in fact, that it's her father that's going to <a href="http://shadow.cat/blog/matt-s-trout/pumpkin-perl-breakdown/">turn into a Pumpkin</a>.  But when Perl 6 finally comes of age -- sooner than you think -- it will be something that the Perl community will be proud of.<br /> <br /> And I apologize in advance for anything Camelia does in her bratty teenage years.<br /> <br /> <h2><a href="http://blog.brentlaabs.com/2013/06/thanking-perl-community-for-awesome-yapc.html">Thanking the Perl Community for an Awesome YAPC</h2></a> <p><h3>Published by Brent Laabs on 2013-06-06T09:37:00</h3></p> My first time at <a href="http://yapcna.org/">YAPC::NA</a> was too incredible for words. That said, because this is a blog, I'm going to have to put it in words anyway.<br /> <br /> So, about a month ago, I didn't even know that I was even going to YAPC.  I was just talking to raiph++ in an IRC back channel, when he asked me if I was going to the conference.  I said that it would be fun, but I didn't really have money to go.  Being unemployed means lots of free time for hacking, but not so much free money for going places.<br /> <br /> Well, raiph told diakopter++, who asked if I could be willing to go, if he found funds.  I responded, "Of course, if you think it's possible."  I soon went to sleep, and twelve hours later, I had a plane ticket to Austin in my inbox courtesy of The Perl Foundation.   So just like <a href="http://blogs.perl.org/users/peter_rabbitson/2013/05/on-the-awesomeness-of-the-perl-community.html">Peter Rabbitson's case</a>, the Perl community eagerly gave me a chance to attend.  So thank you to all 440 attendees, and all of the sponsors for your own personal contribution to my attendance.  Even though I'm new, you all gave a me a chance to participate in the community, and for that I am grateful.<br /> <br /> And what a community it is.  I've long known that Perl programmers were a little strange.  Naturally, I fit right in.<br /> <br /> The conference itself had quite a fun and informative series of talks.  More often than not, I had two or more that I wanted to attend at the same time.  For the most part, I stuck to the Perl 6 "track", where most of my work has been so far.  After all, it's not often that so many of the European contingent make a trip to our humble continent, so I was eager to spend time with them.<br /> <br /> No one warned me that jnthn++ has a tendency to spring wild new features on us at YAPCs.  <a href="http://www.yapcna.org/yn2013/talk/4657">Reversible grammars</a>, seriously?!  I'm still trying to wrap my head around that one.  The announcement of <a href="https://github.com/moarvm/moarvm">MoarVM</a> was equally exciting, as it offers us a chance to start fresh with everything we learned about Perl 6 and virtual machines in the last five years.<br /> <br /> So I have to say diakopter++ once again.  Besides introducing Moar in his talk, Matthew Wilson was constantly busy behind the scenes, making sure that everything ran smoothly the entire conference.  I think the man must be buying tuits on the black market.<br /> <br /> YAPC also helped immensely in my hunt for a job.  The job fair brought me several contacts, and the talks helped me learn which skills I'll really need to learn in those jobs.  Person-to-person contact offers so much more in truly understanding the state of the language, and of what projects are of the greatest use right now.<br /> <br /> Truly, Perl's community is it's greatest strength.  <br /> <br /> It's the community that keeps Perl vital.  After seeing YAPC for myself, the whole "Perl is Dead" meme seems entirely baseless.  Conference attendance was up 19% over last year, which was the previous record high.  Perl feels like a growing language, with lots of experiments in how to revitalize the now 25-year-old syntax with p2, moe, and Perl 6.<br /> <br /> The community keeps Perl relevant.  While it may not be the sole alternative to bash scripts like it once was, it is used for enterprise and homebrew projects alike, from the stock exchange to the surface of Mars.  Projects like DBIx, Moose, and Dancer provide modern frameworks to acomplish more with less work.<br /> <br /> The community keeps Perl open.  No one seemed to be afraid to say what they felt, on CGI.pm or anything else, but everyone remained civil.  <a href="http://shadow.cat/blog/matt-s-trout/hwppna-2013/">Hallway++</a> is a great social hack to get everyone to feel comfortable talking to each other.  So when I found myself sitting across from TimToady, instead of being intimidated as a newbie, I had a great conversation with him about supervolcanoes and nonverbal Japanese language.<br /> <br /> And the community really wants <i>all</i> of the projects to succeed.  I spent a lot of time at non-profit and political events in the past, where we were all theoretically working for a common cause.  And yet scheming, conflict, and political maneuvering were inevitable.  But in Perl, where we actually have multiple implementations and modules competing for mindshare and tuits, people cheer for everything to succeed.  No one fights each other or rages against $language_of_the_week stealing our users, for the real enemy is the lack of tuits.<br /> <br /> I overheard this at dinner last night, from a fellow first-time attendee:<br /> <blockquote> "I'm just happy that the two of you liked my work." -- vanstyn</blockquote> Although he was talking about DBIx, I think that captures the spirit of conference as a whole.  All of us here -- from the n00bs to the pumpkings -- want to share our work and make something useful for others.  It's not an organization where we wait for pronouncements from on high, but one where users create endless variations and share them.  Not an organization so much as a family.<br /> <br /> During TimToady's epistle/speech to the community, he said something like:<br /> <blockquote class="tr_bq"> "We have faith, hope, and love, but the most <i>awesome</i> of these is love." -- Larry Wall</blockquote> A line like this might seem a bit hokey out of context, but it was actually moving when I heard it.  We have faith that we can use Perl to solve our problems.  We have hope that Perl 5 and 6 will continue to get better. And we love Perl, unconditionally, despite all of her flaws.  And as Wil Wheaton says <a href="http://www.youtube.com/watch?feature=player_embedded&v=H_BtmV4JRSc">about us geeks</a>, we just want to love our special thing the best we can, and go the extra mile to share it with others.<br /> <br /> I just want to say that I love the Perl community right back.  You went out of your way to include me and all the other newcomers.  You all gave me all a chance to learn, play, and code with you -- and to be part of your community -- and I am so glad you did. <h2><a href="http://pmthium.com/2013/06/a-perl-6-developers-reply-to-a-russian-perl-podcast/">A Perl 6 developer’s reply to a Russian Perl Podcast</h2></a> <p><h3>Published by pmichaud on 2013-06-03T20:13:53</h3></p> <p>[This is a response to the <a href="http://yapp.rpod.ru/301370.html">Russian Perl Podcast</a> <a href="https://gist.github.com/ribasushi/5670006">transcribed</a> by Peter Rabbitson and discussed at <a href="http://blogs.perl.org/users/peter_rabbitson/2013/05/perlx-sub-communities-etc.html">blogs.perl.org</a>.]</p> <p>I found this translation and podcast to be interesting and useful, thanks to all who put it together.</p> <p>Since there seems to have been some disappointment that Perl 6 developers didn’t join in the discussions about “Perl 7” earlier this year, and in the podcast I’m specifically mentioned by name, I thought I’d go ahead and comment now and try to improve the record a bit.</p> <p>While I can’t speak for the other Perl 6 developers, in my case I didn’t contribute to the discussion because nearly all the things I would’ve said were already being said better by others such as Larry, rjbs, mst, chromatic, etc. I think a “Perl 7” rebrand is the wrong approach, for exactly the reasons they give.</p> <p>A couple of statements in the podcast refer to “hurting the feelings of Perl 6 developers” as being a problem resulting from a rebrand to Perl 7. I greatly appreciate that people are concerned with the possible impact of a Perl 5 rebrand on Perl 6 developers and our progress. I believe that Perl 6’s success or failure at this point will have little to do with the fact that “6 is larger than 5”. I don’t find the basic notion of “Perl 7” offensive or directly threatening to Perl 6.</p> <p>But I fully agree with <a href="http://shadow.cat/blog/matt-s-trout/names-and-numbers/">mst</a> that “you can’t … have two successive numbers in two brands and not expect people to be confused.” We already have problems explaining “5” and “6” — adding more small integers to the explanation would just make an existing problem even worse, and wouldn’t do anything to address the fundamental problems Perl 6 was intended to resolve.</p> <p>Since respected voices in the community were already saying the things I thought about the name “Perl 7”, I felt that adding my voice to that chorus could only be more distracting than helpful to the discussion. My involvement would inject speculations on the motivations of Perl 6 developers into what is properly a discussion about how to promote progress with Perl 5. I suspect that other Perl 6 developers independently arrived at similar conclusions and kept silent as well (Larry being a notable exception).</p> <p>I’d also like to remark on a couple of @sharifulin’s comments in the podcast (acknowledging that the transcribed comments may be imprecise in the translation from Russian):</p> <p>First, I’m absolutely not the “sole developer” of Perl 6 (13:23 in the podcast), or even the sole developer of Rakudo Perl 6. Frankly I think it’s hugely disrespectful to so flippantly ignore the contributions of others in the Perl 6 development community. Let’s put some actual facts into this discussion… in the past twelve months there have been over 6,500 commits from over 70 committers to the various Perl 6 related repositories (excluding module repositories), less than 4% (218) of those commits are from me. Take a look at the author lists from the Perl 6 commit logs and you may be a little surprised at some of the people you find listed there.</p> <p>Second, there is not any sense in which I think that clicking “Like” on a Facebook posting could be considered “admitting defeat” (13:39 in the podcast). For one, my “Like” was actually liking rjbs’ reply to mst’s proposal, as correctly noted in the footnotes (thanks Peter!).</p> <p>But more importantly, I just don’t believe that Perl 5 and Perl 6 are in a battle that requires there to be a conquerer, a vanquished, or an admission of defeat.</p> <p>Pm</p> <h2><a href="http://blog.brentlaabs.com/2013/05/porting-module-to-perl-6.html">Porting a Module to Perl 6</h2></a> <p><h3>Published by Brent Laabs on 2013-05-06T02:43:00</h3></p> <a href="http://search.cpan.org/">CPAN</a> is a huge draw for Perl 5, with approximately umpteen zillion modules available for a wide arrangement of purposes.  It's probably the biggest draw for the Perl 5 language these days, given the newer, hipper scripting languages out there like Ruby, Python, and of course <a href="http://www.intercaloninterstates.org/">INTERCAL</a>.<br /> <br /> The problem is, these modules are not yet usable in Perl 6 directly.  There is an ongoing project to allow <a href="https://github.com/rakudo-p5/v5">Perl 5 code to run in Rakudo</a>, but so far only the most basic code works: like basic loops, quite a few builtins, backticks, etc.  It does inherit from the Perl 6 object system, which is pretty cool, so <code>$foo->WHAT</code> can tell you if you have a <code>Str</code>, <code>Int</code>, or <code>IO::Handle</code>.<br /> <br /> So for right now, the only practical way to use Perl 5 modules is to rewrite them in Perl 6.  I just finished porting the <a href="https://github.com/FROGGS/p6-File-Spec">File::Spec module</a>, one of Perl 5's core modules, to help deal with file paths on different operating systems. FROGGS++ did much of the initial work on it, but he's moved on the P5 in P6 project mentioned above, so I picked up the slack. The end goal of the project is for me to integrate functionality like Perl 5's Path::Class into the core language, so that OS interoperability comes naturally when using the native functions.<br /> <br /> As I got further into the port, I have been convinced that porting the module is a <i>much</i> better choice than relying on the Perl 5 code being integrated.  There are several reasons for this:<br /> <br /> <h4> Code Cruft</h4> <br /> There is a lot of support for operating systems that are now out of date.  This isn't a bad thing.  I'm sure that there's some hobbyist who will want to run Perl 6 on their OS/2 Warp system.  The problem comes when you look inside the code for the OS2 module:<br /> <code>    $path =~ s/^([a-z]:)/\l$1/s;</code><br /> This little no-op snippet from canonpath (to produce the canonically correct path) converts a lowercase drive letter to lowercase.  It's not harmful, but it does illustrate the fact that no one has edited this code in 9 years.<br /> <br /> This isn't the fault of the Perl 5 Porters -- they have plenty of better things to do than to support outdated OSes when not even bug tickets are coming in.  But translating the code sure gives a great opportunity to notice these problems.<br /> <br /> In the end, I ended up cutting the entire OS2 module and delegating to Win32.pm, because it had support for things like UNC paths (<code>//server/share</code>) that OS2.pm had only half-implemented.  And so a huge block of code cruft bit the dust.<br /> <br /> <h4> Readability and Maintainability</h4> <br /> Part of the reason these issues happen in the first place is that it's harder to see what's going on in a given piece of code.<br /> <br /> An example I came across was in this helper for tmpdir, a method to return the first temporary directory that's writable in a list of parameters.  In Perl 5, we get:<br /> <br /> <code>sub _tmpdir {<br />     my $self = shift;<br />     my @dirlist = @_;<br />     my $tmpdir;<br /> <br />     foreach (@dirlist) {<br />     next unless defined && -d && -w _;<br />     $tmpdir = $_;<br />     last;<br />     }<br />     return $self->canonpath($tmpdir);<br /> }</code><br /> <br /> That's actually good, idiomatic code for Perl 5, though it can look like spooky action at a distance if you're not aware of what's going on with <code>$_</code>, <code>@_</code>, and <code>shift</code>.<br /> <br /> Equivalent code in Perl 6 looks like this:<br /> <br /> <code>method !tmpdir( *@dirlist ) {<br />     my $tmpdir = first { .defined && .IO.w && .IO.d }, @dirlist;<br />     return self.canonpath($tmpdir);<br /> }</code><br /> <br /> No messing about with parameters and keeping track of the object -- it all happens in the signature.  You no longer have to read through a loop to understand the code either -- in Perl 6 you can just say that you want the first matching candidate, and first() will lazily test the list for you.<br /> <br /> The P6 version gets to the point much faster, and it's much closer to natural language: "set <code>$tmpdir</code> to the first defined writable directory in <code>@dirlist</code>."  Less, easier to read code is easier to maintain.<br /> <br /> <h4> Changing Old Features</h4> <br /> At some point, your code was working perfectly and passes all the tests.  But then the computer world changes around you, and it no longer makes any sense.  And you would like to refactor, but people rely on the old functionality.<br /> <br /> This is exactly what happened for File::Spec's case_tolerant function.  It essentially looks at the operating system alone, and uses that to determine if the filesystem is case-sensitive.  Which in the old days made perfect sense when Macs used HFS+, Windows used FAT, and Unix used ufs or a variant.  But my computer runs Mac OS X and Windows and has several drive partitions in different formats.  Heck, the NTFS drives are case sensitive in POSIX-land, but as soon as I boot Windows they become case insensitive.<br /> <br /> The only reasonable way to check this now is to actually check the filesystem for a specific directory, given widespread support for symlinks.  This breaks the old functionality.  But there's no time like a major language revision to break old APIs and replace them with shiny new ones.<br /> <br /> However, there are a couple of major downsides to porting:<br /> <br /> <h4> This is really time-consuming</h4> <br /> Sure, you don't have to implement the algorithm from scratch, and you have plenty of tests to help your development.  It would be possible to just translate the existing code, because things aren't that different.  Change an <code>if( $foo )</code> to <code>if $foo</code>, etc.<br /> <br /> However, a major reason for doing the porting is to use the Perl 6 idioms instead, especially in function declarations and regular expressions where it makes a major difference in code readability.<br /> <br /> <h4> Dependencies aren't available</h4> <br /> Sometimes your code relies on separate modules not available, or on not yet implemented functions.  Your choice becomes to either implement the functionality yourself and embark on yet another yak-shaving expedition, or mark it as todo and wait for the appropriate functionality to arrive.<br /> <br /> This has become a much smaller problem as of late as the core language matures.  But "done enough" is not really "done".<br /> <br /> <hr /> <br /> Now that I've written this, I've realized that my own project is a microcosm of the Perl 6 saga.  Making a better codebase takes a lot of time, but it ultimately seems worth the effort.<br /> <br /> Of course, once I had gotten this far, I realized that File::Spec -- or something very much like it -- would be needed to implement IO::Path objects for non-unixlike OSes.  So stay tuned for the next part in this saga: How to add File::Spec to Rakudo.<br /> <br /> <b>Update</b>: It ended up turning into two posts:  One was a simple guide on <a href="http://blog.brentlaabs.com/2013/05/how-to-start-hacking-on-rakudo-perl-6.html">How to Start Hacking Rakudo Perl 6</a>, and the other covered my follies in trying to add to the compiler for the first time.  But the short story is that IO::Path is now added to Perl 6 and <a href="http://blog.brentlaabs.com/2013/05/implementing-iopath-in-rakudo.html">implemented in Rakudo</a> -- this means that both File::Spec and Path::Class' behavior are now available in the core language without adding modules. <h2><a href="http://blog.brentlaabs.com/2013/05/how-to-start-hacking-on-rakudo-perl-6.html">How to start hacking on Rakudo Perl 6</h2></a> <p><h3>Published by Brent Laabs on 2013-05-08T01:39:00</h3></p> In the course of writing modules, I finally got the urge to start implementing features I wanted in Rakudo itself.  And since there wasn't a real guide on how to set up and patch Rakudo, I decided to share what I had learned in the process.<br /> <br /> The nice thing about Perl 6 implementations is that as significant portion of them is written in Perl 6.  (Well, one nice thing anyway.)  This means that if you're comfortable writing Perl 6 modules and classes, you should feel pretty much at home in the source.<br /> <br /> This guide assumes so, and that you have a basic familiarity with Github, <code>git</code>, and <code>make</code> -- enough to commit to repositories and build a software package, anyway.<br /> <br /> <h3> Getting Started</h3> This first thing is to get your own branch of Rakudo to work on.  So go to the <a href="https://github.com/rakudo/rakudo">Rakudo repository</a> and click the fork button in the upper right.  Relax while Github <a href="https://assets.github.com/images/spinners/hardcore-forking.gif">photocopies a book</a>.  Once that's done, find an appropriate directory to <code>git clone</code> it to on your own machine.<br /> <br /> Go ahead and cd into the new <code>rakudo</code> directory.  There are a few setup things that you'll want to do.  First of all, go ahead and build Rakudo, using the normal steps:<br /> <code>    perl ./Configure.pl --gen-parrot<br />     make<br />     make install</code><br /> That will pull a copy of NQP and Parrot, and make sure that everything is working okay to begin with.  Now that that's done, you'll want to add the new perl6 to your system <code>$PATH</code> environment variable.   Which, if you don't know how to do it -- well <a href="https://www.google.com/search?q=how+do+i+add+to+path+environment">here's Google</a>.  In particular, you'll need to add the full path to the <code>rakudo/install/bin</code> directory.<br /> <br /> There's a couple more things you'll want to do now.  First of all:<br /> <code>    make spectest</code><br /> You don't have to run the full tests now, but let it download the roast repository into your <code>t/spec</code> before hitting ^C.  You will need these tests later to make sure you didn't break anything.<br /> <br /> Next, you'll want to set up a link back to the main Rakudo repository, so you can pull changes from there.  So do:<br /> <code>    git remote add upstream git://github.com/rakudo/rakudo.git</code><br /> <br /> You'll also want the module installer, Panda.  Now, obviously, you shouldn't add anything to Rakudo that depends on an outside module.  But Panda is the one piece of software you really don't want to break, ever.  People will still want to be able to download modules even if functionality changes.  We will have to go through a deprecation cycle if you intentionally change something to cause Panda to start failing its tests.  So to download and install it:<br /> <code>    git clone git://github.com/tadzik/panda.git<br />     cd panda<br />     perl6 bootstrap.pl</code><br /> This will set up Panda's dependencies, and test all of those modules.  The bootstrap script will tell you a path to add to your $PATH environment variable -- add it too, so that panda will run from anywhere.<br /> <br /> Finally, you really should set up a new branch to work on, so you can switch back to a working Rakudo if you need to.  Move back into the rakudo directory and run:<br /> <code>    git checkout -b mynewbranchname</code><br /> <br /> <h3> A very short overview of the source</h3> <br /> Now that all the setup is done, let's take a quick look around.  Most of what we build into Perl 6 lives in the <code>rakudo/src</code> folder, so this is where you'll want to edit the contents.<br /> <ul> <li>The <code>vm</code> directory contains files specific to the virtual machines Rakudo runs under.  At this time of this writing, there's only one thing in there, <code>parrot</code>, but very soon there will also be a <code>jvm</code> directory.  Exciting!  Most of the purpose of this code is to map functions to lower-level operations, in either Parrot or Java.</li> <li>The <code>Perl6</code> directory contains the grammar and actions used to build the language, as well as the object metamodel.  The contents of this folder are written in NQP, or Not Quite Perl.  This section determines how the language is parsed.</li> <li>The <code>core</code> directory contains the files that will be built into the core setting.  You'll find classes or subroutines in here for just about everything in Perl: strings, operators like <code>eq</code>, filehandles, sets, and more.  Individual files look similar to modules, but these are "modules" that are available to every Perl 6 program.</li> <li>The <code>gen</code> directory contains files that are created in the compliation process.  The core setting lives here, creatively named <code>CORE.setting</code>.  And if you look at it, it's just a concatenation of the files in <code>core</code>, put together in the order specified in <code>rakudo/tools/build/Makefile.in</code>.  While these files can and do get overwritten in the build process, it's often a good idea to keep a copy of CORE.setting open so you can find what you're looking for faster -- and then go edit it in <code>core</code>.</li> </ul> <br /> <h3> Let's start hacking!</h3> <br /> Now's the time to start changing Rakudo.  Have the appropriate amount of fun!  Be sure to commit functioning changes occasionally, so that you can <code>git bisect</code> for problems later.  And push your edits to Github as a free backup.  If you get stuck, drop by #perl6 on irc.freenode.net and ask questions.<br /> <br /> If it's your first time, you have to fi^W^W^W^W you will probably make a lot of mistakes.  I know I did on my first project, as <a href="http://blog.brentlaabs.com/2013/05/implementing-iopath-in-rakudo.html">explained in detail</a> in a previous post.   But I promise you, the learning curve is surprisingly easy, and your compiler-fu will increase to fuchsia-belt level in no time.  (What?  We're not just giving black belts away... and Camelia likes fuchsias.) <br /> <br /> <h3> Testing and Specs</h3> <br /> When you think you're finished with your code, the first thing you should do is merge in the upstream rakudo, and rebuild:<br /> <code>    git fetch upstream<br />     git merge upstream/nom<br />     perl Configure.pl<br />     make<br />     make spectest</code><br /> The spectests will make sure that you didn't accidentally break the codebase.  You should pass, or at least not fail worse than the <a href="https://github.com/coke/perl6-roast-data">current roast data</a>.<br /> <br /> You should add your own tests into the <a href="https://github.com/perl6/roast/">roast repository</a> about now.  You do have unit tests, right?   Writing tests is "optional", just like brushing your teeth -- you don't have to do it, but if you never do it you're in for a lot of pain later.  Here's a fine and elegantly crafted hyperlink to <a href="http://perlcabal.org/syn/S24.html">S24 (Testing)</a> for reference. <br /> <br /> When editing a file that already exists in roast, you may need to fudge the tests for Niecza and Pugs.  This tells us "we know the test failed or fails to parse, nothing has changed".  Just add lines like the following above broken tests:<br /> <code>    #?pugs 1 skip 'reason'<br />     #?niecza 1 skip 'reason'</code><br /> The "1" is actually the number of tests you want to skip, but really, look at the README in roast for more details.<br /> <br /> If you want to add a whole new test file, you'll need to add it into <code>rakudo/t/spectest.data</code>.  If your code fixes broken tests, then you'll want to *unfudge* by removing the <code>#?rakudo skip</code> lines above the relevant tests.<br /> <br /> You should also test that Panda is still working.  Since you'll have to rebuild panda after recompling Rakudo anyway, just check the rebootstrap for test failures:<br /> <code>    perl6 panda/rebootstrap.pl</code><br /> <br /> <h3> Commiting to Rakudo</h3> <br /> The easiest way to get your code merged is to push it back to Github, and then send a pull request into Rakudo.  If you're really committed to committing, consider sending in a <a href="http://www.perlfoundation.org/legal">Contributor License Agreement</a> to The Perl Foundation.  This makes you eligible for a commit bit to push directly to the Rakudo repo.<br /> <br /> If there's a problem, someone will get back to you pretty fast on the Github issues page.  Hopefully, these problems will be easy to fix, and a standard <code>git commit; git push</code> will add it to the ticket.  If there aren't any problems, someone will just merge it in a couple days.<br /> <br /> Huzzah! \o/  A Rakudo Hacker is you! <h2><a href="http://pmthium.com/2012/09/a-rakudo-performance/">A Rakudo Performance</h2></a> <p><h3>Published by pmichaud on 2012-09-02T23:00:26</h3></p> <p>At YAPC::NA 2012 in Madison, WI I gave a <a href="http://en.wikipedia.org/wiki/Lightning_talk">lightning talk</a> about basic improvements in Rakudo’s performance over the past couple of years. Earlier today the <a href="http://www.youtube.com/watch?v=ILWrbvI8Qfg">video</a> of the <a href="http://act.yapcna.org/2012/talk/22">lightning talks session</a> appeared on YouTube; I’ve clipped out my talk from the session into a separate video below. Enjoy!</p> <p><iframe width="640" height="480" src="http://www.youtube.com/embed/tCel6lpDTZI?feature=oembed" frameborder="0" allowfullscreen></iframe></p> <p> </p> <h2><a href="http://pmthium.com/2012/05/roborama-2012a/">Roborama 2012a</h2></a> <p><h3>Published by pmichaud on 2012-05-28T06:08:52</h3></p> <p>A couple of weeks ago I entered the <a href="http://www.dprg.org/">Dallas Personal Robotics Group</a> <a href="http://www.dprg.org/competitions/roborama2012a/">Roborama 2012a competition</a>, and managed to come away with first place in the RoboColumbus event and Line Following event (Senior Level). For my robot I used one of the <a href="http://mindstorms.lego.com/">LEGO Mindstorms</a> sets that we’ve been acquiring for use by our <a href="http://modelscoutrobotics.org/">First Lego League team</a>, along with various 3rd party sensors.</p> <p>The goal of the RoboColumbus event was to build a robot that could navigate from a starting point to an ending point placed as far apart as possible; robots are scored on distance to the target when the robot stops. If multiple robots touch the finish marker (i.e., distance zero), then the time needed to complete the course determines the rankings. This year’s event was in a long hall with the target marked by an orange traffic cone.</p> <div id="attachment_240" style="width: 310px" class="wp-caption alignright"><a href="http://pmthium.com/wp/wp-content/uploads/2012/05/img-20120528-002134.jpg"><img class="size-medium wp-image-240" title="img-20120528-002134" src="http://pmthium.com/wp/wp-content/uploads/2012/05/img-20120528-002134-300x225.jpg" alt="HiTechnic IR ball and IRSeeker" width="300" height="225" srcset="http://pmthium.com/wp/wp-content/uploads/2012/05/img-20120528-002134-300x225.jpg 300w, http://pmthium.com/wp/wp-content/uploads/2012/05/img-20120528-002134-200x150.jpg 200w, http://pmthium.com/wp/wp-content/uploads/2012/05/img-20120528-002134-640x480.jpg 640w, http://pmthium.com/wp/wp-content/uploads/2012/05/img-20120528-002134.jpg 721w" sizes="(max-width: 300px) 100vw, 300px" /></a><p class="wp-caption-text">HiTechnic IR ball and IRSeeker sensor</p></div> <p>Contestants are allowed to make minor modifications to the course to aid navigation, so I equipped my robot with a <a href="http://www.hitechnic.com/cgi-bin/commerce.cgi?preadd=action&key=NSK1042">HiTechnic IRSeeker sensor</a> and put an <a href="http://www.hitechnic.com/cgi-bin/commerce.cgi?preadd=action&key=IRB1005">infrared (IR) electronic ball</a> on top of the traffic cone. The IRSeeker sensor reports the relative direction to the ball (in multiples of 30 degrees), so the robot simply traveled forward until the sensor picked up the IR signal, then used the IR to home in on the traffic cone. You can see the results of the winning run in the video below, especially around the 0:33 mark when the robot makes its first significant IR correction:</p> <p>http://youtu.be/x1GvpYAArfY</p> <p>My first two runs of RoboColumbus didn’t do nearly as well; the robot kept curving to the right for a variety of reasons, and so it never got a lock on the IR ball. Some quick program changes at the contest and adjustments to the starting direction finally made for the winning run.</p> <p>For the Line Following contest, the course consisted of white vinyl tiles with electrical tape in various patterns, including line gaps and sharp angles. I used a <a href="http://mindsensors.com/index.php?module=pagemaster&PAGE_user_op=view_page&PAGE_id=111">LineLeader sensor</a> from <a href="http://mindsensors.com/">mindsensors.com</a> for basic line following, with some heuristics for handling the gap conditions. The robot performed fine on my test tiles at home, but had difficulty with the “gap S curve” tiles used at the contest. However, my robot was the only one that successfully navigated the right angle turns, so I still ended up with first place. <img src="https://s.w.org/images/core/emoji/2.2.1/72x72/1f642.png" alt="🙂" class="wp-smiley" style="height: 1em; max-height: 1em;" /></p> <p><iframe width="640" height="360" src="http://www.youtube.com/embed/BF1ElplYi74?feature=oembed" frameborder="0" allowfullscreen></iframe></p> <p>Matthew and Anthony from our <a href="http://modelscoutrobotics.org/">FLL robotics team</a> also <a href="http://modelscoutrobotics.org/roborama2012a/">won other events</a> in the contest, and there are <a href="http://www.youtube.com/playlist?list=PL99FB10840A82C950">more</a> <a href="http://www.youtube.com/playlist?list=PL50F30261D8CDCAC5">videos</a> and <a href="http://www.flickr.com/photos/steevithak/sets/72157629871605390/">photos</a> available. The contest was a huge amount of fun and I’m already working on new robot designs for the next competition.</p> <p>Many thanks to DPRG and the contest sponsors for putting on a great competition!</p> <p> </p> <hr /> <script data-cfasync="false" src="/cdn-cgi/scripts/5c5dd728/cloudflare-static/email-decode.min.js"></script></body> </html>