Can Raku replace HTML?

In my last post, I listed three recent posts that got me thinking about Raku and HTML. I wondered if two of these could be used together to streamline the composition of web sites.

Act #1 – LPQ

Is drawn from a great idea of gfldex – Low Profile Quoting. Here’s my interpretation:

method init-qform() {
    my $css = q:to/END/;
    #demoFont {
    font-size: 16px;
    color: #ff0000;
    }
    END
        
    my $size = <40>;
    my $pattern = <[a-zA-Z0-9 ]+>;
        
    my $html = §<html>(§<head>(§<title>()),
        §<style>(:type<text/css>, $css),
        §<body>(
            §<form>(:action<.action>, :method<post>,
                §<p>('Your Name (required)'),
                §<input>(:type<text>, :required, :name<cf-name>, 
                                      :value<.cf-name>, :$size, :$pattern,),

                §<p>('Your Email (required)'),    #email type validates input
                §<input>(:type<email>, :required, :name<cf-email>, 
                                       :value<.cf-email>, :$size,),
 
                §<p>('Your Subject (required)'),
                §<input>(:type<text>, :required, :name<cf-subject>, 
                                      :value<.cf-subject>, :$size, :$pattern,),

                §<p>(:id<demoFont>, 'Your Message (required)'),
                §<p>(§<textarea>(:rows<10>, :cols<35>, :required, 
                                      :name<cf-message>, '<.cf-message>', ),),

                §<input>(:type<submit>, :name<cf-submitted>, :value<Send>,),
            )
        )
    );
    spurt "templates/qform.crotmp", pretty-print-html($html);
}

In the words of the originator “While casting my Raku spells I once again had felt the urge for a simply but convenient way to inline fragments of html in code. The language leans itself to the task with colon pairs and slurpy arrays.

The full code is available at https://github.com/p6steve/CroTemplateTest for your perusal. Here we have configured Raku with a §<html> shortcut that replaces the usual HTML <open attr=”value”>payload</close> tags. (The syntax magic is that ‘§’ is defined as a Class with Associative accessor.)

So what can this do for me?

  • express HTML components within a richer logical context
  • reduces the impedance of forced separation of component logic
  • tags are now function calls – so no more open/close boilerplate
  • the smooth Raku attribute syntax … :name<value> is used
  • variables ($size, $pattern) help you to DRY
  • it works with css

Act #2 – Cro

BUT – how can Act #1 co-exist with the Cro::WebApp::Template concepts? Sharp eyed readers may have noticed that the HTML above has a couple of examples of that already:

  • :value<.cf-email> … places $context.cf-email variable in attribute
  • ‘<.cf-message>’, … places $context.cf-message variable in payload

I thoroughly recommend the curious reader to review the Raku Cro services documentation

So the above init-qform method generates this .crotmp code:

<!DOCTYPE html>
<html>
<head>
<title>
</title>
</head>
<style type="text/css">#demoFont {
font-size: 16px;
color: #ff0000;
}
</style>
<body>
<form action="<.action>" method="post">
<p>Your Name (required)</p>
<input type="text" size="40" required pattern="[a-zA-Z0-9 ]+" value="<.cf-name>" name="cf-name" />
<p>Your Email (required)</p>
<input size="40" required value="<.cf-email>" type="email" name="cf-email" />
<p>Your Subject (required)</p>
<input value="<.cf-subject>" required name="cf-subject" type="text" pattern="[a-zA-Z0-9 ]+" size="40" />
<p id="demoFont">Your Message (required)</p>
<p>
<textarea name="cf-message" rows="10" required cols="35">
<.cf-message>
</textarea>
</p>
<input name="cf-submitted" type="submit" value="Send" />
</form>
</body>
</html>

Then we can set up a context:

class Context {
    has $.action = 'mailto:you@p6steve.com';
    has $.cf-name = 'p6steve';
    has $.cf-email = 'me@p6steve.com';
    has $.cf-subject = 'Raku does HTML';
    has $.cf-message = 'Describe some of your feelings about this...';
}

And apply the context to process the template in a Cro::Routes files;

use Cro::HTTP::Router;
use Cro::WebApp::Template;
use Cro::TemplateTest::Workshop;

my Workshop $ws = Workshop.new;

sub routes() is export {
    route {
        get -> 'qform' {
            my $context = $ws.context;
            template 'templates/qform.crotmp', $context;
        }
    }
}

Best of Both

This post illustrates how Raku can combine detailed syntax control to smoothly embed HTML within code logic. This helps to refactor awkward syntax islands so that the underlying problem-solution logic can be encapsulated and clearly expressed, It demonstrated the practical combination of the Cro template language with innate Raku power-of-expression to drive more comprehensible, consistent and maintainable code.

Comments and feedback very welcome…

~p6steve

Doing Some Funky HTML Sh*t with Raku

Came across some pretty funky PHP/HTML the other day. No, I did not write it! (btw using echo is considered bad practice)

function html_form_code() {
    echo '<form action="' . esc_url( $_SERVER['REQUEST_URI'] ) . '" method="post">';
    echo '<p>';
    echo 'Your Name (required) <br />';
    echo '<input type="text" name="cf-name" pattern="[a-zA-Z0-9 ]+" 
                    value="' . ( isset( $_POST["cf-name"] ) ? esc_attr( $_POST["cf-name"] ) : '' ) . '" size="40" />';
    echo '</p>';
    echo '<p>';
    echo 'Your Email (required) <br />';
    echo '<input type="email" name="cf-email" value="' . ( isset( $_POST["cf-email"] ) ? esc_attr( $_POST["cf-email"] ) : '' ) . '" size="40" />';
    echo '</p>';
    echo '<p>';
    echo 'Subject (required) <br />';
    echo '<input type="text" name="cf-subject" pattern="[a-zA-Z ]+" 
                     value="' . ( isset( $_POST["cf-subject"] ) ? esc_attr( $_POST["cf-subject"] ) : '' ) . '" size="40" />';
    echo '</p>';
    echo '<p>';
    echo 'Your Message (required) <br />';
    echo '<textarea rows="10" cols="35" name="cf-message">' . ( isset( $_POST["cf-message"] ) ? esc_attr( $_POST["cf-message"] ) : '' ) . '</textarea>';
    echo '</p>';
    echo '<p><input type="submit" name="cf-submitted" value="Send"/></p>';
    echo '</form>';
}

By coincidence, three HTML-ish Raku ideas have recently popped into my inbox courtesy of the Raku Weekly rag:

So this all got me wondering what my funky PHP/HTML sample would look like in a fully fledged Cro / Raku style… in the spirit of keeping this post briefish, I will skip the Cro Templates and CSS parsing for now and hope to cover them in subsequent missives…

First, here is gfldex’s code copied to my source file:

constant term:<␣> = ' ';
constant term:<¶> = $?NL;

constant term:<§> = class :: does Associative {
    sub qh($s) {
        $s.trans([ '<'   , '>'   , '&' ] =>
                 [ '&lt;', '&gt;', '&amp;' ])
    }   

    role NON-QUOTE {}

    method AT-KEY($key) {
        when $key ~~ /^ '&' / { 
            $key does NON-QUOTE
        }   
        when $key ~~ /\w+/ {
            sub (*@a, *%_) {
                ##dd @a; 
                ('<' ~ $key ~ (+%_ ?? ␣ !! '') 
                ~ %_.map({ .key ~ '="' ~ .value ~ '"'  }).join(' ') ~ '>' 
                ~ @a.map({ $^e ~~ NON-QUOTE ?? $^e !! $^e.&qh }).join('')
                ~ '</' ~ $key ~ '>') does NON-QUOTE
        }   
    }   
}

Sharp eyed viewers will notice that I have made one change … replacing the constant ‘html’ with term: < § > … this is called the section symbol and lurks towards the top left of your keyboard. I find this greatly improves the readability of my embedded html.

So, here’s how my PHP example looks in modern Raku stylee:

my $action = 'mailto:me@p6steve.com';
my $cf-name = 'p6steve';
my $cf-email = 'me@p6steve.com';
my $cf-subject = 'Raku does HTML';
my $cf-message = 'Describe your feelings about this...';

my $size = <40>;
my $pattern = <[a-zA-Z0-9 ]+>;

put '<!DOCTYPE html>';
put 
§<html>( ¶,  
    §<body>( ¶,
        §<form>(:$action, :method<post>, ¶,       
            §<p>('Your Name (required)'), ¶,
            §<input>(:type<text>, :required, :name<cf-name>, 
                                  :value($cf-name), :$size, :$pattern,), ¶,

            §<p>('Your Email (required)'), ¶,     #email type validates input
            §<input>(:type<email>, :required, :name<cf-email>, 
                                   :value($cf-email), :$size), ¶,  

            §<p>('Your Subject (required)'), ¶,
            §<input>(:type<text>, :required, :name<cf-subject>, 
                                  :value($cf-subject), :$size, :$pattern,), ¶,

            §<p>('Your Message (required)'), ¶,
            §<p>(§<textarea>(:rows<10>, :cols<35>, :required, 
                             :name<cf-message>, $cf-message, ),), ¶,

            §<input>(:type<submit>, :name<cf-submitted>, :value<Send>,), ¶,
        )   
    )   
);

Thoughtfully glfdex includes a para character ¶ term, to make line breaks in the output html source to keep it human friendly. And here is the html output:

<DOCTYPE html>
<html>
<body>
<form action="mailto:me@p6steve.com" method="post">
<p>Your Name (required)</p>
<input pattern="[a-zA-Z0-9 ]+" type="text" name="cf-name" size="40" required="True" value="p6steve"></input>
<p>Your Email (required)</p>
<input size="40" value="me@p6steve.com" type="email" required="True" name="cf-email"></input>
<p>Your Subject (required)</p>
<input name="cf-subject" pattern="[a-zA-Z0-9 ]+" type="text" size="40" value="Raku does HTML" required="True"></input>
<p>Your Message (required)</p>
<p><textarea required="True" cols="35" rows="10" name="cf-message">Describe your feelings about this...</textarea></p>
<input name="cf-submitted" value="Send" type="submit"></input>
</form></body></html>

Personally I love to write (and read) html when done in this kind of programmatic style. Not least it has cut 19 lines of embedded code to 10 lines (and that means I can squish more code into my screen and into my brain). No longer do I have to dance my right pinkie around the < / > keys or worry about leaving out the closing end tags!! Another neat helper is the Raku pair syntax, so if I define a scalar with the same name as the attribute name, I can avoid repetitive typing and the consequent opportunity to make a mistake… e.g. the :$action attribute in the form tag.

Hopefully in the next instalment, I will be able to combine the power of Cro::Web::Template to apply the substitution and escape pieces…

Please do leave any thoughts and comments below.

raku:34 python:19 extreme math

Coming off the excellent raku weekly news, my curiosity was piqued by a tweet about big-endian smells that referenced a blog about “extreme math”. After getting my fill of COBOL mainframe nostalgia, the example of Muller’s Recurrence got me thinking.

The simple claim made in the tweet thread was:

Near the end it [the blog] states that no modern language has fixed point, but Raku (formerly Perl6) has a built in rational type which is quite an interesting comparison. It keeps two integers for the numerator and the denominator and no loss of precision occurs.

I have also covered some of the benefits of the raku approach to math in a previous blog Machine Math and Raku, often the example given is 0.1 + 0.2 =>0.3 which trips up a lot of languages. I like this example, but I am not entirely convinced by it – sure it can be odd when a programming newbie sees a slightly different result caused by floating point conversions – but it is too mickey mouse to be a serious concern.

The Muller Extreme Challenge

This challenge starts with seemingly innocuous equations and quickly descends into very substantial errors. To quote from the Technical Archaelogist blog:

Jean-Michel Muller is a French computer scientist with perhaps the best computer science job in the world. He finds ways to break computers using math. I’m sure he would say he studies reliability and accuracy problems, but no no no: He designs math problems that break computers. One such problem is his recurrence formula. Which looks something like this:

That doesn’t look so scary does it? The recurrence problem is useful for our purposes because:

  • It is straight forward math, no complicated formulas or concepts
  • We start off with two decimal places, so it’s easy to imagine this happening with a currency calculation.
  • The error produced is not a slight rounding error but orders of magnitude off.

And here’s a quick python script that produces floating point and fixed point versions of Muller’s Recurrence side by side:

from decimal import Decimal

def rec(y, z):
 return 108 - ((815-1500/z)/y)
 
def floatpt(N):
 x = [4, 4.25]
 for i in range(2, N+1):
  x.append(rec(x[i-1], x[i-2]))
 return x
 
def fixedpt(N):
 x = [Decimal(4), Decimal(17)/Decimal(4)]
 for i in range(2, N+1):
  x.append(rec(x[i-1], x[i-2]))
 return x

N = 30
flt = floatpt(N)
fxd = fixedpt(N)

for i in range(N):
 print( str(i) + ' | '+str(flt[i])+' | '+str(fxd[I]) )

Which gives us the following output:

i  | floating pt    | fixed pt
-- | -------------- | ---------------------------
0  | 4              | 4
1  | 4.25           | 4.25
2  | 4.47058823529  | 4.4705882352941176470588235
3  | 4.64473684211  | 4.6447368421052631578947362
4  | 4.77053824363  | 4.7705382436260623229461618
5  | 4.85570071257  | 4.8557007125890736342039857
6  | 4.91084749866  | 4.9108474990827932004342938
7  | 4.94553739553  | 4.9455374041239167246519529
8  | 4.96696240804  | 4.9669625817627005962571288
9  | 4.98004220429  | 4.9800457013556311118526582
10 | 4.9879092328   | 4.9879794484783912679439415
11 | 4.99136264131  | 4.9927702880620482067468253
12 | 4.96745509555  | 4.9956558915062356478184985
13 | 4.42969049831  | 4.9973912683733697540253088
14 | -7.81723657846 | 4.9984339437852482376781601
15 | 168.939167671  | 4.9990600687785413938424188
16 | 102.039963152  | 4.9994358732880376990501184
17 | 100.099947516  | 4.9996602467866575821700634
18 | 100.004992041  | 4.9997713526716167817979714
19 | 100.000249579  | 4.9993671517118171375788238
20 | 100.00001247862016 | 4.9897059157620938291040004
21 | 100.00000062392161 | 4.7951151851630947311130380
22 | 100.0000000311958  | 0.7281074924258006736651754
23 | 100.00000000155978 | -581.7081261405031229400219627
24 | 100.00000000007799 | 105.8595186892360167901632650
25 | 100.0000000000039  | 100.2767586430669099906187869
26 | 100.0000000000002  | 100.0137997241561168045699158
27 | 100.00000000000001 | 100.0006898905241097140861868
28 | 100.0 | 100.0000344942738135445216746
29 | 100.0 | 100.0000017247126631766583580
30 | 100.0 | 100.0000000862356186943169827

Up until about the 12th iteration the rounding error seems more or less negligible but things quickly go off the rails. Floating point math converges around a number twenty times the value of what the same calculation with fixed point math produces.

Least you think it is unlikely that anyone would do a recursive calculation so many times over. This is exactly what happened in 1991 when the Patriot Missile control system miscalculated the time and killed 28 people. And it turns out floating point math has blown lots of stuff up completely by accident. Mark Stadtherr gave an incredible talk about this called High Performance Computing: are we just getting wrong answers faster? You should read it if you want more examples and a more detailed history of the issue than I can offer here.

[endquote]

So, basically, python Float dies at iteration #12 and python Fixed/Decimal dies at iteration #19. According to the source text COBOL dies at iteration #18. Then the argument focuses on the need for the Decimal library.

How does raku Measure Up?

I do not buy the no loss of precision occurs claim made on twitter beyond the simpler examples, but I do think that Rats should fare well in the face of this kind of challenge. Here’s my code with raku default math:

my \N = 30;
my \x = []; 
x[0] = 4; 
x[1] = 4.25;

sub f(\y,\z) { 
    108 - ( (815 - 1500/z ) / y ) }

for 2..N -> \i { 
    x[i] = f(x[i-1],x[i-2])   }

for 0..N -> \i {
    say( i ~ ' | ' ~ x[i] )   }

Quick impression is that raku is a little more faithful to the mathematical description and a little less cramped than the python.

The raku output gives:

0 | 4
1 | 4.25
2 | 4.470588
3 | 4.644737
4 | 4.770538
5 | 4.855701
6 | 4.910847
7 | 4.945537
8 | 4.9669626
9 | 4.9800457
10 | 4.98797945
11 | 4.992770288
12 | 4.9956558915
13 | 4.9973912684
14 | 4.99843394394
15 | 4.999060071971
16 | 4.999435937147
17 | 4.9996615241038
18 | 4.99979690071342
19 | 4.99987813547793
20 | 4.9999268795046
21 | 4.9999561270611577
22 | 4.99997367600571244
23 | 4.99998420552027271
24 | 4.999990523282227659
25 | 4.9999943139585595936
26 | 4.9999965883712560237
27 | 4.99999795302135690799
28 | 4.999998771812315
29 | 4.99999926308729
30 | 4.999999557853926

So, 30 iterations with no loss of precision – and with the native raku math defaults. Nice!

Eventually raku breaks at 34 iterations, so raku:34, python:19.

~p6steve

PS. And to reflect the harsh reality of life, Victor Ejikhout’s comment can have the final word: so know your own limits!

This is not a problem of fixed point vs floating point. I think your examples favor Fix because you give it more digit of accuracy. What would happen if you used a Float format where the mantissa is equally long as the total Fix length? Objection #2: I think Cobol / Fix would converge away from 5 if you ran more iterations. The Muller equation has three fixed points: x_n==3, x_n==5, and x_n==100. If you start close enough to 5 it will converge there for a while, but (I’m guessing here; didn’t run all the tests) it will converge to the 100 solution. Since you give the float solution less precision it simply converges there faster.The only real lesson here is not to code unstable recursions.

raku = Easy | Hard

Larry Wall, the inventor of perl and raku (formerly known as perl6) coined the phrase “making the easy things easy and the hard things possible”. One way this applies is that developers are publishers and|or consumers of code. For example,

  • one programmer may write a compiler toolchain in C (or NQP) and another feed it high level language software to be complied
  • someone can write a module, someone else can use it
  • authors publish classes with defined APIs and others compose them into larger applications
  • a client-server model, where structured and scalable back-end data is accessed by web browsers using markup and reactive scripts

In general, this pattern helps system experts do the low level, tricksy stuff (parsers, VMs, threads, optimisers) and domain experts can then employ a higher level abstraction. Each can focus on their specific domain(s) of interest.

This pattern often entails the use of a powerful, low level language in the server, and a quick and flexible language in the client. You know the scene: Javascript and HTML accessing Java Object Oriented business logic and a SQL database with ACID transactions. And this asymmetric architecture has often made good sense, allowing the server to be fine tuned and type checked while still facilitating rapid application development and delivery via a variety of web / application presentations. Rust in general and the recently announced Rust rewrite of Apache come into this category

But, when these specialisations turn into silos, then barriers may arise that can hamper adaptability, speed of delivery and front-to-back consistency. That’s one reason why bridges have arisen between server and client – Javascript and Node.js being one typical response from the market.

Enter raku; like it predecessor perl, raku is a language that can ‘telescope’. From pithy one liners on the command line to deep class model introspection and mutation. Depending on the needs of the situation and the knowledge of the developer. So, raku combines an approachable on-ramp for less experienced coders and it offers power developers the keys they need to open up and adapt underlying structures to fit specialised requirements. Raku can even inline low level code (C, C++) where the limits of the language are reached. A reboot of the original perl philosophy of “making the easy things easy and the hard things possible”.

Those that follow my blog will know I am the author of Physics::Unit and Physics::Measure modules. I’m now working on Physics::Navigation — inspired by a recent theory course. This is a domain specific class library that consumes Physics::Measure and provides abstractions such as Latitude, Longitude and Bearing. My aim is primarily to have a piece of code that exercises the underlying modules in order to road test the API and to have some fun. One great benefit is that I can use it explore the clarity and power of expression that raku can bring.

It started with an example of calculating the spherical law of cosines formula (aka Haversine distance) thanks to the very informative and helpful movable type website. Code examples are based on the Rosetta Code variations.

As set out mathematically the formula to be calculated is:

a = sin²(Δφ/2) + cos φ1 ⋅ cos φ2 ⋅ sin²(Δλ/2)
d = 2 ⋅ R ⋅ asin( √a )

The example code in Java is:

public class Haversine {
    public static final double R = 6372.8; // In kilometers
    public static double haversine(double lat1, double lon1, double lat2, double lon2) {
        double dLat = Math.toRadians(lat2 - lat1);
        double dLon = Math.toRadians(lon2 - lon1);
        lat1 = Math.toRadians(lat1);
        lat2 = Math.toRadians(lat2);
 
        double a = Math.pow(Math.sin(dLat / 2),2) + Math.pow(Math.sin(dLon / 2),2) * Math.cos(lat1) * Math.cos(lat2);
        double c = 2 * Math.asin(Math.sqrt(a));
        return R * c;
    }
    public static void main(String[] args) {
        System.out.println(haversine(36.12, -86.67, 33.94, -118.40));
    }
}

In contrast, the decomposition in Raku is, the “client” ie. a method …

method haversine-dist(Position $p) {
        my \Δ = $.Δ( $p );

        my $a = sin(Δ.φ / 2)² + 
                sin(Δ.λ / 2)² * cos($.φ) * cos($p.φ);

        my $value = 2 * R * asin(sqrt($a));

        Distance.new( :$value, :units<m> )   
}   

… which is aided by the “server” ie. a class with class attributes and helper accessor methods that help partition the problem cleanly and deliver a new level of clarity, code reuse and maintainability.

class Position is export {
	has Latitude  $.lat;
	has Longitude $.long;

	# accessors for radians - φ is latitude, λ is longitude 
	method φ { +$.lat  * π / 180 }
	method λ { +$.long * π / 180 }

        # get delta Δ between two Positions
	method Δ( $p ) {
		Position.new( ($p.lat - $.lat), ($p.long - $.long) )
	}
        #...
}

So hopefully, this example illustrates how a thoughtful partitioning of “client-server” code can drive conceptual clarity and productivity. Raku helps by providing:

  • strong native unicode support, for maths symbols, superscripts and so on
  • a very concise ‘no boilerplate’ class / role model to aid composition
  • super lightweight object / method / functional syntax
  • a language design that works at the symbolic level (ie character by character)

This is what I love about raku, the language that tries to fade away and to leave the resulting code speak for itself and reflect the original problem domain. In some ways the raku formulation is superior even to the original mathemetical formula.

I am very excited to learn of recent initiatives as reported in the last edition of Rakudo Weekly News 2021.04 Grant Reporting about the work by Vadim Belman on A New Release Of Cro::RPC::JSON.

This has the potential to functionally partition a program across client and server systems while retaining a consistent set of class definitions, data models and patterns with minimal remote method call overhead. I can even imagine a cadre of data scientists using this technology to create, share and exploit raku code as domain-specific programmers.

And, back to the title of this blog post, “making the easy things easy and the hard things possible”. Here’s a way to say that in raku, a Junction

enum Names <Easy Hard>;           #Map.new((Easy => 0, Hard => 1))
my \raku = Easy | Hard;           #any(Easy, Hard)
raku.^name;                       #Junction

~p6steve

Raku Performance and Physics::Unit

I have been able to spend some time on the Physics::Unit module over the holidays and to expunge some of the frustrations that have crept in regarding the compile times of raku.

The basic problem I have been wrestling with is the desire to express physical SI units using the raku custom postfix operator mechanism without having to wait for 10 mins for raku to compile my module.

This is the story of how judicious design, lazy execution, trial & error and the raku power tools got from 10 mins+ to under 13 secs!

Let’s start by looking at the sunlit uplands. Imagine a raku which provides a simple and intuitive tool for scientists and educators to perform calculations that automatically figure out what the physical units are doing.

Something like this:

use Physics::Constants;
use Physics::Measure :ALL;

$Physics::Measure::round-to = 0.01;

my \λ = 2.5nm; 
my \ν = c / λ;  
my \Ep = ℎ * ν;  

say "Wavelength of photon (λ) is " ~λ;              #2.5 nm
say "Frequency of photon (ν) is " ~ν.norm;          #119.92 petahertz 
say "Energy of photon (Ep) is " ~Ep.norm;           #79.46 attojoule

Now, before you ask, yes – this is real code that works and compiles in a few seconds. It uses the latest version Physics::Measure module which in turn uses the Physics::Units module. Let me point out a few cool things about how raku’s unique combination of features is helping out:

  • unicode to keep familiar symbols such as λ (lambda) and ν (nu)
  • variable names without the $ sigil to keep equations clean
  • Physics::Constants – for c (speed of light) and ℎ (Planck’s constant)
  • Physics::Measure :ALL to imports all the SI unit postfix operators
  • postfix:<nm> to do Measure.new( value => 2.5, unit => ‘nanometre’ )
  • does Measure math with custom ‘/’ and ‘*’ operators
  • knows that a Frequency class type takes SI unit hertz
  • knows that an Energy class type takes SI unit joule
  • can normalise a Measure object and round the output

So – how can it be that hard? Well the devil is in the little word all [the SI unit postfix operators]. Consider this table:

So we have 27 units and 20 prefixes – that’s, err, 540 combinations. And you want me to import all of these into my namespace. And you want me to have a library of 540 Physics::Unit types that get loaded when I use the postfix. Have you thought this through!!??

So – by way of sharing the pain of this part of my raku Physics::Journey – here are the lessons I have learned to optimise my module code:

Attempt 1 – Ignore It

My first instinct was to punt on the issue. The original perl5 Physics::Unit module allows coders to specify a unit type via a string expression – something like this:

my $u2 = GetUnit( 'kg m^2 / s^2' );

Anyway I knew I would need unit expressions to cope with textual variants such as ‘miles per hour’ or ‘mph’, or ‘m/s’, ‘ms^-1’, ‘m.s-1’ (the SI derived unit representation) or ‘m⋅s⁻¹’ (the SI recommended string representation, with superscript powers). So a new unit expression parser was built into Physics::Unit from the start with raku Grammars. However, it became apparent that saying:

my $l = Length.new( value => 42, units => 'yards' );

Is a pretty long-winded way to enter each measurement. Still, this was a cool way to apply (and for me to learn) raku Grammars and Actions which has resulted in a flexible, human-friendly unit expression slang as a built-in piece of the Physics::Unit toolkit.

Attempt 2 – Working but Dog Slow

So far, my Physics::Unit module would happily take a unit string, parse it with the UnitGrammar and create a suitable instance of a Unit object. Something like this:

Unit.new( factor => 0.00016631, offset => 0, 
    defn => 'furlong / fortnight', 
    type => Speed, dims => [1,0,-1,0,0,0,0,0], 
    dmix => ("fortnight"=>-1,"furlong"=>1).MixHash, names => ['ff'] );

This user-defined object is generated by iterating over it’s roots (e.g.) 1 fortnight => 2 weeks => 14 days => 336 hours => 2,016 mins => 120,960 secs (thus the factor attribute). More than 270 built in unit and prefix definitions – covering SI, US (feet, inches), Imperial (pints, gallons) and so on. And the .in() method is used for conversions. [There does not seem much point in a Unit library unless it can support common usage such as mph and conversion between this and the formal SI units.]

But, now I come to implement my postfix operators – then I need to pass 540 definitions to the Grammar on first compilation and it needs to build 540 object instances. Welcome to 10 mins+ compile times.

Before I go too far with this critique – I would like to note a couple of very important caveats:

  1. “So finally, we have an actual Perl 6 that can compete with all those languages too, at least in terms of features. One of the ways it does not yet compete with Perl 5 is in the area of performance, but the first half of this year we’ve already got it running nearly twice as fast as it was at Christmas. We’ve still got a lot of headroom for optimization. But Perl 6 has its eye on surpassing all those other languages eventually. Perl has always been about raising the bar, then raising it again, and again. ” Larry Wall on Slashdot in 2016 … and optimisations and enhancements are coming all the time.
  2. Raku recompilation is a very big speed multiplier – even with 30 min compile times, the precompiled binary loaded and ran in about 12 seconds.
  3. Personally I echo the view that raku is a very high level language and that the ensuing programmer productivity benefits in precision and expression outweigh a few seconds of compile time. I am confident that the hardware will continue to improve and will soon eliminate any noticeable delay – for example the recent Apple M1 launch.

Attempt 3 – Stock Unit Blind Alley

Sooo – the third attempt to combine the desired features and reasonable speed was to pre-generate the 540 units as literals – “Stock” units. So the code could be run in “dump” mode to generate the unit literals using the Grammar and store to a text file, then paste them back into the module source so that in the released version they are just read in via a “fast start” mode.

By reusing the same Grammar for pre-generation and on the fly generation of user-defined Units, this eliminated any potential compatibility issues. I chose not to compromise with any of the MONKEY-SEE-NO-EVAL for module security and code integrity reasons.

Performance improvements were dramatic. By bypassing the Grammar step, first compile times came down to ~340s and precompile start times to under 3s. Nevertheless, I was still unhappy to release a module with such a slow first compile time and searched for a better design.

Attempt 4 – Lazy Instantiation

On the fourth pass, I decided to re-factor the module with lazy Unit instantiation. Thus the unit definitions are initialised as hash data maps, but only a handful of objects are actually created (the base units and prefixes).

Then, when a new object is called for, it is generated “lazily” on demand. Even in an extreme case such as the ‘furlong / fortnight’ example, only O(10) objects are created.

By eliminating the Stock units, this reduced the module source by 2000+ lines (540 x 4 lines per object literal). Performance improved substantially again – this time about 60s first compile and 2.6s precomp start times.

However, the Physics::Measure code still had to embody the postfix operators and to export them to the user program. Thus 540 lines to individually compile. Each postfix declaration like this:

sub postfix:<m> ( Real:D $x ) is export { do-postfix( $x, 'm' ) }

Attempt 5 – UNIT::EXPORT package

Even better, I could learn from the excellent raku documentation again – this time to discover UNIT::EXPORT that gave me a good start to produce a programmatic export all 540 postfixes in just 6 lines of code. Goodbye boilerplate!

my package EXPORT::ALL {
  for %affix-by-name.keys -> $u {
    OUR::{'&postfix:<' ~ $u ~ '>'} := 
                    sub (Real:D $x) { do-postfix($x,"$u") };
  }   
}

This had the additional performance boost – the final numbers below…

Attempt 6 – Selective Import

Finally, a word on selective import. Prior to Attempt 5, I experimented with labelling the less common units (:DEFAULT, :electrical, :mechanical, :universal and :ALL if you are interested). But even by reducing :DEFAULT to only 25% of the total, this did not reduce the first compile time measurably. I interpret this as the compiler needing to process all the lines even if the import labels are not specified by the user program.

But with the package approach, Physics::Measure :ALL will export all of the SI units. Just drop the :ALL label if you want even more speed and plan to go without the postfix operators.

Final Outcome

So, the latest speed measurements (on my low spec 1.2GHz/8GB laptop) are:

# use Physics::Measure;      ...10s first-, 1.2s pre- compiled
# use Physics::Measure :ALL; ...13s first-, 2.8s pre- compiled

YMMV!

~p6steve (pronounced p-sics)

Raku Santa Emoticon [}:]>*

Santa has been fretting about the most concise way to use his personal emoticon [}:]>* programatically in a raku one-liner.

The best he can do is…

raku -e 'sub santa($x is copy){$x~~s/ <[}:]>* /claus/; $x.say}; santa(":")'
#OUTPUT claus

Can you help him? If so – please send your version via the Comments field below.

The rules are:

(i) to use raku (the language formerly known as perl6), perl5 and other languages will be considered too

(ii) to use the character sequence [}:]>* (or reversed, no spaces)

(iii) these characters must have semantic meaning in the code (ie. to just as a comment)

(iv) it should be a one-liner that uses raku -e on the command line

The objective criteria is: shortest count of chars between the ”

Kudos will also be given to the overall Christmassy-ness … in the example, a call to ‘santa’ creates output ‘claus’

Merry Christmas to one and all (and any)

PS. Code golf is very, very naughty, so be sure not to do this in your day job!

Machine Math and Raku

In the machine world, we have Int and Num. A High Level Language such as Raku is a abstraction to make it easier for humans. Humans expect decimal math 0.1 + 0.2 = 0.3 to work.

Neither Int nor Num can do this! Huh? How can that be?

Well Int is base-2 and decimals are base-10. And Num (IEEE 754 Floating Point) is inherently imprecise.

Luckily we have Rat. Raku automatically represent decimals and other rational numbers as a ratio of two integers. Taking into account all the various data shuffling any program creates, the performance is practically as fast as Int. Precision, accuracy and performance are maintained for the entire realm of practical algebraic calculations.

And when we get to the limit of Rat (at around 3.614007241618348e-20), Raku automatically converts to a Num. After this, precision cannot be recovered – but accuracy and dynamic range are maintained. Modern Floating Point Units (FPUs) make Num nearly as fast as Int so performance is kept fast.

Here, we have smoothly transitioned to the world of physical measurements and irrational numbers such as square root and logarithms. It’s a one way trip. (Hint: If you want a Num in the first place, just use ‘e’ in the literal.)

Good call, Raku!

Footnote: FatRat and BigInt are not important for most use cases. They incur a performance penalty for the average program and rightly belong in library extensions, not the core language.

perl7 vs. raku: Sibling Rivalry?

It was an emotional moment to see the keynote talk at TPRCiC from Sawyer X announcing that perl 7.00 === 5.32. Elation because of the ability of the hardcore perl community to finally break free of the frustrating perl6 roadblock. Pleasure in seeing how the risky decision to rename perl6 to raku has paid off and hopefully is beginning to defuse the tensions between the two rival communities. And Fear that improvements to perl7 will undermine the reasons for many to try out raku and may cannibalise raku usage. (Kudos to Sawyer to recognising that usage is an important design goal).

Then the left side of my brain kicked in. Raku took 15 years of total commitment of genius linguists to ingest 361 RFCs and then synthesise a new kind of programming language. If perl7 seeks the same level of completeness and perfection as raku, surely that will take the same amount of effort. And I do not see the perl community going for the same level of breaking changes that raku did. (OK maybe they could steal some stuff from raku to speed things up…)

And that brought me to Sadness. To reflect that perl Osborned sometime around 2005. That broke the community in two – let’s say the visionaries and the practical-cats. And it drove a mass emigration to Python. Ancient history.

So now we have two sister languages, and each will find a niche in the programming ecosystem via a process of Darwinism. They both inherit the traits (https://en.wikipedia.org/wiki/Perl#Design) that made perl great in the first place….

The design of Perl can be understood as a response to three broad trends in the computer industry: falling hardware costs, rising labor costs, and improvements in compiler technology. Many earlier computer languages, such as Fortran and C, aimed to make efficient use of expensive computer hardware. In contrast, Perl was designed so that computer programmers could write programs more quickly and easily.

Perl has many features that ease the task of the programmer at the expense of greater CPU and memory requirements. These include automatic memory management; dynamic typing; strings, lists, and hashes; regular expressions; introspection; and an eval() function. Perl follows the theory of “no built-in limits,” an idea similar to the Zero One Infinity rule.

Wall was trained as a linguist, and the design of Perl is very much informed by linguistic principles. Examples include Huffman coding(common constructions should be short), good end-weighting (the important information should come first), and a large collection of language primitives. Perl favours language constructs that are concise and natural for humans to write.

Perl’s syntax reflects the idea that “things that are different should look different.” For example, scalars, arrays, and hashes have different leading sigils. Array indices and hash keys use different kinds of braces. Strings and regular expressions have different standard delimiters. This approach can be contrasted with a language such as Lisp, where the same basic syntax, composed of simple and universal symbolic expressions, is used for all purposes.

Perl does not enforce any particular programming paradigm (proceduralobject-orientedfunctional, or others) or even require the programmer to choose among them.

But perl7 and raku serve distinct interests & needs:

Thingperl7raku
compilationstatic parserone pass compiler
compile speedsuper fastrelies on pre-c0mp
executioninterpretedvirtual machine
execution speedsuper fastrelies on jit
module libraryCPAN nativeCPAN import
closuresyesyes
OO philosophyCor not modulepervasive
OO inheritanceRoles + IsRoles + Is + multiple
method invocation->.
type checkingnogradual
sigilsidiosyncratic consistent
referencesmanualautomatic
unicodefeature guardcore
signaturesfeature guardcore
lazy executionnopecore
Junctionsnopecore
Rat mathnopecore
Sets & Mixesnopecore
Complex mathnopecore
Grammarsnopecore
mutabilitynopecore
concurrencynopecore
variable scope“notched”cleaner
operatorsC-likecleaner (e.g. for ->)
switchnogather/when
regexenclassiccleaner
evalyesshell
AST macroshuh?
…and so on

A long list and perhaps a little harsh on perl since many things may be got from CPAN – but when you use raku in anger, you do see the benefit if having a large core language. Only when I made this table, did I truly realise just what a comprehensive language raku is, and that perl will genuinely be the lean and mean option.

Ariel Atom 3.5 review, price, specs and video | Evo
perl7
Model X | Tesla
raku

And, lest we forget our strengths:

When I first saw Python code, I thought that using indents to define the scope seemed like a good idea. However, there’s a huge downside. Deep nesting is permitted, but lines can get so wide that they wrap lines in the text editor. Long functions and long conditional actions may make it hard to match the start to the end. And I pity anyone who miscounts spaces and accidentally puts in three spaces instead of four somewhere — this can take hours to debug and track down. [Source: here]

Raku Objects: Confusing or What?

Chapter 1: The Convenience Seeker

Coming from Python, the Raku object model is recognizable, but brings a tad more structure:

Screenshot 2020-05-07 22.36.37

What works for me, as a convenience seeker, is:

  • the attributes $.x, $.y are automatically provided with setter and getter methods
  • the constructor new() is automatically provided
  • the output method e.g. ‘say $p.Str’ is automatically provided
  • I can simply assign to an attribute with ‘=’

These are the things you want if you are writing in a more procedural or functional style and using class as a means to define a record type.

Chapter 2: The Control Freak

Here’s the rub…

When we describe OO, terms like “encapsulation” and “data hiding” often come up. The key idea here is that the state model inside the object – that is, the way it chooses to represent the data it needs in order to implement its behaviours (the methods) – is free to evolve, for example to handle new requirements. The more complex the object, the more liberating this becomes.

However, getters and setters are methods that have an implicit connection with the state. While we might claim we’re achieving data hiding because we’re calling a method, not accessing state directly, my experience is that we quickly end up at a place where outside code is making sequences of setter calls to achieve an operation – which is a form of the feature envy anti-pattern. And if we’re doing that, it’s pretty certain we’ll end up with logic outside of the object that does a mix of getter and setter operations to achieve an operation. Really, these operations should have been exposed as methods with a names that describes what is being achieved. This becomes even more important if we’re in a concurrent setting; a well-designed object is often fairly easy to protect at the method boundary.

(source jnthn https://stackoverflow.com/questions/59671027/mixing-private-and-public-attributes-and-accessors-in-raku)

Let’s fix that:

Screenshot 2020-05-07 22.38.41
Now, I had to do a bit more lifting, but here’s what I got:

  • the private attributes $!x, $!y are formally encapsulated
  • the BUILD submethod does constructor .new() – zero boilerplate needed
  • it takes a method call [$p.y( 2 )] or the colon variant [$p.y: 3] to affect state

And, in contrast to Chapter 1:

  • I cannot assign to has attributes using ‘=’
  • since accessors are explicit I can easily code for constraints and side-effects
  • it’s a pita to code accessors encouraging proper separation of behaviours

Chapter 3: Who Got the Colon in the End?

I also discovered Larry’s First Law of Language Redesign: Everyone wants the colon

Apocalypse 1: The Ugly, the Bad, and the Good https://www.perl.com/pub/2001/04/02/wall.html/

I conclude that Larry’s decision was to confer the colon on the method syntax,  subtly tilting the balance towards the strict model: by preferring $p.y: 3 over $p.y = 2.

Raku vs. Perl – save 70%

Having hit rock bottom with an ‘I can’t understand my own code sufficiently enough to extend/maintain it’, I have been on a journey to review the perl5 Physics::Unit design and to use this to cut through my self made mess of raku Physics::Unit version 0.0.2.

Now I bring the perspective of a couple of years of regular raku coding to bear, so I am hoping that the bastard child of mature perl5 and raku version one will surpass both in the spirit of David Bowie’s “Pretty Things”.

One of the reasons I chose Physics::Units as a project was that, on the face of it, it seemed to have an aspect that could be approached by raku Grammars – helping me learn them. Here’s a sample of the perl5 version:

Screenshot 2020-04-17 18.40.05

Yes – a recursive descent parser written from scratch in perl5 – pay dirt! There are 215 source code lines dedicated to the parse function. 5 more screens like this…

So I took out my newly sharpened raku tools and here’s my entire port: 

Screenshot 2020-04-17 18.42.08

Instead of ranging over 215 lines, raku has refined this down to a total of 58 lines (incl. the 11 blank ones I kept in for readability) – that’s a space saving of over 70%. Partly removal of parser boilerplate code, partly the raku Grammar constructs and partly an increased focus on the program logic as opposed to the mechanism.

For my coding style, this represents a greater than a two-thirds improvement – by getting the whole parser onto a single screen, I find that I can get the whole problem into my brain’s working memory and avoid burning cycles scrolling up and down to pin down butterflies bugs.

Attentive students will have noted that the Grammar / code integration provides a very natural paradigm for loading real-world data into an OO system, the UnitAction class starts with a stub object and populates ‘has’ attributes as it goes.

Oh and the raku code does a whole lot more such as support for unicode superscripts (up to +/-4), type assignment and type checking, offsets (such as 0 K = 273.15 °C), wider tolerance of user input and so on. Most importantly Real values are kept as Rats as much as possible which helps greatly for example, when round tripping 38.5 °C to  °F and back it is still equals 38.5 °C!

One final remark – use Grammar::Tracer is a fantastic debugging tool for finding and fixing the subtle bugs that can come in and contributing to quickly getting to the optimum solution.