# Vittorio Zaccaria

Random character sequences from an academic.

Progresso del corso di Informatica B:
Stato correzione prima prova:

## Taming asynchronous programming with Harmony

### What is Chained

Chained (home) is an experiment and a prototype library for Javascript I developed in the past weeks. It is meant to show a concept and to provide some limited but useful functionality. All the examples below are written in Coffeescript (and I guess that this makes me a hippie, right?).

Chained allows to chain functions explicitly — both those that return promises and those who don’t — without using then-based constructs.

For example, here’s how you could create a chained computation that:

• downloads with jQuery’s get a user’s npm page

• filters the links to his projects with underscore

  jQuery       = require('jQuery')
underscore   = require('underscore')
string       = require('underscore.string')
{ using, _ } = require('./chain').chain.init()

using(jQuery)
using(underscore)
using(string)
using(console)

getUser = (user) ->
_("https://npmjs.org/~#{user}")
.get()
.filter( -> /package/.test(arguments[0]) )
.map( -> "https://npmjs.org#{arguments[0]}" )
.log()

getUser("vzaccaria")

Here, we have chained together promise-based functions (like get from jQuery) and normal functions with additional parameters (like map and filter from underscore). All function invocations receive an implicit argument that is the value computed by the previous link in the chain.

The function defined in getUser implements a processing pipeline (à la streaming). It starts with the link to be retrieved by specifying it with _():

_("https://npmjs.org/~#{user}")

this value is then passed to get() of jQuery. Chained knows that jQuery has a get method since we introduced its scope with

using(jQuery) 

at the beginning1.

The result of get() — i.e. the webpage — is sent to extractLinks (from linklib) when the associated promise is resolved. The rest of the computation proceeds as you can intuitively imagine.

If an exception is thrown in any of the links of the processing chain then the following links are not executed, just as you would expect from promises.

We can add additional arguments (besides the implicit one) to method invocations; for example the filter function (from underscore) is invoked as:

…
.filter( -> /package/.test(arguments[0]) )
…

where the lambda function is passed effectively as the second parameter to underscore.filter (the first being the implicit parameter).

So we nailed a few things here:

• We can chain promise-based functions and normal functions without distinction using the natural syntax of Javascript.
• We can use directly their names instead of using then.
• We can chain any method from an arbitrary number of modules, provided that the module is bound to Chained with the using clause.
• We don’t modify dynamically the module that are imported. We don’t change any prototype.
• The chain stops executing when one of the links throws an exception or a promise is rejected.
• We are not using any coffee-script mambo-jumbo for chaining; we can write with the same expressiveness in pure Javascript.

### Where’s the catch?

We are exploiting ECMA6/harmony implementation of introspection. You need an ECMA6/harmony implementation, either in node (node --harmony) or in your browser (Firefox is ok at the moment) to make it work.

### Does it work in the browser?

Yes, check out this example Domain Specific Language for form validation directly built with Chained. Warning: it needs a recent build of Firefox. In principle also a harmony-enabled Chrome should work, but I did not try it.

### How to install it?

On node:

npm install chained

• reflect.js
• dsl.js,
• chain.js

You should be good to go.

### Implementation

The idea for Chained came while tinkering with domain specific languages. DSLs provide the developer with the vocabulary of a problem domain to express a programmatic solution that could be understood by a problem domain expert2.

I will not cover all the general concepts of DSLs here. For an in depth overview, I’d suggest to look at these resources:

Internal DSLs are built using the syntactic and semantic features of another language; they are used in the context of a broader application for which they address a narrow but important problem.

My quest was to find a way to implement a DSL in Javascript. Let’s see what features are needed by a general purpose language like Javascript to support the definition of internal DSLs:

• Support for named arguments (or hashes) in function invocation (aka Smart API), e.g.:

  move(the: pen, on: theTable)
• Support for a property/method missing exception to implement method chaining:

  take('spaghetti').and().cookIt(at: 100C)

here and is a non existing method of the return value V of take('spaghetti'). The missing exception handler will return the same value V so that cookIt could be called on it.

• (optional, but highly desirable)

- Reduced boilerplate code.

- Synthetic expression of closures (w/ implicit arguments).

- Limited presence of parenthesis {} and ():

    withObject chair, ->
move it, in: theLivingRoom

At some point, it became clear to me that DSLs’ method chaining can be used to implicitly structure promise based chains. In fact, you could build a promise-chain by handling a sequence of method missing exceptions.

As a byproduct, it is straightforward to use the same technique to chain normal functions, without any exogenous construct like then.

Javascript and its relatives (such as Coffee and LiveScript) have almost all the characteristics to build a sophisticated DSL. The only problem is that they lack of a method missing exception; at least until Harmony came out.

Dylan Barrell already demonstrated that with the new reflection API, it is possible to manage missing methods. His technique exploits the concept of Proxies. Think about a Proxy like a wrapper around your original object that is able to intercept calls to methods that are not defined. Chained is based on Dylan’s work.

#### The basic idea

The basic idea goes as follows:

1. We create an object that can handle missing methods through an appropriate mechanism. Let’s call it o.

2. o contains a promise property. Whenever a method missing is invoked on o:

o.m()

we catch it and handle it with the following handler:

handleMethodMissing = (m) ->
this.promise = this.promise.then( (it) -> scope[m](it) )
return this

now, note that:

• we need a scope in which to look for m. In Chained, we use using(module) to specify/extend this scope.
• m is called with the final value of the previous promise.
• even if m returns a new promise, this is perfectly fine since the then method is going to resolve the original o.promise accordingly.
• we return o, so that we can repeat the same process by seemingly chaining methods that are effectively missing.
3. We need a way to fire the very first promise off. In Chained, we use the function _() both for creating o and to resolve the first promise of the chain with the value passed to it:

_(v) = ->
o = new methodMissingObject()
o.deferred = defer()
o.promise = o.deferred.promise
o.deferred.resolve(v)
return o

### API

The API is pretty simple at the moment; however, I think it offers a good level of flexibility to be leveraged.

chain.using(module)

Extends the scope within which all missing methods are searched for. You can include multiple modules:

    using(jQuery)
using(underscore)
o = chain._(value)

returns a deferred object whose promise property will eventually be resolved to value. The object handles missing methods by chaining them using Q.

#### Deferred object methods and properties

Given the deferred object o, the following methods apply; remember that they return again a deferred object:

o.foo(...)
if o eventually resolves to v, this is essentially equivalent to building a deferred foo(v, ...).
o._(foo)
specify a one shot function foo to be chained with the rest, without extending the scope with using.
o._forEach(bar)
if o will eventually resolve to an array, apply bar to each element; the combined promise is created with Q.all.
o.promise
this is a Q promise corresponding to the deferred o.

MIT

1. The using clause extends the scope within which Chained looks for functions to be chained.

2. Make, SQL, CSS are all famous examples of DSLs widely used by developers.

## Great Fun at Infographics Workshop in Ferrara

As many of my colleagues know, I am delving more and more into infographics. Far from being a graphic designer or anything similar, I am really enjoying what I am learning and the italian community in general.

I am back from a 3 day workshop given by Michael Robinson (Graphic Designer and past Head of Graphics at the Guardian newspaper) and it was an awesome experience. The workshop was one of several events held at Internazionale in Ferrara in the past weekend.

During the three days, we have been given the opportunity to collect personal data of the attendants to the workshop and sketch out ideas for an infographics that was based on it.

1. In such a short time, the infographic was inevitably created with some elements crawled from the web in a rush. I did not have the time to acknowledge the original authors of those elements nor to take note of who they were. If you happen to be one of them just drop me an email telling me whether you want to be named (and aknowledged) or you want the graphic element to be removed.

## A dumb easy model for promises

Promises are significantly taking ground as a programming construct for asynchronous systems such as web applications. In its simplest form, a promise represents the state of the result of some future computation.

I was introduced recently to this construct when studying jQuery and AngularJS. At the beginning, I was not able to wrap my head around it. While I understood the basic callback mechanism — allowing to be notified when the computation is finished — I struggled a lot capturing the efficiency with which promises allow to easily compose asynchronous programs.

## Enter Petri nets

Petri nets have helped me a lot while creating an easy mental/visual model of promises (and their composition patterns).

In this post, I will refer to the “Promises/A+” specification. I assume you have already some basic knowledge of it — especially of the special cases. Besides, I will not indulge in any mathematical notation of Petri Nets here. If you are interested, I’ve put some suggested reading link at the end of this post.

## Basic form of promise

A promise is an object that has a then method1:

promise.then(onFulfilled, onRejected)

then is used to specify two callbacks — onFulfilled and onRejected — that are mutually executed when the state of the promise is resolved to either fulfilled or rejected.

In a Petri net, the promise state is represented by a token — black disk — that is positioned on a place. In the following figure, the token is on the pending place:

The token can move to another place when the transition events — black squares — fire. Transitions can fire only when they are enabled, i.e., only when a predetermined number of tokens (weight) is positioned on their input places. Without any particular specification, the weight corresponds to the number of input places. The number of marks emitted into a place, when not explicitly shown, is 1.

When the promise is resolved, the output transition from the pending place is fired. If the promise is fulfilled, the marker ‘moves’ to the fulfilled state:

Similarly, the token moves to the rejected place if the promise has been rejected. So far so good.

## Producing a new promise

As shown above, the then method returns a new promise:

p2 = p1.then(onFulfilled, onRejected)

Let’s see, with our new visual model, how the execution of the methods is synchronized with the promise itself:

Once p1 is fullfilled, the transition event onFulfilled is fired — i.e., the callback onFulfilled is invoked. If the callback (being it onFulfilled or onReject) returns a value, p2 will be fulfilled with that value.

On the other hand, if either callback throws an exception, p2 is going to be rejected. So listen, if you want to escalate an error when the first promise is rejected, throw an exception in your onRejected callback!

## Creating synchronization points with promises

Here I think that the real power of promises is unleashed. If your callbacks — onFulfilled or onRejected — return a promise (let’s call it R) you can make sure that p2 is resolved only when both p1 and R have been resolved; in fact according to the specification:

If either onFulfilled or onRejected returns a promise (call it returnedPromise), promise2 must assume the state of returnedPromise.

This Petri net allows to understand very well what’s going on:

Petri nets allow to model these synchronization points very easily. This is done by forcing the number of tokens that can enable a place transition. In fact, the transition of 1 token to the p2-fullfilled place can only be fired when there are two tokens on the input of the commanding transition.

## Cheat sheet

I am astonished about how Petri nets allow us to think in a very clear way about how we structure our promise chain. I hope this can be useful for you as it was for me.

1. We are not going to deal with how this object is generated. We assume that some computation has started and an object representing its future value has been created.

## Can we improve a university class with UX design?

Recently, I’ve been delving more and more into User Experience design, and I thought that my basic computer science class would be an awesome starting point to learn how to apply UX principles to a practical case.

It would be great if — by rethinking some of the aspects of classic academic teaching with modern UX — one could make a more engaging class both for students and the teacher. Of course, it is very difficult to modify the logistic of a in-person class. However, rethinking the web technology supporting the class can be beneficial to improve the user (student) experience.

In an ideal environment, students could:

• interact more with other students by using on-line games, competitions (e.g., DuoLingo).
• train directly in the web browser by leveraging modern technology (e.g., Codecademy) to do their homework.
• comment teacher material (e.g., Medium)

on the other hand, the teacher could:

• measure improvement by considering the execution time and success rate of homework.
• dynamically reschedule the class in the case a topic results into a much harder (or softer) subject.
• use the same web environment during in-loco exams

Above all, being able to implement and measure, the teacher could apply lean techniques to validate his own product — the class — and build a succesfull learning experience.

I’ve realizied a first initial User Journey Map to sketch out how the new web environment (green boxes) could be integrated into the class. You can download the pdf from this link.

Thoughts?

## Wmake - makefile for webapps

Just published wmake on NPM. It is the tool that I use to generate makefile for my webapps. Check it out at his own landing page!

## Meet fluent-fsm

fsm-express just evolved to fluent-fsm. The API has been updated to be much more fluent and the project has also a new landing page!

## Building and provisioning an Ubuntu 12.04 VM with Vagrant and VirtualBox 4.0.6

I am in the process of deploying a monitoring infrastructure based on Node and Phantomjs (well, actually Casperjs, but that’s another story). Usually, I’ve always deployed by configuring manually the target machine but now this is become a pain due to the sheer number of combinations between needed tools, versions and operating system releases.

I needed something to help me automatically replicate the development environment, by guaranteeing consistency among versions. In short, I wanted to reduce the risk of something going wrong or missing when moving from development to production.

## Enter Vagrant

Vagrant allows to build exactly such environments. It is an awesome tool that simplifies virtual machine creation, provisioning and management.

My target machine was an Ubuntu 8.04 (pretty old, huh?). Unfortunately, binary packages of Vagrant were not working correctly on such machine. I have thus installed it by using its corresponding Ruby gem:

gem install vagrant


which worked fantastically. The machine had already a Virtual Box 4.0.6 working and I didn’t want to mess up with existing vm configurations during the upgrade. I thus decided to stick with 4.0.6 and so far so good.

I thus downloaded the vanilla Ubuntu 12.04 Vagrant Box (precise32), initialized it, and started it up:

vagrant box add precise32 http://files.vagrantup.com/precise32.box
vagrant init precise32
vagrant up


From now on, I could connect to the virtual machine with a simple ssh:

vagrant ssh


## Provisioning

Since the initial machine was very vanilla, I had to install all the remaining tools and most importantly, automatize the installation. Here comes another fundamental tool for provisioning management: Chef.

With Chef, you can download various recipes that can be used to install software on your machine. Recipes should be stored by default in the cookbooks directory where you start Vagrant and you should link Vagrant with Chef in the Vagrantfile (written in Ruby). My configuration was as follows:

config.vm.provision :chef_solo do |chef|
chef.json =     {
"nodejs" => {
"version" => "0.8.11"
}
}


To install the above recipes, I cloned their repositories (nodejs, subversion and casperjs) from Github into the cookbooks directory.

In this way, when you bring up your vm with vagrant up, Chef takes all the responsibility of installing the to-be-provisioned tools. Keep in mind that some of the recipes have dependencies that should be met by downloading the corresponding cookbooks into the same directory.

The cool thing is that Chef caches provisioning packages, so the next time you invoke vagrant provision you dont have to download or build them again!

## Final steps

I was not able to refine the installation with Chef alone. Since I had to install some other npm tools (like the awesome Livescript), I resorted to a shell script to do this work1:

sudo chown -R vagrant.vagrant /usr/local/bin
sudo chown -R vagrant.vagrant /usr/local/lib
npm install -g LiveScript
npm install -g coffee-script
npm install express
npm install moment
npm install optimist
npm install less
npm install ansi-color


and that’s it. From now on, I have a few files to replicate my complete environment, whatever VPS I’ll choose in the future (even AWS!).

1. Note that I had to change the ownership of /usr/local/bin and /usr/local/lib to make the environment more similar to the brew based environment on my Mac (where you can install global node modules without invoking sudo).

## Slides of my talk at 'Making Computer Engineering a Science'

Very interesting talks in the same session such as ‘Towards Continuous Evaluation of Software Tools’ from Christoph Reichenbach (Johann-Wolfgang Goethe Universitat Frankfurt, Germany).

## Escaping callbacks with Livescript and Async

I spent some time evaluating ‘async’ frameworks for nodejs to create a wrapper around Rserve (a web based interface for R).

I had a look to IcedCoffeeScript, continuation.js, various kind of promises frameworks, and Livescript+Async.

Although these frameworks are all great, I was looking for features that allowed me to write clean and tidy code that was easily extendable.

It turns out that the syntactic sugar provided by Livescript toghether with ‘async’ was just what I was looking for.

I’ll copy and paste the code without any error management. Of course course, every interaction with the server should be checked out.

### Setup

In this post, we are going to use only ‘rserve-client’ and ‘async’:

r = require('rserve-client')
require! async


We also need some functions to format messages for Rserve:

formatRinvoke = (fun, args) ->
"toJSON(#{fun}(#{fJson jStrfy args}))"

jStrfy = (m) ->
"\'" + JSON.stringify(m) + "\'"

fJson = (m) ->
"fromJSON(#{m})"


### Connect to Rserve

To connect to rserve, we create a bound function and initialize the stateful object @client1 that will be used for all the requests:

connect = (cb) ~>
(err, client) <~ r.connect 'localhost', 6311
@client = client
cb(err)


The bound backcall created with <~ is converted by Livescript into a javascript callback, essentially splitting the above function in two parts. The code is equivalent to the following (note the indent madness due to callbacks):

connect = (cb) ~>
r.connect 'localhost', 6311, (err, client) ~>
@client = client
cb(err)


Note that the backcall should be ‘bound’, since we are referring to @client.

### Evaluate an expression

To evaluate an expression, the connection to ‘rserve’ must be done through the @client object created before:

eval-exp = (exp, cb) ~~>
(err, ans) <~ @client.eval exp
cb(err, ans)


eval-exp is a curried function (very peculiar to Livescript). The function can be partially evaluated to create another function. This is helpful when we want to evaluate different expressions, as we are going to see now.

### Invoking native R functions

We can define functions for invoking ‘R’ by using standard javascript hashes as arguments (that are converted to dataframes with the ‘rjson’ R-package) and use them to create more complex functions (as plotting into a pdf file):

invoke = (fun, args, cb) ~~>
(err, ans) <~ @client.eval formatRinvoke(fun,args)
console.log "#{formatRinvoke(fun,args)} -> #err, #ans"
cb(err, JSON.parse(ans))

plot-pdf = (data, filename, cb) ~~>
(err, res) <~ async.series [
eval-exp "pdf(\'#{process.cwd()}/#filename\')"
invoke "plot", data
eval-exp "dev.off()" ]
cb(err,res)


### Putting all toghether with async

As all the above functions are callback-based, we can use async to combine them in series for a small R-like script:

library = (name, cb) ~~>
(err, ans) <~ @client.eval "library('#name')"
cb(err)

err, res <- async.series [
connect
library('rjson')
eval-exp('1+1')
invoke "sin", [1,2,3]
plot-pdf {x: [1,2,3], y: [4,5,6]}, "prova.pdf"
]
console.log res


async.series combines, in series, the callback-based functions defined above. The final console.log is called when all the functions have been invoked.

### Conclusions

Overall, I like how Livescript allows to clean up callback-based code with minimal effort. In particular:

1. Curried functions allow to tidy up callback-based function generation; for example eval-exp('1+1') seems a normal function invocation but, instead, it generates a callback-based closure that is used by ‘async’.
2. The backcall operator <~ allows to tidy up callback-based functions by providing a seemingly sequential syntax for otherwise hell-ish nested callbacks.

1. @client is equivalent to this.client; this is the current module

## fsmexpress - specify, run, and debug finites state machines with Javascript

Just wrote a new npm package to work with state machines, since I thought that current packages weren’t funny enough.

fsmexpress provides an expressive way to specify, run and debug finite state machines in Javascript/Livescript/Coffeescript/Whatever:

Main features:

• Express compact state transitions with regular expressions
• Debug your FSM on line with a mini-server (powered by socketio).

Note: The fsm runs server-side! This is not compatible with browsers at the moment.

## Installation

To install, use npm:

npm install fsmexpress

## Example

For a complete livescript example, check this link out.

## Usage

Import the prototype in your program (livescript code)1:

fsm = require('fsmexpress').fsm;
any-of = require('fsmexpress').any-of;

### Create fsm and instantiate states

Create a finite state machine:

fs = new fsm()

Define states (livescript code):

fs.define-as-states([   'II' 'SI' 'PI' 'OI'
'IS' 'SS' 'PS' 'OS'
'IP' 'SP' 'PP' 'OP'
'IC' 'SC' 'PC' 'OC' 'error' ])

fs.define-as-initial('II')

### Define transitions

Define a transition (using a regular expression) from all states beginning with ‘I’, excluding some states (IP, IC) on a specific event (an_event) and register action action_to_trigger (function) when that happens:

fs.from('I(.+)')
.but-not-from(any-of(['IP' 'IC']))
.on('an_event')
.next-is('S-')
.but-before-do(action_to_trigger)

Note: the target state S- is a state beginning with S and ending with the matched text in (.+) in the from expression. So the above statement will generate only two different state transitions (because 'IP' 'IC' are not allowed from states:

II -> SI
IS -> SS

You can also define what to do when the starting state is not among the allowed ones (by using the otherwise-is function):

fs.from('(.+)S')
.but-not-from(any-of(['IS' 'SS' 'PS']))
.on('another_event')
.next-is('-P')
.otherwise-is('error')

In the above case, whenever the starting state is among ['IS' 'SS' 'PS'], the next state associated with another_event is error.

### Unfold and optimize

After the state transitions have been setup, invoke unfold to generate actual state transition rules:

fs.unfold()

Prune states that are not reachable:

fs.optimize()

### Linking to an event emitter

To register an event emitter:

fs.register-event-emitter(the_event_emitter)

So, everytime the_event_emitter emits a signal the fsm is triggered according to the rules. Practically, let’s assume that we have the following event emitter:

class tester extends EventEmitter

run_op: ~>
@emit 'anEvent'
setTimeout(@run_tr, 300)

run_tr: ~>
@emit 'anEvent2'
setTimeout(@run_fl, 300)

run_fl: ~>
@emit 'anotherEvent'
setTimeout(@run_op, 300)

# @emit 'triggerOpen'
# @emit 'executedOpen'

Let’s register it and start the finite state machine:

tst = new tester()

# Register event emitter and start the fsm
fs.register-event-emitter(tst)
fs.start()

# Start the event generation:
tst.run_op() 

State transitions will happen according to the emitted events.

## GUI debug

You can have a visual representation of the FSM that is served through a small web service (screenshot above):

red = "#9d261d"
gre = "#46a546"
blu = "#049cdb"

# GUI related stuff..
fs.prepare-emit()
fs.mark transition: '.+',       with-color: 'lightgrey'
fs.mark transition: '.+Open',   with-color: "#gre"
fs.mark transition: '.+Close',  with-dashed-color: "#gre"
fs.mark transition: 'failed.+', with-color: "indianred"
fs.mark state:      '.+',       with-color: 'lightgrey'
fs.mark state:      'error',    with-color: 'indianred'
fs.mark state:      fs.initial, with-color: "#gre"
fs.mark state:      fs.final,   with-color: "lightsteelblue"
console.log fs.data

fs.serve(6970, 'my fsm')

You can see live state transitions (wherever the fsm is, even remotely, provided that the port can be accessed).

1. In livescript, dashes “-” are used to create camelized Javascript identifiers. So, any-of is translated to anyOf by the livescript compiler.

## Tutorial Slides at the Conference on Advanced Topics on Auto Tuning in HPSC - Taipei, TW

Just finished the tutorial on Platform Customization. The slides can be found at this address.

## Keynote at the Conference on Advanced Topics and Auto Tuning - Taipei, Taiwan

Boarding for Taipei where I’ll be doing a plenary keynote on March 27th. Here’s the conference outline.

## Tobi Lütke on experience and learning

Nice quote from Shopify Founder and CEO:

Not that degrees matter anymore. They do not. Experience does. That is one of the things my apprenticeship and the dual education system in general taught me: experiencing and learning things quickly is the ultimate life skill. If you can do that, you can conjure up impossible situations for yourself over and over again and succeed.

.

## The next *mouse*

Supercool. Indeed.

The MYO armband lets you use the electrical activity in your muscles to wirelessly control your computer, phone, and other favorite digital technologies.