Josh On Design

Functional Reactive Programming with Bacon.js

Lately I've been experimenting with Functional Reactive Programming, or FRP. There are several good libraries to use FRP with Javascript. I chose Bacon.js. I'm finding FRP to be very useful but hard to understand.

The way FRP is explained often leads to confusion. Most people know what the P is, and the F seems fairly understandable, but the R causes confusion. Examples usually talk about the difference between expressions and statements. Rather than c = a + b setting a value right now, it is an expression which defines that c is always a plus b. It defines a relationship. In another life I would have called this a binding expression. I suppose expressions get to the heart of what FRP is but it doesn't really explain why it's useful. Let me try another way.

FRP is this: working with streams of values that change over time.

Perhaps an example would help. Imagine moving a mouse over your browser. It produces a stream of x and y values. Rather than using a callback for every mouse move, we can work with the mouse events as a single object over time: an event stream. Suppose we want to know when the mouse moves past a line on the screen at 100px. In regular code we could do this:

$("body").on('mousemove', function(e) {
     if(e.pageX > 100) {
          console.log("we are over 100");
     }
};

With FRP we would create a stream based on mouse move, then filter it to only have X values over 100, like this:

$("body").toEventStream('mousemove')
     .filter(function(v) { return v > 100; })
     .onValue(function(v) {
          console.log("we are over 100: " + v);
     });

This is roughly the same amount of code so it doesn't seem like a big improvement. Trust me, it is. We have separated the action, printing a message, from the source of the stream and any filter operations. We can also add more operations to the stream if we want, and abstract the filters out further.

//create a stream of clicks
var clicked = $('body').asEventStream('click');
//look for clicks for no longer than 300ms
clicked.bufferWithTimeOrCount(300, 2)
     .filter(function(x) {
          return x.length == 2;
          })
     .onValue(function(x) {
          console.log("double clicked: ", x);
     });

The magic part here is bufferWithTimeOrCount(). It buffers the clicks, sending them to the next stage in the pipeline only if there are two samples or it has been 300ms since the first click. This means the filter function will get an array that is either one or two elements long. If it's one element then it clearly wasn't a double click, so we can filter to include just the two clicks. We already know they are no more than 300ms apart, so we are done.

This code still looks a bit clunky, though. I don't like all of those nested functions. JavasSript has a way to deal with this: partial application and function references. In JavaScript a function is a real object so you can pass it around with variables. Let's make some reusable functions for printing to the log and determining if the length of an array is 2.

var isTwoValues = function(x) { return (x.length == 2); }
var log = function(str) {
     return function(obj) {
          console.log(str,obj);
     }
}

isTwoValues is pretty simple: return true if the array is two elements. The log function is a bit trickier. It doesn't print to the console. Instead it returns a new function which prints to the console. However, it still remembers the first string passed in, so the new function can combine the original string with whatever new object is to be printed. In functional programming this is called a partially applied function. The returned function already has the first argument, str, applied. Later it, when it gets the second argument, it will fully apply the function. These sorts of functions are very handy when dealing with deferred programing like stream processing and event handling. Let's update the code from before using our new functions:

clicked.bufferWithTimeOrCount(300,2)
     .filter(isTwoValues)
     .onValue(log("double clicked");

Ah. Much nicer looking.

Using this same system we could implement a press and hold detector like this:

clicked.bufferWithTimeOrCount(600,2)
     .filter(isOneValue)
     .filter(function(x) return (x[0] == true); }) 
           //x[0] = true if it's a mouse down. false if it's a mouse up
     .onValue(log("hold"));

Now lets try something really tricky to show off the power of this approach: a swipe detector. A swipe on a touch screen is when the cursor point is moving faster than a certain velocity. Velocity is tricky, however, because we need to calculate an average over multiple samples, and do it in both the x and y directions, and only when the mouse is down. First let's filter out mouse moves when the mouse isn't down, then calculate the difference between each event and the previous one.

mouse_move
    .filter(mouse_down.toProperty())
    .diff(null, function(a,b) {
            if(a) {
                return {
                    dx: Math.abs(b.pageX-a.pageX),
                    dy: Math.abs(b.pageY-a.pageY),
                    dt: b.timeStamp - a.timeStamp
                }
            } else {
                return { dx:0, dy:0, dt: 0 }
            }
    })

Pretty straight forward. Now we want to calculate the average of the last five samples. The slidingWindow function makes this easy.

    //the five most recent samples
    .slidingWindow(5,5)
    //calculate the sum of the samples and velocity
    .map(function(v) {
            var tx = 0; var ty = 0; var tt = 0;
            for(i in v) {
                tx += v[i].dx;
                ty += v[i].dy;
                tt += v[i].dt;
            }
            var txy = Math.sqrt(tx*tx + ty*ty);
            return txy/tt*1000;
    })

Finally, let's only fire an event when the value is greater than 2000

    .filter(greaterThan(2000))
    .onValue(log("current velocity in pixels per second =, > 2000"));

See this code in action here

Now we have a swipe detector that is robust and extendable. Most importantly, all of these detectors can coexist without interfering with each other, and without a bunch of state variables lying around. To improve this we could make it indicate which direction the swipe took place, and if it started on the edge of the screen or the middle. FRP makes it easy to add these features without breaking existing code.

To learn more about Bacon.js, check out the github project and theses tutorials.

Concise Computing: a PNG Parser in 20 lines

Mon Mar 18 2013 22:07:33 GMT+0000 (UTC)

By what standard should we measure if code is "beautiful"? I argue it should be not lines or speed, but rather conciseness and clarity. Can someone who is not familiar with the language still understand what the algorithm does? Can someone not familiar with the task still get a feel for how it works? This metric favors shorter code over longer, but not at the expense of readability. Beautiful code should be as close to expressing the underlying algorithm as possible. How close is the actual code to the most straight forward pseudo code?

Some will argue that an elegant implementation is useless if it is too slow; but what if the elegant version is only 5% slower? Is that a good tradeoff? Almost certainly. What if it's 10x slower but only runs for 1% of the time? Then the trade off might be worth it. It depends. Certainly it's good to start with the clear and concise version and only move to a more complex implementation if the speed profiling warrants it. This is what I like to call concise computing.

Complexity

Concise code is easier to read, easier to write, and much, much, much easier to maintain. Fewer lines of code means fewer places for bugs to hide. But let me take it from another angle:

If you open up the typical programming library, especially older ones, you will find a bunch of complex code; often difficult to understand. This is the case even if the underlying task should be straight forward. Why should code be like this?

I think there are two reasons: first, the programming language used to write the library wasn't expressive enough to write the implementation in a way that looks like the algorithm. If a task is naturally described using objects then a language with out direct object support will be at a disadvantage. If the task is really about matching patterns, then a language without matching will necessarily represent the algorithm in a less elegant way. And of course once written we try not to rewrite our code for fear of breaking things. Code is instant legacy.

The other reason for complex libraries is speed. While I can describe sound synthesis using pure math... math is expensive to compute. A set of pre-calculated wavetables will be far faster than raw cosine functions. Changing an algorithm to be fast often makes the code hard to understand. But is this really a good excuse?

Our software doesn't have to be fast. It has to be fast enough. But enough depends entirely on context and it changes over time as hardware evolves. Math.sin() used to be hugely expensive. When John Carmack wrote Doom he would massively change code to save a few divides. The code was amazing but came at a cost of obscuring the underlying algorithm and being hard to maintain. It doesn't have to be like that anymore.

Ometa

I did not invent concise computing (though I haven't heard the specific term before). It has been a computing goal for over fifty years. The sweet spot for any implementation has shifted over the years as hardware changes, but I think the time has come for most of our coding to move in the direction of conciseness. We have such an excess of CPU power, even in our mobile devices, that in almost every case the concise version wins over a fast but complex one. However, let me stop with the theoretical discussion and give you a concrete example of what I'm saying.

I recently discovered a language called OMeta. OMeta is actually a extension to let you write DSLs in a general purpose host language. My preferred version is OMeta/JS, which extends JavaScript, but there are implementations for SmallTalk, C#, and others. OMeta really exemplifies my thoughts on concise computing.

Alex Warth, the creator of OMeta, realized many computing tasks, especially writing parsers and compliers, are just different forms of matching. So why not bake it into the language in a way that is convenient but still lets you get at the power of the host language, and treat the parsers as objects that can be extended in various ways. Here's what a simple calculator looks like:

SimpleCalc <: Ometa {
   exp = number | sub | add,
   sub = exp:a "-" exp:b =<  (a-b),
   add = exp:a "+" exp:b =< (a+b),
}

result = new SimpleCalc().matchAll("6+7);  //result is set to 13

Even if you don't know anything about OMeta you can follow what is going on. This says that exp is a number or a pattern called sub or add. Then it defines sub and add with recursive definitions back to exp. The => part says that the matched expression goes to a small Javascript expression, in this case the part that does the actual math.

This code looks a lot like your typical BNF grammar. The magic here is that everything after the => part is regular Javascript. Instead of (a+b) it could be console.log("doing an add"), or call some other function to add in base 16 math, or generate C fucntions. Normally I would have to break out to a special library with weird syntax, or write in a completely different language. OMeta lets us implement algorithms that are naturally expressed with matching, but still remain in our general purpose host language, not some special matching only parser. This is concise computing.

OMeta was created as part of Alex Warth's graduate thesis. You should definitely check it out. For an academic paper it is highly readable. Alex continued his work at VPRI, a research group set up by Alan Kay. Yes, that Alan Kay. Though Alex's OMeta is an excellent improvement the original idea goes back to Meta II, a meta-compiler created back in 1962!!! The battle for concise computing is far from recent.

A PNG Parser in OMeta

Most public uses of OMeta show small examples like the calculator above or language compilers, which is probably it's most obvious application. (I myself am working on my own cross-language system with OMeta which I'll detail in a future blog post). Earlier this month I showed you a markdown word processor with embedded arithmetic expressions built in OMeta. However, today I want to show you a different kind of task: parsing a binary data PNG file in JavaScript.

PNG is a fairly simple data format yet many parsers are surprisingly complex. As documented in Wikipedia, PNG consists of a header followed by a series of chunks. Each chunk has a length, type, and payload. Most of the chunks contain metadata about the image. Only one, IDAT, contains a bitmap. The example below does not decompress the actual image data (which requires a DEFLATE decompressor) , but rather parses and validates all of the metadata chunks; the most likely use for an external parser.

Parsing binary data in Javascript has always required hacks because JS has no native binary types. Fortunately, the W3C and Khronos recently remedied this situation with Typed Arrays, a new data type built to support textures and vertices for WebGL. Using typed arrays we can fetch a PNG file from a server and convert it into a large buffer; like this:

function fetchBinary() {
    var req = new XMLHttpRequest();
    req.open("GET","icon.png",true);
    req.responseType = "arraybuffer";
    req.onload = function(e) {
        console.log("loaded");
        var buf = req.response;
        if(buf) {
            var byteArray = new Uint8Array(buf);
            console.log("got " + byteArray.byteLength + " bytes");
            var arr = [];
            for(var i=0; i



The Uint8Array is an array of unsigned bytes (8 bit long 'words').  With this array in hand we can parse the entire structure in less than 20 lines of Ometa. Again, even if you don't know how Ometa works you can get a vague idea of what the program does by reading it.

ometa BinaryParser <: Parser {
    //entire PNG stream
    start  = [header:h (chunk+):c number*:n] -> [h,c,n],
   
    //chunk definition
    chunk  = int4:len str4:t apply(t,len):d byte4:crc
        -> [#chunk, [#type, t], [#length, len], [#data, d], [#crc, crc]],
   
    //chunk types
    IHDR :len  = int4:w int4:h byte:dep byte:type byte:comp byte:filter byte:inter
        -> {type:"IHDR", data:{width:w, height:h, bitdepth:dep, colortype:type, compression:comp, filter:filter, interlace:inter}},
    gAMA :len  = int4:g                  -> {type:"gAMA",value:g},
    pHYs :len  = int4:x int4:y byte:u    -> {type:"pHYs", x:x, y:y, units:u},
    tEXt :len  = repeat('byte',len):d    -> {type:"tEXt", data:toAscii(d)},
    tIME :len  = int2:y byte:mo byte:day byte:hr byte:min byte:sec
        -> {type:"tIME", year:y, month:mo, day:day, hour:hr, minute:min, second:sec},
    IDAT :len  = repeat('byte',len):d    -> {type:"IDAT", data:"omitted"},
    IEND :len  = repeat('byte',len):d    -> {type:"IEND"},
   
    //useful definitions
    byte    = number,
    header  = 137 80 78 71 13 10 26 10    -> "PNG HEADER",        //mandatory header
    int2    = byte:a byte:b               -> bytesToInt2(a,b),    //2 bytes to a 16bit integer
    int4    = byte:a byte:b byte:c byte:d -> bytesToInt(a,b,c,d), //4 bytes to 32bit integer
    str4    = byte:a byte:b byte:c byte:d -> toAscii([a,b,c,d]),  //4 byte string
    byte4   = repeat('byte',4):d -> d,
   
   
       
    END
}


This parser returns a JSON structure that we can loop through to find any chunk we care about.
chunk: {"type":"IHDR","data":{"width":33,"height":36,"bitdepth":8,"colortype":2,"compression":0,"filter":0,"interlace":0}}
chunk: {"type":"gAMA","value":55555}
chunk: {"type":"pHYs","x":2835,"y":2835,"units":1}
chunk: {"type":"tEXt","data":"Software0x0 QuickTime 6.5.2 (Mac OS X)0x0 "}
chunk: {"type":"tIME","year":2005,"month":4,"day":5,"hour":17,"minute":6,"second":20}
chunk: {"type":"IDAT","data":"omitted"}
chunk: {"type":"IEND"}
In a sense the code above is not really an algorithm, but rather a definition of the PNG format. A runnable specification. For an even more impressive example,  the OMeta guys created a TCP/IP parser which uses the actual ASCII art header definitions from the RFC as the parser code itself.

More

What I've shown you today is merely a hint of what concise computing could be.  Imagine an entire operating system from boot to GUI, complete with apps, written in less than 20k lines of code? It sounds crazy yet that is what VPRI has done.

By writing the most clear and concise code possible we can make our software more maintainable and longer lived. Next time I'll show you the same principle applied to audio synthesis. Feel free to ask questions and continue the discussion.



A Magically Delicious Wordprocessor with OMeta
Tue Mar 05 2013 22:39:47 GMT+0000 (UTC)
I've always wanted a magic wordprocessor. Something that helps me organize my thoughts and build ideas organically, rather than spend all of my time worrying about formatting. Something for the internet / cloud age.  Given that Microsoft Word hasn't fundamentally changed in over a decade (or possibly two), we aren't likely to get such software from them. Instead, I decided to play around with some ideas using my new favorite programming tool: OMeta.

OMeta

OMeta's story starts with compilers. Compilers are built of many phases: lexing, parsing, applying optimizations, then generating machine code. OMeta's creator, Alessandro Warth, realized that these are all essentially the same thing, pattern matching. The only difference is whether you are matching a stream of characters, tokens, or parts of a tree. Alex created a single language that lets you match over objects (ie: anything) to implement all parts of the compiler using a single tool.

OMeta is that single tool: a domain specific programming language for creating pattern matchers. More properly, it is a language extension that can be laid on top of an existing host language. I am using OmetaJS, an extension to JavaScript. The wonderful thing about OMeta is that it's easier to use and extend than dedicated parser generators like Antlr or Yacc because you always have the host language available to do whatever computation you need.

Editor Demo

As my first experiment with OMeta, and new wordprocessors, I created a simple Markdown editor. You type raw text into the left side of the screen and see your properly styled text on the right side.  But this editor has a trick up it's sleeve. You can type simple math expressions inside curly braces the editor will evaluate it for you.

Play with the editor here

Here's what the OMeta code looks like:


ometa Foo {
    //white space
    toEOL = (~seq('\n') char)*:t '\n' -> t.join(""),
    
    //headers
    h1 = "#"   ' ' toEOL:t -> tag("h1",t),
    h2 = "##"  ' ' toEOL:t -> tag("h2",t),
    h3 = "###" ' ' toEOL:t -> tag("h3",t),

    
    //paragraph
    paraend   = seq('\n\n'),
    para      = (expr|strong|em| (~paraend char))+:t -> tag("p",t.join("")),
    text   =      (~seq('\n\n') char)*:t      -> t.join(""),
    strong = "**" (~seq('**')   char)*:t "**" -> tag("strong",t.join("")),
    em     = "*"  (~seq('*')    char)*:t "*"  -> tag("em",t.join("")),
    expr    = "{"  exp:t "}"  -> tag("b",t),
    
    //code block
    codeblock = fromTo("```\n", "```\n"):t -> tag("pre",tag("code",esc(t))),
    
    //inline expressions
    num = :n -> parseInt(n),
    term = num,
    expadd = term:a "+" term:b -> (a+b),
    expmul = term:a "*" term:b -> (a*b),
    expsub = term:a "-" term:b -> (a-b),
    expdiv = term:a "/" term:b -> (a/b),
    exp = (expmul|expdiv|expadd|expsub):e -> (" "+e+"") ,
    
    //pull it all together
    line = space* (h3|h2|h1|codeblock|para):t space* -> t,
    top = line*,
    
    END
};


This editor only understands simple two term arithmetic but it could easily be expanded with variables and more complex functions.


Improved Easing Functions
Fri Mar 01 2013 06:17:03 GMT+0000 (UTC)
Animation is just moving something over time. The rate at which the something moves is defined by a function called an easing equation or interpolation function. It is these equations which make something move slowly at the start and speed up, or slow down near the end. These equations give animation a more life like feel. The most common set of easing equations come from Robert Penner's book and webpage.  

Penner created a very complete set of equations, including some fun ones like 'spring' and 'bounce'. Unfortunately their formulation isn't the best.  Here is one of them in JavaScript



easeInCubic:function(x,t,b,c,d){
returnc*(t/=d)*t*t+b;
},
easeOutCubic:function(x,t,b,c,d){
returnc*((t=t/d-1)*t*t+1)+b;
},


They really obscure the meaning of the equations, making them hard to understand. I think they were designed this way for efficiency reasons since they were first implemented in Flash ActionScript, where speed would be important on the in-efficient. It makes them much harder to understand and extend, however. Fortunately, we can refactor it to be a lot clearer.

Let's start with the easeInCubic equation. Ignore the x parameter (I'm not sure why it's there since it's not used).  t is the current time, starting at zero. d is the duration in time. b and c are the starting and ending values. 

easeInCubic:function(x,t,b,c,d){
returnc*(t/=d)*t*t+b;
},


If we divide t by d before calling this function, then t will always be in the range of 0 to 1, and d can be left out of the function. If we define the returned version of t to also be from 0 to 1, then the actual interpolation of b to c can also be done outside of the function. Here is some code which will call the easing equation then interpolate the results:

                var t = t/d;
                t = easeInCubic(t);
                var val = b + t*(c-b);


We have moved all of the common code outside the actual easing equation. What that leaves is a value t from 0 to 1 which we must transform into a different t.  Here's the new easeInCubic function.


        function easeInCubic(t) {
            return Math.pow(t,3);
        }


That is the essence of the equation. Simply raising t to the power of three. The other formulation might be slightly faster, but it's very confusing, and the speed difference today is largely irrelevant since modern VMs can easily optimize the cleaner form.

Now let's try to transform the second one.  An ease out is the same as an ease in except in reverse.  If t when from 0 to 1 then the out version will go from 1 - 0. To get this we subtract t from 1 as shown:

        function cubicOut(t) {
            return 1-Math.pow(1-t,3);
        }


However, this looks awfully close to the easeIn version. Rather than writing a new equation we can factor out the differences. Subtract t from 1 before passing it in, then subtract the result from one after getting the return value.  The out form just invokes the in form:

function easeOutCubic(t) {
     return 1 - easeInCubic(1-t);
}

Now we can write other equations in a similarly compact form. easeInQuad and easeOutQuad go from:

easeInQuad:function(x,t,b,c,d){
returnc*(t/=d)*t+b;
},
easeOutQuad:function(x,t,b,c,d){
return-c*(t/=d)*(t-2)+b;
},

to

function easeInQuad(t) {  return t*t; }
function easeOutQuad(t) { return 1-easeInQuad(1-t); }

Now let's consider the easeInOutCubic. This one smooths both ends of the equation.  In reality it's just scaling the easeIn to the first half of the t, from 0 to 0.5. Then it applies an easeOut to the second half, from 0.5 to 1.  Rather than this complex form:

easeInOutQuad:function(x,t,b,c,d){
if((t/=d/2)<1)returnc/2*t*t+b;
return-c/2*((--t)*(t-2)-1)+b;
},


We can compose our previous functions to define it like so:

        function cubicInOut(t) {
            if(t < 0.5) return cubicIn(t*2.0)/2.0;
            return 1-cubicIn((1-t)*2)/2;                
        }


Much cleaner.  

Here is the original form of elastic out, which gives you a cartoon like bouncing effect:

easeOutElastic:function(x,t,b,c,d){
vars=1.70158;varp=0;vara=c;
if(t==0)returnb;if((t/=d)==1)returnb+c;if(!p)p=d*.3;
if(a


and here is the reduced form:

        function easeOutElastic(t) {
            var p = 0.3;
            return Math.pow(2,-10*t) * Math.sin((t-p/4)*(2*Math.PI)/p) + 1;
        }


Moral of the story: "Math is your friend" and "always refactor".

These new equations will be included in a future release of Amino, my cross platform graphics library.


Make Libraries More
Wed Feb 13 2013 18:00:02 GMT+0000 (UTC)
Software Libraries are good. They allow abstraction and encapsulation; which encourages reuse. They also allow the library to be written by one person and used by another. This is reuse at the programmer level, not just the system level. A library can be written by a person who has domain expertise but used by someone else who has less or no expertise in that domain. For example: an XML parser. The implementer knows the XML spec inside and out, but the user of the lib needs only a basic understanding of XML in order to use the lib.

Now that we've established that libraries are good (regardless of whether are are implemented as classes, packages, DLLs, etc.), how can we extend them?  Traditionally libraries are very limited in how they can influence the experience of the programmer.  The coder simply has new functions to call.  A more advanced case is component driven GUI builders where the lib exposes more information so that the IDE can make the library easier to use. Annotating of types in a map component could enhance the GUI builder draw markers, use a dummy tile set,  and set positions.  This is all standard stuff. Could libraries do more? How could the library author further influence the experience of the library user?

 Let's do a quick brainstorm:



Extend the language with new operators and syntax specific to the library.  A physics library could import operator overloads to enable more math like syntax when dealing with physical units and vectors.

Tests that verify aspects of the code that calls the library. This could automate the advice that would be in the documentation, such as "don't pass null to certain functions" or "this function returns an object that you must dispose of".

If the lib is used from within an IDE, the lib can insert new documentation into the searchable index of docs.
The lib could come with example code and reusable annotated code snippets that would only be applied to projects using the lib.
The lib could insert a new package manager into the compiler toolchain. No more messing with Maven POMs.
The lib can provide the compiler with new code optimizers for the particular tasks. An imaging analysis library could instruct the compiler on how to better use SIMD instructions when available.




Obviously modifying the environment of the programmer is a dangerous activity so we would need very strict scoping on the changes. A library which extends the language applies only in the parts of the code that actually import the library. Changes are scoped to the code which asks for them.

Fundamentally a library is a thing created by one programmer for use by another programmer (or the same programmer at a future date).  We should extend the concept of a library to apply to the entire programming experience, not just adding objects and methods. This would allow all environment extensions to be managed in a standard way that is more understandable, flexible, and scoped. I would never need to set up a makefile again. I could simply import in my main.foo code the package manager, repos, code chunks and syntax that I want to use for this project.  The library extensions to the compiler handle everything else automatically.

Clearly there are challenges with such an approach.  We would have to define all of the various ways a library could modify your environment.  How would compiler hooks actually work? Annotation processors? A function to modify the AST a different stages of compilation? Would it imply a particular back end like LLVM?  I don't have the answers. There clearly are challenges to be solved, but I think the approach is a step forward from today where we have many different components of our programming environment that are dependent on each other but don't actually know about each other.  Having a way to organize this mess, even if imperfect, would certainly help us make better software.