vpalos.com // milk, cookies, segfaults…

Fully dynamic JSON handling in Scala

by Valeriu Paloş on October 2, 2014

For some time now I (badly) wanted to be able to work with JSON structures in Scala just as easily as in JavaScript (though even in JS you need some wizardry to be able to to this elegantly).

I don’t particularly like Json4s since I feel that it’s overly complex (no, I don’t want to extract data into case classes using for-combinators), documentation is difficult to grasp and use (especially for edge cases) and Jackson and various other libraries are mainly oriented towards the Java-style JSON handling which basically means mapping to/from POJOs or similar.

What I wanted was to be able to navigate and/or alter a JSON structure directly using simple dot/call notations (e.g. json.server.hosts(2).name = "storage.mysite.com") and have a cheat-sheet-like documentation which whould (always) be simple and sufficient.

So here is my first swing at this using pure Scala parser combinators and other Scala-specific magic like Dynamics. The only dependencies (beside the Scala library) are the Scala parser combinators and Apache commons lang (I need this for string-escaping, especially Unicode stuff). License is Apache 2.0.

Disclaimer: This library is a first serious attempt at this problem and thus it is not optimized for high parsing speed, or low memory footprint; it does, however, deliver on ease of working with the JSON structure, which is the main goal. A second release is in the works which tries to optimize for better parsing speed and lower memory consumption.

Furthermore, I had no time yet to write a complete reference (i.e. the mentioned cheat-sheet), however the 120+ unit tests should be enough to get you going for now.

Check-out the sources here!


Normally you would just copy the com/vpalos/Json.scala file inside your source tree and ensure you bring the dependencies in your project (i.e. see build.gradle), but if you want to build a Jar or run the tests just do this (you should be connected to the Internet for all tests to pass):

git clone "https://github.com/vpalos/com.vpalos.Json.git"
cd "com.vpalos.Json"
./gradlew test
# ...or...
./gradlew build


Please visit the unit tests for a more complete overview! However, here are thre one-liners that should make you interested:

"""{"fox": {"quick": true, "purple": false}}""".toJson.fox.quick.asBoolean // true

"""{"points": [12.3, 991.8, 4.009, 1]}""".toJson.points(-2) // 4.009

Json.parse("{}").some.missing.field.isDefined // false, no exceptions thrown


Universal getter for plain JS objects.

by Valeriu Paloş on April 15, 2013

Many times in JavaScript we have to reach deep into objects to access fields that could possibly not be there at all; for example let’s assume I have a JS object called options into which I’ve just read an entire JSON configuration file. Trying to access a field like options.application.cache.enable directly will potentially fail (e.g. when any of the intermediate levels is missing)…

if (options.application.cache.enable) {
    // do stuff

…therefore this kind of work will usually involve some kind of testing for the existence of each level, so as to gracefully handle any misbehaving data. Also, many times I’ve seen the following approach used in these scenarios which is quite bad for anything more than a single depth level:

if (options && options.application && ...) {
    // do stuff

So, just as a little helper, here is a small method that may prove to be quite handy when having to deal with data whose structure we’re not certain of (here’s a proper Gist entry).
(more →)

Visualizing Git diffs in Meld.

by Valeriu Paloş on March 7, 2013

Here’s a quick way of configuring your (Linux-based) Git installation to use the excellent Meld program by default for displaying/editing the outputs of the git diff and git merge commands (e.g. git diff «filename»).

Install Meld

…if you haven’t already (note that we assume Git to be present):

# on Debian-based distros (ymmv)
sudo apt-get install meld

Viewing/editing file diffs via Meld

Configure Git to use Meld as the default tool for git diff:

git config --global diff.tool meld
git config --global difftool.prompt false

Now, rather than the classic git diff, simply use:

git difftool

Resolving merge conflicts via Meld

Basically do the same for the git mergetool command:

git config --global merge.tool meld
git config --global mergetool.prompt false

And then, instead of git merge, simply use:

git mergetool

…in the conflicting areas of your Git project.

Full directory comparison

Git supports full directory comparison via the following command:

git diff --no-index «folder-inside-git-project» «folder-to-compare-against»

While both useful and powerful, this command can be quite difficult to use, especially is you have a large tree structure. However, it just so happens that this is another aspect where Meld shines. Just run:

meld «folder A» «folder B»

And you will be able to visualize and edit the entire set off differences of both folder structures (including files).



  • The *tool.prompt false bits are needed to prevent the prompting of the user on each command invocation (which is the default behaviour).
  • The full directory comparison mode supported by Meld is obviously available for any folders, not just those inside Git projects.
  • I’m using Ubuntu 12.10, your setup may require different configuration options (although if you have a decently recent Git you should be fine).


by Valeriu Paloş on February 20, 2013

Again, I have to break the tradition to only post programming-related articles. This is because on the September 22nd 2012, this guy said hello. I know I was a bit late in letting him say say “Hi!” on the web too, but daddy’s been really busy and he forgot :(. But have no fear sonny-boy, here’s your shot…

Quickly filtering large lists of texts.

by Valeriu Paloş on December 19, 2012

Recently, I had to make a mock-up of a filtering function (something I need for a project I work on) so I can show-case it around. And since this mock-up seems to impress some people, I decided to share this function with everybody so others may enjoy the “loot” :).


The function (i.e. filter()) basically receives a query, a large list of text items and (optionally) some options. It then proceeds to filter that list based on the query and returns an array with the index numbers of the matching items and (optionally) an array with the marked texts of the matching items (i.e. highlighted captures). The documentation comment should provide all the needed insight on this thing.


Here you can play with the mocked-up demo (works on a list of ~23000 items).


Since I usually store these short snippets of code among my gists, you can find this function here. You should always check the gist for the most up-to-date code!

 * Demo: http://vpalos.com/sandbox/filter.js
 * A generic search algorithm designed for filtering (very) large lists of strings; when an input string
 * contains all the parts (words or characters; whitespace is ignored) of the query, spread-out over the text
 * then the string is considered to be a match. It works with the way internet browsers (e.g. Firefox, Google
 * Chrome) filter address-bar suggestions on user input. It is also quite fast; on my i7 laptop, filtering
 *     1) a list of ~23000 items takes around 50ms (yes, milliseconds!);
 *     2) a list of ~1 million text items took under 1 second.
 * It works both in NodeJS as well as in browser environments (so far I only tested FF and GC).
 * It has two functioning modes:
 * 1) word-mode: each (whitespace-separated) word in the input query must be found **whole** in the text:
 *     e.g. "foo bar" will match "123foo456bar789" but not "f oo ba r";
 * 2) charater-mode: the input query is matched per-character (whitespace is completely ignored):
 *     e.g. "foo bar" will match "f o o b a r" and even "-f.oo-ba.r-".
 * Options (values below are the defaults):
 * {
 *     "case": false,               // true: case-sensitive
 *                                  // false: case-insensitive
 *     "mark": true,                // true: returns item numbers + matches with highlighted captures
 *                                  // false: returns item numbers only
 *     "prefix": "<strong>",        // highlight prefix string
 *     "suffix": "</strong>",       // highlight suffix string
 *     "word": true,                // true: whole-word mode
 *                                  // false: character mode
 *     "limit": 0                   // limit results to this amount
 *                                  // 0 means return the whole result (unlimited)
 * }
 * @param {string} query The search query, consisting of space-separated chunks of characters.
 * @param {string|array} items A string or an array of strings to filter; if a string is given then it is
 *                             first split into an array of individual lines of text and then filtered (note
 *                             that).
 * @param {string} prefix String which will come before every highlighted substring (i.e. capture).
 * @param {string} suffix String which will come after every highlighted substring (i.e. capture).
 * @return {object} Returns the following object with matched items information:
 *                  {
 *                      "items": [...],     // array of matched item-numbers
 *                      "marks": [...]      // if mark == true, array of matches with highlighted captures
 *                  }
function filter(query, items, options) {

    // option producer
    function option(name, value) {
        options = options || {};
        return typeof(options[name]) !== 'undefined' ? options[name] : value;

    // prepare options
    var o_case   = option("case",   false);
    var o_mark   = option("mark",   true);
    var o_prefix = option("prefix", "<strong>");
    var o_suffix = option("suffix", "</strong>");
    var o_word   = option("word",   true);
    var o_limit  = option("limit",  0);

    // prepare query
    query  = o_case ? query : query.toLowerCase();
    query  = query.replace(/\s+/g, o_word ? ' ' : '');
    query  = query.replace(/(^\s+|\s+$)/g, '');
    query  = query.split(o_word ? ' ' : '');
    var ql = query.length;

    // prepare items
    if (typeof(items) === "string") {
        items = items.split('\n');

    // prepare results
    var matches = {
        items: [],
        marks: []

    // search
    for (var ii = 0, il = items.length; ii < il; ii++) {

        // prepare text
        var text = o_case ? items[ii] : items[ii].toLowerCase();
        var mark = "";

        // traverse
        var ti = 0;
        var wi = 0;
        var wl = 0;
        for (var qi = 0; qi < ql; qi++) {
            wl = query[qi].length;
            wi = text.indexOf(query[qi], ti);
            if (wi === -1) {
            if (o_mark) {
                if (wi > 0) {
                    mark += items[ii].slice(ti, wi);
                mark += o_prefix + items[ii].slice(wi, wi + wl) + o_suffix;
            ti = wi + wl;

        // capture
        if (qi == ql) {
            if (o_mark) {
                mark += items[ii].slice(ti);
            if (matches.items.push(ii) === o_limit && o_limit) {

    // ready
    return matches;

[Update] Thanks to vladpalos for pointing-out some flaws in my initial code which, in the end, led me to simply using String.indexOf() for finding sub-matches.


Let me know how this can be improved!

JS classes for the masses.

by Valeriu Paloş on December 1, 2012

Following on an older post of mine, I decided to write a new article about an(other) implementation of OOP Classes in JavaScript. This is something that brings a lot of value from very few lines of code and I want to share it with everyone and see how it can be improved.

First let’s see the code (you should check here for the latest version) and then I’ll try and explain the relevant benefits/features but jump straight to the goodies if you like, knock yourself out! 🙂

 * Class.js: A class factory.
function Class(members) {

    // setup proxy
    var Proxy = function() {};
    Proxy.prototype = (members.base || Class).prototype;

    // setup constructor
    members.init = members.init || function() {
        if (Proxy.prototype.hasOwnProperty("init")) {
            Proxy.prototype.init.apply(this, arguments);
    var Shell = members.init;

    // setup inheritance
    Shell.prototype = new Proxy();
    Shell.prototype.base = Proxy.prototype;

    // setup identity
    Shell.prototype.constructor = Shell;

    // setup augmentation
    Shell.grow = function(items) {
        for (var item in items) {
            if (!Shell.prototype.hasOwnProperty(item)) {
                Shell.prototype[item] = items[item];
        return Shell;

    // attach members and return the new class
    return Shell.grow(members);


Each Class points to (via .prototype) a corresponding Proxy object that holds the class’s member fields. In turn, this Proxy points to (via .prototype) the Proxy of the inherited class (i.e. the super-Class) and so on. So, each Class points to its own Proxy which points to the ancestor Proxy and so on, until the top level ancestor is reached. (more →)

Loomiere 2.0.1-beta finally out!

by Valeriu Paloş on July 17, 2012

All right! I finally got around to releasing this code to the public. I won’t say much now (since I’m tired) but I will say that this software has been functioning in production environment for over 1 year and a half serving all the video content available on the romanian website http://peteava.ro (about 800 TB of data each month out of 14 servers).

Some (old) statistics showing it’s power can be found here.

Source code: https://github.com/valeriupalos/loomiere.

Any software downloaded from this website is *never* to be associated in any form with pornographic or erotic content! I.E. “Loomiere” must *never* be used to stream pornographic or erotic videos! There are plenty alternatives if you can’t help it.

Lua-TTyrant: a TokyoTyrant binding for Lua.

by Valeriu Paloş on February 19, 2012

Some time ago I started a small project to implement a C-to-Lua binding of the TokyoTyrant library, called “lua-ttyrant“. A few days ago, while working with a script that was actually using it, I decided to polish it up, finish it and eventually submit it to be included in the LuaRocks repositories. Given the fact that the “rock” was just accepted by the LuaRocks team, I thought a new post was in order.


The lua-ttyrant source code can be found here (among my github repos).
The lua-ttyrant rockspec can be found here (among all the public Lua rocks).


  • TokyoTyant library and headers.
  • LuaRocks.

Prerequisites on Debian/Ubuntu distros:

sudo apt-get install libtokyotyrant-dev


sudo luarocks install lua-ttyrant


The binding is in beta for now. Please contribute a ticket if you find anything wrong with it. For anything else please see the doc/DETAILS file. Enjoy! 🙂