Norbert Hartl

mostly brackets and pipes

System Monitoring for Pharo Images

In our daily programming tasks we all know that having tests is great way of keeping our software under control. It is something that we regularly execute in order to see that we didn’t break any of our previous assumptions. If we are keen on tests we even try to have a 100% code coverage meaning that every available code path is executed at least once.

Deploying the software adds additional precautions to take. Apart from having a software running without errors it also needs to deal with resources that are available. And resources run short sometimes. Having the software tested for execution doesn’t tell us it doesn’t waste resources which prevents the software from running a long time. The software can leak memory which is unseen while we execute isolated single test cases. Having the software run for a longer time will reveal this problem.

Even if there isn’t a significant leakage of resources the software can run short on resources if used more frequently. There is a point where we need to think about scaling the deployment to meet the resources needed. Regardless where the shortage comes from we need a way to see an immediate problem or to see a tendency that projects a shortage in an overseeable time frame.

In order to deal with that system monitoring comes into play. System monitoring is the task of inspecting the resources of an environment all the time. These inspections of resources are combined with defined thresholds we find feasible with the environment. Whenever are threshold is exceeded the inspecting software can send us alerts making us have a look at the software before it is going to break. The inspected values of the resources can also be graphed so we can see tendencies of resource usage over time.

I’ve implemented a utility that helps monitoring a pharo image. It is called Monitoring. The source is availble on smalltalkhub

Install a fresh pharo image (pharo 4.0 at the moment)

$ curl | bash

Open the image using

$ ./pharo-ui Pharo.image

Open a playground and load the project with the following expression

Gofer it
    smalltalkhubUser: 'NorbertHartl' project: 'Monitoring';
    configurationOf: 'Monitoring';

The following text describes some ways how to enable monitoring of a pharo system.


Munin is a tool that can produce graphs like this one.

a memory graph produced by munin

It inspects the image every 5 minutes and puts the inquired values in a RRDTool. From this tool it creates graphs for overviews over day, week, month and year. This enables use to see the progression of resource usage over a whole year which makes it quite easy to project something into the future.

If you’ve installed the project with the snippet above you should now execute the following in the playground (formerly known as workspace).

MonitorMuninExampleServer image

It starts a zinc server on port 5000 and adds everything needed to monitor vm memory and the garbage collector.

You can request it using curl and the following command line

$ curl

and you should see something like this (the values will be different of course)

oldspace.value 163980984
youngspace.value 1336248
tenures.value 165317588
free.value 5123340

A munin graph needs to be configured. We can request that using the command

$ curl

You should see this

graph_title VM memory
graph_category test-server
graph_vlabel bytes
graph_args --base 1000
memory.label Memory
memory.warning 60000000
memory.critical 80000000
memory.type GAUGE
oldspace.label Oldspace
oldspace.type GAUGE
youngspace.label Youngspace
youngspace.type GAUGE
tenures.label Tenures
tenures.type GAUGE
free.label Free
free.type GAUGE

The configuration of the graph is read by munin at startup. It configures the graph and then every 5 minutes it requests the value and is able to plot it.

Now we need to install munin. In linux execute

$ apt-get install munin

adding a web server

Installing a web server depends highly on which web server you like to use and what environment you are targeted for. In depth configurations of several web servers is beyond the scope of this post. I’ll take a shortcut here to just get you something to see. I assume an actual installation of ubuntu (in my case this is a 14.10 distribution). I leave the fine grained configuration as an exercise for the reader. You can find the munin documentation here

The default installation of munin includes a default configuration that is suitable to us. It assumes everything is local. If you need to adjust it you can find the configuration in /etc/munin/munin.conf.

This example uses nginx and assumes you don’t have it installed yet. Execute

$ apt-get install nginx

and add the following

    location /munin/static/ {
            alias /etc/munin/static/;
            expires modified +1w;

    location /munin/ {
            alias /var/cache/munin/www/;
            expires modified +310s;

in the server section of


You will find a location statement there already. Just add the above on the same level as the other statement. After you saved the file restart the nginx server by doing

$ service nginx restart

If everything went well you should be able to open the following URI using a web browser


and you should see something like this

a memory graph produced by munin

Connecting the parts

We have now downloaded a pharo image, installed the Monitoring project, started a pharo zinc server, installed munin, installed and configured a web server. Now the only thing we need to do is to add our memory graph to munin.

Create a file /etc/munin/plugin-conf.d/pharo-monitoring with the following content


This tells munin that every plugin that has a prefix of test-server_ will get an environment containing a variable monitorUrl pointing to our pharo image.

Next we need to create a munin plugin (a shell script that translates the way munin calls its plugins to our zinc server). I’ve made a git repository to ease that task. Install it using

$ cd /opt
$ git clone 

Activate the plugin using

$ cd /etc/munin/plugins
$ ln -s /opt/pharo-scripts/monitoring/munin-plugin test-server_memory

We created the plugin with the prefix test-server_ so munin will assign it the monitorUrl we configured above. The script takes the suffix after the underscore to construct the complete URI.

You can test it by issuing

$ munin-run test-server_memory

that should show the values we saw above and

$ munin-run test-server_memory config

that should show the configuration of the graph from above. The last two invocations of the script are exactly how munin will call them and are a great way to debug problems like permissions et al.

If everything went well we can restart munin

$ service munin-node restart

and after a while the new plugin should appear like you can see here.

the plugin is there

Clicking on the test-server link shows us

more to come

and you can see on the right side of the left graph there are few lines starting to build up. Now we just have to wait for the data to fill the graph.


I might say this is an easy way to start monitoring pharo images. But as this blog shows there are quite some things to set up in order to have it work. These are all basic building blocks like installing and configuring a web server and doing system administration tasks. I assume that if you are interested in system monitoring you know most of this already and you can easily translates everything that has been written here into your own environment.

This post did a quick and rough shortcut throughout the components that need play together. I tried every step but I’m sure there are a lot of things that can go wrong like permission problems or the like. If you have problems don’t hesitate to bug me about it.

If you got this far the future of monitoring is bright. The next task to monitor your garbage collector is as simple as

$ cd /etc/munin/plugins
$ ln -s /opt/pharo-scripts/monitoring/munin-plugin test-server_gc
$ service munin-node restart

and wait. The graph will appear automatically. You are now prepared to orchestrate your whole server farm. With munin it is quite easy to collect graphs from a lot of hosts and combine them in a single web page. The example server sets also some thresholds for warning and critical. Just waste a bit memory and see the label in the graph turn yellow in case of exceeding the warning threshold and read in case of critical. You can configure munin the send e.g. an email if the thresholds are exceeded.

This should be enough for an introduction to system monitoring in pharo. In following articles I will present more tools and I’ll dig into the code and explain how you can easily monitor any parameter of your system you like.

As always I like to have feedback about things being good and things being bad. Hope you enjoy system monitoring your pharo images.

Happy Monitoring!

Mustache Templates for Smalltalk

Mustache is a popular templating engine that is supported in many programming languages. I first encountered them in javascript while researching web toolkits. The syntax is small and covers a wide range of use cases. Maybe I should say that I didn’t find something I can’t do with it so far. Although it was designed to be a templating engine for HTML pages it is useful in different areas.

I have done an implementation for Mustache in smalltalk. It is available at smalltalkhub. There is documentation available on how the syntax is to be used. Here are a few examples to show the smalltalk methods you have to call in order to make it work.

How to use?

Disclaimer: I’m talking about an implementation for smalltalk but to be honest I developed and tested it only on pharo smalltalk 2.0. If you need it for another dialect drop me note and we see what we can do.

A simple Mustache template looks like this (taken from the documentation):

templateString := 'Hello {{ name }}
You have just won ${{value}}!
Well, ${{taxed_value}}, after taxes.

Given a context object with content

context := {
  'name' -> 'Chris'.
  'value' -> 10000.
  'taxed_value' -> (10000 - (10000 * 0.4)).
  'in_ca' -> true } asDictionary

we execute the template

(MustacheTemplate on: templateString) value: context


templateString asMustacheTemplate value: context

we get the following output

Hello Chris
You have just won $10000!
Well, $6000.0, after taxes.

Note: You’ll get a lot more newlines and whitespaces than the string shown here. It is not clear to me what the rules are for condensing whitespace. The will be changed in an upcoming version of mustache

As context object we can use Dictionaries and Objects. Dictionaries need to have a key that is used in the template and Objects need a selector with the same name. In this post I use Dictionaries because they are easier to illustrate with.

Working with lists

We can use collections to make loop constructs in templates

templateString := 'A list of numbers
{{# list }}
Number: {{ number }}
{{/ list }}'.

A context object with content

   'label' -> 'fine.
   'list' -> { 
      { 'number' -> 1 } asDictionary.
      { 'number' -> 2 } asDictionary.
} asDictionary

gives us the output

'A list of numbers
Number: 1
Number: 2

And Blocks as well

We can use blocks in context objects. They will be evaluated at the time the template is filled out.

'The alphabet: {{ alphabet }}' asMustacheTemplate 
   value: { 'alphabet' -> [ Character alphabet ] } asDictionary


The alphabet: abcdefghijklmnopqrstuvwxyz

Partial templates

Mustache templates have a notion of sub templates that are called partials. With partials we can nested templates this way

templateString := '<h2>Names</h2>
{{# names }}
   {{> user }}
{{/ names }}'.

userTemplateString := '<strong>{{name}}</strong>'.

templateString asMustacheTemplate
   value: {
      'names' {
         { 'name' -> 'Username' } asDictionary } } asDictionary
   partials: {'user' ->  userTemplateString} asDictionary

prints the output


The dictionary given as partials: argument is supposed to be a dictionary that contains MustacheTemplates itself. A dictionary of strings will do as well because the strings are converted internally.

how about json?

Json is really easy to apply to the templates. If you have a pharo2.0 image just click outside a window. From the upcoming menu select tools and then “Configuration Browser”. Scroll down to NeoJSON and click the install button. After that it is just

'I can use {{name}} easily with {{format}}' asMustacheTemplate 
   value: (NeoJSONReader fromString: '{ "name" : "mustache", "format" : "json" }')

Copy that to pharo workspace and execute to see the result.

templates made easy

Mustache can make template dependent tasks very easy from a simple token replacement up to nested structures to create HTML pages. I use them e.g. for generating SOAP templates. The purpose of this post is to show the basic usage of it. The strength of Mustache lays in the syntax and the combination of context objects. So, there is more for you to find what can be done with it. If what you find is a bug I like to know. Happy templating !

Recover Corrupted Mac OS Software Updates

Somehow I’ve managed to break my Mac OS software updates / iTunes. I cannot remember doing anything special than updating software when I got a notification from my operating system. I think there were 5 updates and I just activated updating them all. After the third package had been updated an error occurred.

In such case I suspect some network problems and try again. But it didn’t work. So my suspicion readjusted itself towards caching. Software often includes retry timeouts to avoid wasting too much resources in the unusual case. So I waited for a day and retried then. Nothing changed, still broken! Well, I thought if the system is updating its software catalog it could fix the entry if it is bogus. This story could go on for some more time. As a software developer I’m experienced with all sorts of problems that can occur in those situations. But even waiting for a whole week didn’t help. The packages didn’t want to update.

A short research showed it is possible that the software registry of installed packages can become corrupted. One way to fix this is to reset software registry.

Open a terminal and execute the following line

defaults write ShowDebugMenu -bool true

Open software updates application and you should see an additional menu entry Debug on the right. In the Debug menu you’ll find an entry called Reset application. Restarting the sofwtare updates application should reload its data and the problem should be fixed.

I’ve found this information on this thread

Changing Blog Engine From Pier to Octopress

Several years I’ve been using the pier content management system for my web site. I loved pier for being an excellent example on how content can be managed. It lowers the border between static content and dynamic components by breaking both down into structure. Components can access content that is stored as page structure as well as dynamic content parts can access the whole structure of a site.

On the other hand pier is a rather complex piece of software that is hard to digest. Maybe this is the reason why only few people became familiar with the code. Having a decent user base is essential for software to survive. I’ve spent a lot of time with the code but I still don’t have the feeling I understand it.

From time to time I remember that reducing complexity in software and deployments is something important. I only write blog posts occasionally and for this using a complex content management system is overkill.

By occasion I got notice of the octopress blogging framework. It is written in ruby and produces static files. This is quite the contrary to what I had before. I’ve always been a big fan to make a web site out of static files. No dynamic components that can break, no performance issues. It is like in the good old times where I managed a news paper web site that could service 1 Mio. page impressions on a AMD K6 CPU with 128MB RAM.

By switching the blog engine I got a nice modern HTML blog at the same time. Octopress needs to be operated from the console in order to create blog posts. I’m quite familiar with working in a terminal so this is not a problem. Octopress is meant that way. It is called a blog engine for hackers. I’m curious how long it will take until I need to dive into it to change something and learn some ruby on the way.

The downside of having static files is the difficulty to have dynamic things included. Most blogs allow to comment blog posts. That wouldn’t be easy to do with static files. Octopress has support for using disqus as an external commenting engine. I might try this but I’m not sure if a blog needs comments. We’ll see!

How to Activate Double Tap to Drag in Lion

This week I got the new MacBook Pro with Retina display. The display is outstanding and makes the internet look a little like EGA graphics :) To switch from one MacBook to another it is always a decision how to recreate your environment. This time I decided to do a clean migrate. It means no time machine, no big backup/restore procedures that are built-in. The goal is to have as less cruft as possible on the new computer.

By doing this you also loose all the tweaks you’ve added in the past. For most of them I have a note or a blog entry like this. But there are a lot things that went unnoticed. I updated my old MacBook two times to a new system and I always had a behavior that you can double tap and drag a window using the trackpad. On the new MacBook the behavior was gone. Apple replaced it with yet another revolutionary something. Gladly there is a way to restore the behavior although it is kind of hidden which should tell you that apple want you to stop using it.You can find it in

System Preferences  
   > Universal Access  
   > Mouse and Trackpad  
   > Trackpad Options  
   and there enable the "Dragging" checkbox.

If you have a german language setting like me you’ll find it in

   > Maus & Trackpad 
   > Trackpad-Optionen...  
   and the checkbox is called "Bewegen"

I got this helpful trick from cnet

Vortrag über Javascript Funktionen

Dies ist ein Nachtrag zu meinem Vortrag vom 7. Juli 2012 auf der cologne.js. Ich hatte vergessen, es vorher anzukündigen. Ich war recht zufrieden mit dem Vortrag. Es gab genügend Fragen und kleine Diskussionen, die meiner Meinung nach das wichtigste an solch einem Vortrag sind. Sobald das Thema javascript ist, finde ich es vorteilhaft, wenn man Code einerseits visuell präsentieren, andererseits den Code auch ausführen kann. Dafür habe ich ein gutes toolkit gefunden, das meine html5slides ablöst. Ich benutze nun deck.js und das code mirror plugin. Mit deck.js sind slides sehr einfach zu erstellen und mit dem code mirror plugin bekommt man eine textarea, die den Code inklusive syntax highlighting darstellt. In die textarea ist noch ein run button integriert, mit dem man den code ausführen kann. Man kann den Code in der textarea einfach verändern und dann nochmals ausführen. Das plugin leitet ebenfalls alle console.log Aufrufe in einen Bereich unterhalb der textarea um. Damit lässt es sich sehr schön präsentieren und diskutieren.Die slides vom Vortrag sind hier

Simple Patches in Homebrew

I just learned how to apply a simple patch to piece of software that is being build by homebrew. I write it down not to forget. A simple patch is easily applicapable in brew. You just need to do the following: Open a terminal and typebrew edit myformulaAn editor opens containing the text of the package myformula Add a method “patches” like this def patches DATA endIn the patches method you can do all kind of things with the source. The method is called automatically by homebrew. If this is a simple patch and we want to included right in the formula the DATA will do the trick.

Add an end label to the formula as the last line of the forumla__END__This indicated the end of the formula

Paste your patch after the __END__ label

Installing the software by doing

brew install myformulait 

should patch the source before building it

LRUDictionary: A Small Least-recently-used Style Dictionary

For my last project I needed some kind of cache. I had one web service providing the main functionality and that was using another service for session and user information. Having each web request triggering multiple requests internally to resolve user and session data was not an option. In pharo there is a class called LRUCache. It is like a dictionary where the only implemented method is #at: .

The LRUCache is created by providing a size of the dictionary and a factory block. When an element is requested by using at: the factory block will be invoked if the element is not present. LRUCache does not give you control over the storage of the elements. Either it is there or it will be created and exists then. For my user and session data the situation was slightly more complicated. I don’t want to store user information when the user is not confirmed (it can change within the next seconds if the user confirms). And sessions should be removed from the dictionary if they are expired. The removal of expired sessions is not necessary as they would be sorted out automatically. It is an optimization because the cache will be less efficent if expired session block upper position in the LRU order until the reach the tail of the list and get discarded.

To serve my purpose I created a LRUDictionary class. It works like the LRUCache class (yes, I preserved the nice statistics the LRUCache provides about its usage) but adds more of the dictionary protocol to it. I added


The code can be found on smalltalkhub.Feel free to improve, comment or criticize

Smalltalk Dev Room @ FOSDEM 2012

At the weekend 4th/5th of february there will be this years FOSDEM, a big free and open source conference held in Brussels. Some of the smalltalk guys arranged a smalltalk dev room on sunday, 5th of february. If you are interested in smalltalk don’t hesitate to visit us in the smalltalk dev room (room number seems to be AW1.126). If you have interests in smalltalk and REST then you are even more welcomed to visit my talk. It is the first in the schedule (I should have been more careful :) ).Here is the schedule:

09:30 Norbert Hartl, Take a small REST. Simple approaches for REST in smalltalk
10:00 Stephane Ducasse, Marcus Denker, The next steps for the Pharo Vision
11:00 Laurent Laffont, John Thornton, Amber, the Smalltalk for web developers
12:00 Nick Ager, An introduction to jQuery Mobile
12:30 Stefan Marr, RoarVM, Sly
13:00 David Chisnall, Compiling Smalltalk to fast native Code
13:30 Craig Latta, Spoon
14:00 Stephan Eggermont, Willem van den Ende, Diego Lont, Back to the future,(re)learn smalltalk (till 16:30).

See you @ FOSDEM

Smalltalk Inspect Interview über GemStone

In der aktuellen Ausgabe von Smalltalk Inspect gibt es ein Interview mit mir zum Thema GemStone (Die Ankündigung findet sich hier). Ich denke, die Diskussion gibt einen guten Überblick darüber, was GemStone ist und was man damit machen kann. Viele der angesprochenen Dinge handeln vom Erstkontakt mit der Materie, der gerade für Leute ohne Vorkenntnisse interessant sein dürfte. Aber auch Details über die Funktionsweise der Persistenz und dem Umgang mit GemStone in einer grösseren Installation fehlen nicht. Ich hoffe, der podcast ist für viele interessant. Dies ist in Anbetracht der “leichten” Überlänge :) nicht selbstverständlich. Kritik und Anregungen gerne direkt an mich per Email.

Smalltalk Inspect ist das erste deutschsprachige podcast zum Thema Smalltalk. Es wird betrieben von Marten Feldtmann, Sebastian Heidbrink und Joachim Tuchel. Das podcast widmet sich in jeder Ausgabe einem smalltalk-spezifischen Thema. Und für jedes Thema lädt sich die Smalltalk Inspect Crew einen Verteter/Experten für das jeweilige Thema ein.