Online Village

Install Graphite On A SmartMachine (SmartOS Zone)

2012-01-05T10:45:00.001-08:00

I've been hearing a lot of good things about Graphite lately. So I thought I'd give it a try on a SmartOS Zone (SmartMachine?). I had one heck of a time installing it. Here's how I did it.

First, for some reason I had to use the 32bit version SmartOS 1.4.7. If you try the 64bit version, the first error you get:


Checking for header Python.h             : Could not find the python development headers

But this appears to be a red herring and the real issue is related to 32 vs 64bit issues. Investigating this helped me understand why people don't like waf. Just use the 32bit zone and safe some pain.

Install all the dependencies that are available via pkgin:

pkgin install apache gcc-compiler scmgit python27 pkg-config cairo xproto renderproto-0.11 kbproto py27-setuptools py27-django py27-twisted ap22-py27-wsgi

I ran into a problem where it said apache was missing when I tried to download it. This was fixed by doing 'pkgin -f update'. The '-f' was necessary.

Dependency - pycairo

Install pycairo, we need py2cairo as its for python 2.7:


git clone git://git.cairographics.org/git/py2cairo
cd py2cairo
PREFIX="/opt/local" ./waf configure
./waf build 
./waf install

Install Graphite components 'by hand'. Download, untar and run 'python setup.py install' in each of the resulting directories:

http://pypi.python.org/pypi/carbon
http://pypi.python.org/pypi/graphite-web
http://pypi.python.org/pypi/whisper

Install django-tagging by hand. Download, untar and run 'python setup.py install'.
http://code.google.com/p/django-tagging/downloads/list

Inside graphite-web run:


./check-dependencies.py

You might see warning, but there shouldn't see any FATAL messages.

(Under Construction) You still need to:
- copy and edit the apache config
- copy the .wsgi example into place
- copy the carbon.conf.example and the storage-schemas.conf.example
- patch the settings.py file so that it doesn't prepend the third party path

How Node.js Multiprocess Load Balancing Works

2011-11-05T19:10:00.000-07:00

As of version 0.6.0 of node, load multiple process load balancing is available for node. The concept of forking and child processes isn't new to me. Yet, it wasn't obvious to me how this was implemented at first. It's quite easy to use however:

var cluster = require('cluster');
var http = require('http');
var numCPUs = require('os').cpus().length;

if (cluster.isMaster) {
  // Fork workers.
  for (var i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  cluster.on('death', function(worker) {
    console.log('worker ' + worker.pid + ' died');
  });
} else {
  // Worker processes have a http server.
  http.Server(function(req, res) {
    res.writeHead(200);
    res.end("hello world\n");
  }).listen(8000);
}

This is quite a beautiful api, but it hides some clever details. For instance, how is possible for all child processes to each open up port 8000 and start listening?

The answer is that the cluster module is just a wrapper around child_process.fork and net.Server.listen is aware of the cluster module. Specifically, cluster.fork uses child_process.fork to create child processes with special variable set in their environments to indicate cluster was used. Specifically process.env.NODE_WORKER_ID is non-zero for such children.

 envCopy['NODE_WORKER_ID'] = id;
 var worker = fork(workerFilename, workerArgs, { env: envCopy });

Then net.Server.listen checks to see if process.env.NODE_WORKER_ID is set. If so, then the current process is a child created cluster. Instead of trying to start accepting connections on this port, a file handle is requested from the parent process:

//inside net.js
require('cluster')._getServer(address, port, addressType, function(handle) {
      self._handle = handle;
      self._listen2(address, port, addressType);
    });

//inside cluster.js
cluster._getServer = function(address, port, addressType, cb) {
  assert(cluster.isWorker);

  queryMaster({
    cmd: "queryServer",
    address: address,
    port: port,
    addressType: addressType
  }, function(msg, handle) {
    cb(handle);
  });
};

Finally, this handle is listened on instead of creating a new handle. At which point you have another process that is listening on the same port. Quite clever, though I think the beauty of the api comes at the cost of some not so hideous hacks inside the api.

Why KVM on Illumos matters - ZFS

2011-10-23T11:32:00.000-07:00

When Joyent first announced they had ported KVM to Illumos, the value proposition didn't occur to me right away. Both KVM and Xen under Linux have been around for while and you if want/need to run a Linux guest, why not use a linux Host? One reason is a feature of ZFS known as a zvol.

To understand what a zvol is, you must first understand what a zpool is. In short its an abstraction on top of one or many disks turning it all into a logical unit on which filesystems can be created. Multiple disks can be used for both redunancy and performance. In particular, you can create a zpool that uses a pair of super fast state of the art SSDs for the ZIL, or intent log. This makes random writes blazingly fast. You can add a couple of fast SSDs as a read caching layer (l2arc). Finally, use those boring old -but relatively high capacity- magnetic disks for raw storage. The result is a hybrid storage system that provides performance and redundancy at a terrific price.

The zpool is where the zfs filesystems live. So once you create a pool you can create a bunch of different file systems in there. You can also create and export block devices from a zpool, these are called zvols. These block devices can now be passed to qemu running on top of kvm inside Illumos, providing the underlying VM's with potentially blazingly fast redundant storage. A Linux VM will use this block device as if were a raw disk.

More generally, by using a zvol for a guest disk you can use all the ZFS features that are below the zfs filesystem layer. For instance it's possible to snapshot a live/attached zvol then export that snapshot to a file which can be used to create backups and clones. The implications for cloud/cluster management are clear.

Here's some other posts on zvols I found interesting:

http://www.markround.com/archives/37-ZFS-as-a-volume-manager.html

http://blogs.oracle.com/scottdickson/entry/fun_with_zvols_-_ufs

OpenSource Drobo replacement - Update

2011-10-20T12:06:00.000-07:00

After over two years of faithful service, I've upgraded my OpenSource Drobo replacement. I thought it fitting to write an update post as my new setup is even better though mostly the same.

For perspective, not long after I setup this system, Apple discontinued work on the ZFS port to OS X. Thankfully the work they had done was solid. During the last 2 years I lost 2 drives and was easily able to replace them and get back to full speed within 24 hours of purchasing a new disk.

My new system consists of the same Mediasonic 4 bay ProBox headed by a Zotac ATOM based NetTop I purchased for around $300. You can buy a similar model now without ram or disk for under $150. I'd go for this one if I could go back in time. You can use a cheap USB stick and two gigs of ram for under $100 these days.

For the new head operating system I tried a couple of different Open Solaris/Illumos based distrobutions such as Smart OS, Open Indiana and Nextenta Core Platform. Unfortunately but unsurprisingly, these didn't seem to like my cheap hardware. I finally stumbled upon FreeNas, a FreeBSD based NAS appliance distribution complete with Web UI . Because it's FreeBSD based, support for ZFS is included. It's also designed to run from a USB stick in order to free up space for another drive which is a great idea.

Migrating to FreeNAS was a breeze once I learned to trust the Web UI. I still had to manually chown and chmod everything from a shell. Chown fixed the ownership for sharing everything properly in the FreeNas way. Chmod was required to fix an ACL problem with the filesystem caused my the OS X port. I also elected to upgrade to version 15 of ZFS which took all of two seconds (literally). It's possible I could have easily accomplished all this in the FreeNAS UI but old habits die hard.

Setuping up sharing was as easy as following the directions in the documentation. In short, create user, add user to group, configure the volume to be owned by that group with appropriate perms, create share, start/restart CIFS service. All in all it took about an hour!

Here's an update to the caveat's section from first post:

- you don't really need to know how to use a command line to get up and running with FreeNAS unless you're migrating coming from OS X ZFS
-esata does improve performance, at least it seems to though I've done no rigorous tests
-reboot the system, everything will come back online
- the green drive problem doesn't seem to be an issue anymore

Mobile HTML5/JavaScript Apps Part II

2011-10-17T16:04:00.000-07:00

Challenges Throttling Adoption

In my previous article I outlined 10 reasons why the HTML5/JavaScript platform was the future of mobile app development. Adopting HTML5/JavaScript as a mobile development platform isn’t without it’s challenges however. In this article I outline the problems that are throttling the adoption of this platform.

The App Store Model

Presently there is no standard format for bundling HTML5/JavaScript apps for inclusion into app stores. This is disadvantageous for a number of reasons. Foremost, app stores help your customer’s find your app. They provide a platform integrated place where people can rank and recommend your app. It can help you grow your business.

Secondly, an app store also provides an integrated payment processing solution, often through cell phone billing. Sure they all take their cut and you could go the ad-supported route instead. But having a choice of models is one advantage the native app developers have. The seamless experience of well designed apps store is important.

Offline Support

Your customer’s expect to keep working when they enter the subway or fly on airplane. Native applications are bundled in to discrete packages and so capability typically comes for free. Traditional web applications host application logic remotely. As such, logic is transferred across the internet each time the app gets started or it even worse it remains on the server entirely.

In either case your customer is at the mercy of their mobile network which is bad for two reasons. The first is that the user experience will probably be slow or unusable, at least occasionally. The second is this architectural model causes a lot more data to be transferred. Mobile bandwidth isn’t cheap now that carriers are moving to plans with quotas.

HTML5 does provide a offline web app solution in the form of offline manifests. Anyone who’s ever tried to use it knows how rough around the edges it still is. Eventually they’ll smooth out those edges, but the in the meantime it’s painful to use.

The Solutions

In my next post I’ll describe the new technologies that are emerging to solve these problems. A perfect storm is converging to make HTML5/JavaScript the dominant platform for mobile applications. The future is bright and open source software is leading the way.

Mobile HTML5/JavaScript Apps Part I

2011-10-17T16:01:00.000-07:00

I'm convinced that HTML5+JavaScript is the future of mobile development. That’s not to say there aren’t challenges throttling the adoption of this technology in the mobile space. But suffice it to say we feel the advantages of JavaScript are so compelling that critical mass has been achieved.

In my next article I’ll highlight the challenges facing mobile HTML5/JavaScript adoption. But first, here are the top 10 reasons why I believes HTML5+JavaScript is the future of mobile app development:

1) iOS (Apple)

2) Android ( Google )

3) BlackBerryOS ( RIM )

4) Windows Phone 7 ( Microsoft )

5) WebOS ( HP formerly Palm )

The first three mobile platforms each have close to a third of the market share. Yet their ‘native’ application platforms are all different. All of these devices have modern HTML5 browsers.

As a Mobile App Developer, you want to target as many different platforms and customers as you possibly can. At the same time, your app needs to look GOOD and it needs to be consistent across platforms. Finding the talent to target all these native platforms would be expensive.

6) JavaScript Is The Most Widely Available Development Platform, Ever.

There are over 2,000,000,000 internet users in the world now. Each uses a web browser, and that browser runs JavaScript. Eventually HTML5/JavaScript will work on every phone, laptop and desktop in the world.

7) You Can Still Sell HTML5/JavaScript Apps Through Regular App Stores

Great things are happening at fellow Canadian startup Nitobi. They’ve created an open source HTML5 app platform that allows you to author native applications with web technologies and get access to APIs and app stores. With PhoneGap it’s possible to write your application in HTML5/JavaScript, package it in a native wrapper, then distribute it through traditional app store channels.

8) JavaScript Is F A S T, Fast.

Google, Apple, Microsoft and Mozilla are all throwing cash and talent at making JavaScript faster. Techniques like JIT Compilation and Inline Caching are raising the bar on JavaScript performance. JavaScript is already fast and it will only get faster.

9) There Are *A Lot* Of JavaScript Developers Out There And The Number Is Growing

Chances are if you do Web development in PHP, Ruby, Python, Perl, Java etc, you’ve also implemented client side logic in JavaScript. That is, almost every web developer out there has at least dabbled in JavaScript. As the Web as a platform continues to gain traction, so does JavaScript. There’s a large pool of talent waiting to code in this language.

10) Windows 8

Even Microsoft is committed to supporting HTML/JavaScript as a core development technology. In a recent press release they describe Windows 8 as a re-imagining of the operating system. Here’s the part that stands out:

“Windows 8 apps use the power of HTML5, tapping into the native capabilities of Windows using standard JavaScript and HTML to deliver new kinds of experiences. These new Windows 8 apps are full-screen and touch-optimized, and they easily integrate with the capabilities of the new Windows user interface.”

No where in the press release do they mention .NET or Silverlight, technologies that Microsoft developed which compete directly with HTML5. So it seems that Microsoft is acknowledging that HTML5/JavaScript is platform to back if they’re going to retain developer mind-share.

A Fragmented Future United By Web Technologies

2011-10-17T15:58:00.000-07:00

This post is in reponse to Peter Bright’s article on arstechnica.

Microsoft shocked many of it’s developer community when at D9 it announced HTML5/JavaScript would form the basis of Window’s 8 developer platform. Peter argues that saying nothing about the future of these technologies can only hurt Microsoft. However, I’m not sure Microsoft has much to lose and quite a bit to gain by spinning it this way. Here’s why.

Microsoft may rule the desktop now, but the future is mobile and they know it. Windows Phone 7 may be great and it’s apps may be excellent but they’re playing some serious catch up to iOS, Android and even Blackberry. Remember, Microsoft makes more off Android royalties than it does on Windows Phone 7. So while it’s true that Microsoft risks alienating their developer community, at present course that community will become smaller and more marginalized as time goes on anyways.

The truth is that if Microsoft wants be more than niche player in the mobile/tablet market, it needs to embrace HTML5. It’s just to far behind to expect to act as if it were Apple and try to force a proprietary platform. You can look towards the Financial Times launch of an HTML5 ‘iPhone’ app as model for the future here. Developing multiple high quality native applications is expensive and becoming increasingly unneccessary.

Now microsoft hasn’t said they won’t support their traditional platforms. No, far from it. Chances are there’ll be support for Silverlight as well. So while windows developers might be afraid right now, they’ve invested too much hold a serious grudge. Once Microsoft finally placates their fears they’ll suck it up and leverage the skills they’ve invested so much in.

By focusing the spotlight on their ‘choice’ of the only truly open cross-device app platform, Microsoft can been seen to be taking Windows in a bold new direction. Perhaps even buying themselves some creditability from the much larger web development community, many of whom still hold a grudge over IE 6. Microsoft has much to gain by spinning this decision as a bold and re-invention rather than reactionary concession. If that means temporarily upsetting their development community. They’ll get over it.

Is JavaScript Faster Than C?

2011-03-16T05:10:00.000-07:00

There's been a lot great work happening in the VM performance space over the last few years. The problems of performance are beginning to be well understood as even dynamic languages begin to challenge the incumbents. This article reviews a project which aims to bring more empirical testing to the language/runtime performance debate. Rather than argue about the theorectical, we let the code speak for itself. You may be suprised by what it says.

Stop!

The author of this article is well aware of its provocative nature. This is just one test and readers should take care when extrapolating these results to dissimilar use cases. The C code provided by the project being reviewed follows an Object Oriented style. It is certainly possible to optimize the C code further, for instance by using a more procedural style with less memory allocation. But then a similar approach should be used by the other versions (languages) presented as well. I whole heartedly encourge you to submit pull requests with these new versions in separate directories based on the style/technique used.

My intention is to challenge readers to question their assumptions regarding language/runtime performance. I also hope to highlight the fact that server side JavaScript has become a viable platform choice when performance is a consideration. That said, this is one test and I can only encourage more work in this area. If you're interested in such things, check out The Computer Language Benchmarks Game. Let's discuss results rather than argue fanatically about predictions.

The Problem

Solving 137846 games of the 15-Hole Peg Soltaire by calculating every possible move using an Object Oriented style. Solutions are provided in C, C#, Java, JavaScript, PHP, Python and Ruby (more submissions encouraged!). Most of the code was written by Jonathan Fuerth his presentation on Java/HotSpot performance. I've updated the JavaScript so that it uses prototypical method inheritance. You can find my 'fork' on github.

The Code

    git clone git://github.com/gflarity/peg-performance.git

Pre-requisites

This was all developed and tested under OS X however there shouldn't be any platform dependencies. You'll need a number of things for the test itself. I've put the version I'm using in brackets: gcc (4.2.1), Mono (2.10.1), Java (1.6.0_24), node.js (0.4.2), PHP (5.3.3), Python (2.7.1), Ruby (1.8.7).

Run the tests!

Note this will take a few minutes as some platforms are definitely faster than others.

    cd src/main
    make test

For each of the languages the test is run 10 times. I throw out the best and worst result then compute an average. A chart is created to show the results.

Results

Results can be viewed by opening .reports/report.html.

Here is the graph produced by this test on my MacBook Air:

Analysis

Here's what I find interesting about these results:

0) The JavaScript implementation is even faster if you load the browser version in Chrome 10. It clocks in around 2300ms on my mba.

1) HotSpot Java is F.A.S.T., FAST. I'd really like to see a C++ version for comparison in this test. But given the ecosystem out there and the fact that it's automatically cross platform, I'd have hard time justifying the use of C++ for many things these days... Well, maybe a kickass JavaScript engine ;)

2) I'd also like to see the results of this test if this problem was solved in a non object oriented style. That said, there are distinct advantages to using OOP.

3) Much respect to the Mono project. While I'm sure Microsoft's .NET runtime approaches the HotSpot VM in terms of performance, Mono is hanging in there.

4) JavaScript is a dynamically typed language. We must consider this when comparing its performance vs a statically typed Java. While some might argue that static typing is 'safer', performance is its only virtue in my opinion. (Edit: Static typing also helps IDEs with code complete and other useful features, this is also a virtue :) Typically I trade reduced performance for increased development efficiency every time I choose Perl, Python or Ruby. So despite its quirks, it appears that JavaScript can offer the best of both worlds in terms of performance and development efficiency. Not bad for the language that's in every browser and hence is the most widely deployed language/runtime in the world.

Understanding Inheritance In JavaScript

2011-02-26T12:41:00.000-08:00

In this article I will present the JavaScript inerhitence model in a concise, and I hope, easy to understand manner. The intended audience consists of developers with some previous experience with dynamically typed languages. It is my opinion that any difficulty associated with understanding the JavaScript inheritance model can be attributed to the difficulty teaching recursive definitions in general. It's sometimes hard to know where to start, when your definition is circular.

That said, let's boil JavaScript inheritance down into 5 key points in the following order:

1) In JavaScript, everything is an 'object' and all variables simply hold references to them. The other 'primitive' or basic types (of objects) are functions, booleans, strings, numbers and arrays.

2) Objects are collections of properties keyed by a reference. Most often, references to string objects are used as keys as they are immutable. This means that if foo and bar reference string types which contain the same data they are in fact the same object. This essentially makes Objects equivalent to hashes or dictionaries in other languages.

3) In JavaScript functions are special objects. Not only can they be called or applied, they have a special 'prototype' property which holds an object that can be accessed and modified. When a function is used as a constructor with the 'new' key word, the resulting object that is created will *dynamically* inherit from the object referenced by the constructor function's 'prototype' property.

Example:

  Constructor = function() { };
  Constructor.prototype.name = 'alpha';
  Constructor.prototype.say_name = function() { console.log("my name is: " + this.name) };
  object = new Constructor();
  object.say_name();

4) Upon initial definition, the prototype property of every new function can be thought of like this:

  func = function() {}
  func.prototype = { 'constructor': func };

That is, the value referenced by func.prototype is an object contains a reference to func itself. This recursive definition is a bit eye-crossing but actually very beautiful in that way that only recursion can be.

5) Here's where the recursive definitions start to get difficult to visualize. In JavaScript, every object (including functions) dynamically inherit from the special 'Object.prototype' object. In other words, 'Object' is THE constructor function of ALL objects in the JavaScript universe. The corollary being that Object.prototype.constructor == Object.

Try it:

  > Object.prototype.constructor == Object
  true

Implications

These 4 points form the foundation of inheritance in JavaScript. Let's make use what we've learned:

Corollary 1:

Object types, or 'classes' can be (and generally are) referred to by their contructor function. Examples: Object, Function, String, Boolean, Array etc.

Corollary 2:

If you want to add a method to every object in the javascript universe (even after it is constructed!), mess with Object.prototype.
Example:
> Object.prototype.foo = 'bar'
> func = function() {}
> func.foo
"bar"
> Object.prototype.foo = 'baz'
> func.foo
"baz"

Corollary 3:

There's no such thing as multiple inheritance in JavaScript. That is class 'Foo' can not inherit from both 'String' and 'Array' directly. Though, you can use some clever tricks to hack multiple inheritance into the language. For examples, see http://www.amirharel.com/2010/06/11/implementing-multiple-inheritance-in-javascript and http://code.google.com/p/joose-js/.

Ubunt+Node+cluster on Amazon EC2 Quickstart

2011-02-26T06:47:00.000-08:00

Just Show Me The Scripts

If you're just looking for a script to setup nodejs under ubuntu quickly, check out this set of scripts.

Walk Through

First you'll need to Signup for AWS/EC2 and generate a key pair. Google it, there should be plenty of tutorials.

Next, create an instance using the AWS Management Console and log into it using your key pair. You can find the AMI's here. I picked a 32bit 10.04 LTS AMI in US-East.

Install node's perquisites (or those that aren't the default ubuntu LTS ami):

  sudo apt-get update
  sudo apt-get -y install build-essential libssl-dev

Now let's create a nodejs system user:

  sudo adduser --home /home/nodejs --shell /bin/bash --disabled-password --disabled-login --system nodejs

Now login as that user:

  sudo su nodejs

For the heavy lifting we use the excllent 'nvm' from creationix

  git clone git://github.com/creationix/nvm.git ~/.nvm
  . ~/.nvm/nvm.sh

Install nodejs:

  nvm sync
  time nvm install latest 
  nvm use latest

Setup the nodejs user's enviroment to have that latest nodejs in it's path

  echo -e -n "\n. ~/.nvm/nvm.sh\n" >> ~/.bashrc
  echo -e -n "\nnvm use latest\n" >> ~/.bashrc

Setup npm, this makes life much easier:

  mkdir ~/git
  cd ~/git
  git clone https://github.com/isaacs/npm.git
  cd npm/
  make install

Taking Full Advantage Multiple Cores/CPUs

Assuming you want to take full advantage ec2's multiple cores then install 'cluster' - https://github.com/LearnBoost/cluster

  npm install cluster

Setup hello world example:

   mkdir ~/git/sandbox
  cd ~/git/sandbox

Edit cluster.js to contain cluster example:

      var cluster = require('cluster')
        , http = require('http');

      var server = http.createServer(function(req, res){
        console.log('%s %s', req.method, req.url);
        var body = 'Hello World';
        res.writeHead(200, { 'Content-Length': body.length });
        res.end(body);
      });

      cluster(server)
        .use(cluster.logger('logs'))
        .use(cluster.stats())
        .use(cluster.pidfiles('pids'))
        .use(cluster.cli())
        .use(cluster.repl(8888))
        .listen(3000);

Test it!

  node cluster.js

Using the domain you ssh'd into earlier, browse to http://:3000, you should see hello world.

From the EC2 instance, login into the cluster admin interface:

  telnet 0 8888
  Type help()

Realtime Performance Visualizations using Node.js

2011-02-12T06:49:00.000-08:00

Preamble

This article outlines how to create a realtime heatmap of your syscall latency using HTML5, some great node modules, and DTrace. It was inspired by talk that Bryan Cantrill and Brendan Greg gave on Joyent's cool cloud analytics tools. While specific, the code provided could easily be adapted to provide a heatmap of any type of aggregation Dtrace is capable of providing.

System Requirements

First thing's first, you're going to need a system with DTrace. This likely means Solaris (or one of its decedents), OS X, or a BSD variant. There doesn't appear to be Dtrace available for Linux.

Security

Secondly, please be aware that at the time of writing the demo code contains a fairly substantial secruity vulnerabilty. Namely the d script is sent from the client with no authentication what so ever. If you bind to localhost this shouldn't be a big deal for a demo. Time permitting I intend to clean up the code.

Dependencies

For this tutorial you'll also need:

node - http://nodejs.org/#download (duh)
npm - https://github.com/isaacs/npm (makes installing modules a breeze)
node-libdtrace - https://github.com/bcantrill/node-libdtrace (provides dtrace functionality)
Socket.IO - 'npm install socket.io' (websockets made easy)

Server

Now we're ready to start writing our web server:

heat_tracer.js

Client

In order to display our heatmap, we're going to need some basic HTML with a canvas element:

heat_tracer.html

Finally the JavaScript client which translates the raw streaming data into pretty picture:

heat_tracer_client.js

Run It!

Run Heat Tacer with the following. Note, sudo is required by dtrace as it does kernal magic.

sudo node heat_tracer.js

If all goes well you should see a moving version of something like the image below.

Contribute

You can find the latest version of Heat Tracer here. It is my hope that this article will provide the ground work for a much more abitious performance analytics project. If you're interested in contributing please let me know.

Further Research

More information about Bryan and Brendan's demo can be found here
Socket.IO can be found here.

JavaScript is the future... wait wha?

2010-11-21T17:36:00.000-08:00

1) JavaScript engines have become ridiculously fast thanks to corporate investment from the likes of Apple, Google, Mozilla etc.

2) There are a lot of JavaScript developers out there, the number is growing.

3) Browser based JavaScript is the most widely available programming platform, ever.

3) JavaScript development is non-blocking (event driven), and closure intensive (Ruby and Perl fans rejoice).

4) Node.js uses Chrome's V8 to make non-blocking event based service development easy and performant. The community momentum is truly astounding.

Telus FAIL

2010-07-30T07:09:00.000-07:00

Update: Somewhere around the middle of November the Telus HTC Hero was updated to Android 2.1. The battery life has decreased but at least they released it before Android 2.3 is out.

I love the Android platform for a whole host of reasons, including the fact that it's so open. I accept that this sort of openness can lead ( has ) to fragmentation. This fragmentation means that when the core operating system is updated, it will inevitably take longer for the version specific to my phone, the Telus HTC Hero, to be released.

I've been waiting for months to receive the Android 2.1 (Eclair) release for my Telus HTC Hero. At this time, I still haven't received it. It was ported to the HTC Hero months ago, but has yet to be released for Telus. Meanwhile, Android development has moved forward and Android 2.2 (Froyo) has been released.

The HTC Hero will never see Froyo, and this pisses me off. But not as much as the fact that I've entered into a 3 year contract for a phone that has yet to receive a single software update. If you're going to lock user's in, you have a responsibility to keep the software on the phone reasonably updated. I understand hardware requirements might preclude certain releases, but that's the only excuse.

In any event, I've learned my lesson, my next phone will be an official dev phone, as I always want to run the latest and greatest. In the meantime I'm going to be rooting my phone and installing a third party rom. Here's the info:

http://b2basiks.com/?p=701

http://forum.xda-developers.com/showthread.php?t=581981

http://www.villainrom.co.uk/viewforum.php?f=95

Grails & App Engine is just not there yet

2010-06-13T08:46:00.000-07:00

Just a quick follow up to my previous post; I've unsuccessfully tried to get the 'Hello, World' application running on App Engine. The java script pop-up never works and I run into timeout's far too frequently once deployed. There also doesn't seem to be much in the way of support for this combination yet.

For those interested, there was a brief mention of the problem during a fireside chat at Google IO. Here's the wave which summarizes the Q/A. If you haven't yet, I highly recommend the Google IO videos.

Groovy, Grails and GWT (G3WT) Tutorial

2010-05-30T08:40:00.000-07:00

While HTML 5 holds tremendous promise, my days of fiddling with CSS are over. Fortunately GWT offers an alternative to the traditional Web Application paradigm; by cross compiling Java into Javascript you only need one language for both your server and your client. Back to the future.

Being a big believer in rapid prototyping, I'd much prefer having a Groovy -> Javascript compiler. Unfortunately Groovy isn't supported in the Client side GWT code since it's the .java files themselves that get cross compiled , not the bytecode. Still, it is possible to use Groovy ( Grails ) on the server side. using Grails and the GWT plugin. This provides the utility of Grails helper/scaffolding scripts and if you know Groovy it may save you time. Groovy's close coupling with Java also means that we need to not depart to far from the GWT paradigm either.

In order to learn Grails/GWT I decided to create a 'Hello World' application which uses RPC. Here's the step by step instructions I used to make it for those who are curious:

First make sure Groovy, Grails, and GWT are installed. Then we can create our Grails app:

>grails create-app g3wt

Now let's install the gwt plugin:

>cd g3wt

>grails install-plugin gwt

Create a GWT to provide the server side logic:

>grails create-gwt-module g3wt.Application

Create the client facing page which exposes the module we just created:

>grails create-gwt-page main/index.gsp g3wt.Application

It will (may?) prompt to ask if you want to create the main controller. Choose y for yes.

Now, lets git rid of the default grails page in favour of our new gwt page. You can do so by editing grails-app/conf/UrlMappings.groovy. Change "/"(view:"/index") to "/"(view:"/main/index") so that it picks up our newly created gwt page.

Now you're ready to try it. From a terminal the following to start up your development web server:
>grails run-app

Now start up GWT's "Development Mode" tool by running the following in another terminal:
>grails run-gwt-client

Click on the launch default browser button. Your new GWT should open and establish a connection with the Development Mode tool. At this point there's not really much here, so let's add a hello world rpc service.

First we create our Grails service:
> grails create-service Data

Edit the file grails-app/services/g3wt/DataService.groovy. Remove the empty serviceMethod method. Then add the following below "static transactional = true":


static expose = [ 'gwt:g3wt.client' ]

This line tells the grails helper script where to put the Java interfaces for the RPC calls.

Next replace serviceMethod with:


String helloWorld() {
return "Hello, World!"
}

Then we create the required normal and asynchronous interfaces under the src/java directory:
>grails generate-gwt-rpc

That's it for the server side. Now we edit the client scaffolding that was generated for us in

src/gwt/g3wt/client/Application.java.

Add the following import statements:


import com.google.gwt.core.client.GWT;
import com.google.gwt.user.client.rpc.ServiceDefTarget;
import com.google.gwt.user.client.rpc.AsyncCallback;
import com.google.gwt.user.client.Window;

Place the following in the onLoad method:


DataServiceAsync myService = (DataServiceAsync) GWT.create(DataService.class);

ServiceDefTarget endpoint = (ServiceDefTarget) myService;

// Note the URL where the RPC service is located!                                                                   
String moduleRelativeURL = GWT.getModuleBaseURL() + "rpc";
endpoint.setServiceEntryPoint(moduleRelativeURL);

// Call a method on the service!                                                                                    
myService.helloWorld( new AsyncCallback() {
  public void onSuccess(Object result) {
      // It's always safe to downcast to the known return type.                                               
      String resultString = ( String ) result;
      Window.alert( resultString );
  }

  public void onFailure(Throwable caught) {
  }

}
);

There's a lot of GWT magic here. My understanding is that the GWT looks at this entire line when it cross compiles and generates the right javascript object for our rpc calls. Then we can perform an rpc call using anonymous call that implements the AsyncCallback interface.

Run it again:

>grails run-app

Start up GWT's "Development Mode" tool by running the following in another terminal:
>grails run-gwt-client

You should see our "Hello, World!" alert box.

In the next installment I'm going demonstrate how to run our example under Google App Engine.

Groovy++ Sources

2010-05-16T19:06:00.000-07:00

It seems as though my cynicism was unfounded. You can find the Groovy++ source here.

My apologies to the author(s?). In my defence browsing the source tree through google code only yields the demo examples and last I checked the downloads only provided the examples as well.

Groovy++? Ok, so where's the beef?

2010-05-15T10:32:00.000-07:00

Update: You can find the sources at http://code.google.com/p/groovypptest/downloads/list

If you're a software professional who uses Groovy, you or your colleagues greatest concern about it is likely to be performance. Groovy++, while having a horrible name, provides a jar which apparently provides some mag-i-cal Groovy performance gains. According to its authors', this new 'Groovy-ness' will be released under the Apache Public License. Terrific! Wonderful! Hurray!

So where's the fracking code?

I can understand a need for control. I can understand the author's desire to turn this into something significant for themselves financially. A man's got to eat. But until the code is open and can be inspected by some experts, I remain skeptical of Groovy++. I am hopeful however that performance improvements can be made given enough manpower and talent. Too bad we all have day jobs ;)

Go adds JSON-RPC to core lib

2010-04-29T18:34:00.000-07:00

Recently I started looking for a small but useful project to serve as a vehicle for learning Go. After mulling on a few ideas I decided to create a JSON-RPC library. I generally appreciate the concept of RESTful webservices, but in my opinion this technology needs a simple common RPC protocol. JSON-RPC is exactly that. I find it particularly useful in protocols for controlling Daemon's remotely.

Well... I'm glad that I hadn't gotten very far in my implementation; as of 27/04/2010 JSON-RPC has been included in the core Go packages. Great minds thing a like I guess... :)

Playing with Go Memory Synchronization

2010-04-26T19:23:00.000-07:00

Go has its own model of process/threads/light-weight processes/coroutines, these are called goroutines. Channel communication is the main method of synchronization between goroutines. There are buffered and unbuffered channels. Both types of channels block on receive. Unbuffered channels block on send, buffered channels block on send once the buffer is full.

I've created some test code based on the sample provided in the documentation in case someone else want's to play poke black-box-style at the memory model. Channel 'events' cause memory synchronization to occur. But as the documentation describes you need to be careful with buffered channels, the synchronization occurs during receive not send. If you play around with it a bit you'll see the importance of using Locks, although you'd probably want to rely on the flexibility, safety and abstraction offered by Channels.

Enjoy.

Go Go Gadget-Concurrency

2010-04-03T18:40:00.000-07:00

Premature optimization may be the root of all evil, but timely optimization is often required. When optimization becomes necessary, coding in Groovy can make refactoring in Java painless. However, Java is a less than ideal "systems programming" language for a number reasons, the least not being that it wasn't designed to be such. Yet if you've ever spent significant time using a dynamically typed language such as Groovy or even a statically typed yet garbaged collected like Java, you'll find having to go back to C, or C++ to be onerous to say the least. .

Enter Go. Taken from the Go Language Design FAQ:
"Go is an attempt to combine the ease of programming of an interpreted, dynamically typed language with the efficiency and safety of a statically typed, compiled language. It also aims to be modern, with support for networked and multicore computing. Finally, it is intended to be fast: it should take at most a few seconds to build a large executable on a single computer. "

This sounds almost too good to be true, mainly because it's been a long time coming. There hasn't been much inovation- at least that which has received popular traction- in the systems programming space since C/C++. Indeed, Go looks much like C itself with some of the syntax reversed. Digging a little deeper there are number of features that a programmer used to dynamic typing would consider must have. Namely maps, foreach-like loops and minimizing the type declcaration necessary for variable instantiation.

At the same time, Go offers some impressive concurrency features: light-weight 'threads' or goroutines, segmented stacks and channels as first class objects. Go multiplexes independently executing functions. When a function blocks, such as by calling a blocking system call, the run-time automatically moves other coroutines on the same operating system thread to a different, runnable thread so they won't be blocked. Combined these features make high performance concurrent applications conceptually simple.

If you haven't already, take a look at Go.

Transcoding Videos Using VLC

2009-12-25T17:12:00.000-08:00

I found this great how-to for using VLC to converter/shrink/transcode videos. This is so you can shrink them to play on your ipod, iphone, Android device, whatever.

Its a bit slow as I think it actually plays the file normally as it saves the output file. Regardless, It was simple and I got it working on my Mac although the preferences menu entry is in a different spot (under the VLC menu entry ).

Easy Hex SHA-1/MD5 Hashes with Groovy

2009-12-22T14:49:00.000-08:00

The Java API has included support for a number of cryptographic hash functions ( MD5, SHA-1, etc) since Java 1.4 via the java.security.MessageDigest class. If you ever tried to use these in a web-centric context you've probably been a bit annoyed by the fact that it only provides hash results as byte arrays. Here's a quick and easy solution using Groovy that I thought I'd share:


def messageDigest = MessageDigest.getInstance("SHA1")
messageDigest.update( stringToHash.getBytes() );
def sha1Hex = new BigInteger(1, messageDigest.digest()).toString(16).padLeft( 40, '0' )

Notice the padding of zeros since the length of the SHA-1 hex needs to be 40 characters. If you're using a different hashing hex string length will change.

Note: Code updated to reflect an improvement submitted in the comments.

Production Groovy Part I: The trouble with Grape

2009-12-16T19:51:00.000-08:00

Like Perl, Groovy's utility is most evident while leveraging the work of others. In Groovy 1.7, Grape makes doing so intuitive with the addition of annotations on import statements:



@Grab( group = 'com.sun.jersey', module='jersey-core', version = '1.1.2-ea' )
import com.sun.jersey.api.core.DefaultResourceConfig
@Grab( group = 'com.sun.jersey', module='jersey-server', version = '1.1.2-ea' )
import com.sun.jersey.api.container.httpserver.HttpServerFactory

The trouble with Grape is that by default it resolves dependencies by contacting or attempting to contact certain Maven2 repositories. Now, this isn't a problem if you're a web startup looking to bring a product to market as quickly and cheaply as possible. But it is a problem if you're looking to start deploying Groovy to a secure production environment. Fortunately, Groovy's close coupling with Java provides a clean and elegant solution.

The key is that Maven2 remote repository caching and local repository hosting has been made easy by Artifactory. It can be configured to dynamically dial out and cache artifacts on demand from remote repositories. It can also more simply be used to manually manage artifact deployment. Thus provinding both flexibility and control.

In order to take advantage of Artifactory you'll want to modify the default Grape/Ivy configuration so it points to your internal Artifactory repository by default. The bad news is that you'll need to modify the an xml file and rebuild the the jar. The good news is that the groovy source itself uses Ivy so modifying the xml and building Groovy is really easy.

Simply download the latest Groovy source zip. Then modify '/src/main/groovy/grape/defaultGrapeConfig.xml'. You can then build yourselve a new zip distribution by running 'ant createJars -DskipTests=true'.
Skipping the tests is optional, of course.

Thats it. Once you explore Artifactory a bit you'll discover that it can provide as little or as much control of the artifacts available to Grape as you desire.

Fido iPhone Roaming Fraud

2009-12-11T06:45:00.001-08:00

Here's an interesting post which underscores why Canadians need more wireless broadband competition. Click on the lick to read about how Fido's data roaming screws over a customer.

Groovy Serialization - Part 2 - Objects

2009-12-02T18:19:00.001-08:00

In my previous post I discussed the serialization of simple Groovy data structures. Arbitrary objects can also be serialized using ConfigObject and ConfigSluper and a little trickery. The fact Groovy config files may also contain code makes this all possible.

In order to instantiate objects inside a Groovy config file, you need import the referenced objects. You also need to override the toString to create the proper instantiation string.


package MyPackage
class MyObject {
   def data;
   String toString() {
          def serializedString = "new MyObject( data : '${data}' )"
          return serializedString
   }   
}

//create the datastructure
def myObject = new MyObject( data: 'wtf')
def configObj = new ConfigObject()
configObj.testing = [1, 2, 3]
configObj.nested = [ objects : myObject ]

//serialize it
new File( 'newout.groovy' ).withWriter{ writer ->
  writer.write( "import ${myObject.getClass().getName()}\n" )
  configObj.writeTo( writer )
}

Parsing it is simple:


def config = new ConfigSlurper().parse(new File('newout.groovy').toURL())
println config

Finally one could use .metaClass to override the toString of any Java object and make it work with this serialization technique. Be careful not to shoot yourself in the foot by doing this, as other code might expect the regular toString().

Groovy Serialization

2009-12-01T17:55:00.000-08:00

Groovy's lists and maps provide the simple building blocks of more complex structures. Occasionally you want to serialize these complex structures to files/streams/whatever. Previously I wrote about using JSON with Groovy. While this is still my weapon of choice for this problem, another alternative would be to use the ConfigObject and the ConfigSlurper classes from the Groovy API.

Here's how you would serialize the data using this method:


//create the datastructure
def configObj = new ConfigObject()
configObj.testing = [1, 2, 3]
configObj.nested = [ objects : 'wtf' ]

//serialize it
new File( 'newout.groovy' ).withWriter{ writer ->
  configObj.writeTo( writer )
}

Then when you want to parse it:


def config = new ConfigSlurper().parse(new File('newout.groovy').toURL())
println config

One disadvantage of this method is that the highest level structure must be a map, since that's what ConfigObject inherits from. The data also contains a lot of white-space formatting. Which could be a good or bad depending on the problem you're solving.

Groovy 1.7-RC1 Released / Groovy 1.7 Before Christmas

2009-11-27T19:34:00.000-08:00

Groovy 1.7 RC1 has just been released and the final Groovy 1.7 is to be released by Christmas. By far my favourite new feature is the ability to add annotations on import statements. This allows you to put your @Grape() calls at the import statement that actually loads the packages grabbed which is where it should be IMHO.

While I'm still wrestling with the correct model for packaging 'grapes' in a production environment, as a veteran Perl coder I think grape will take Groovy to another level; It finally makes the Java ecosystem accessible and leveraging other's work easy. Now we just need a repository and IDE's that are grape aware :)

SQLite vs Derby / Derby SQLite Comparison

2009-11-20T21:38:00.000-08:00

I stumbled upon this wonderful comparison of Derby and SQLite. It even contains some details on SQLite's concurrency model that I had been looking for.

Groovy JSON / Using json-lib with Groovy / Using json_simple with Groovy

2009-11-17T16:25:00.000-08:00

After a bit of research I've concluded there are two clear options for using JSON with groovy.

1) Using json-lib . Note the annotation is Groovy 1.7 syntax and it rocks.


import net.sf.json.groovy.*
@Grab(group='net.sf.json-lib', module='json-lib', version='2.3', classifier='jdk15')

def books = builder.books {
book(title: "Groovy in Action", author: "Dierk Koenig")
}

assert books.toString() == '''{"books":{
"book":{"title":"Groovy in Action","author":"Dierk Koenig"}}}'''

2) Using the json_simple library .


import org.json.simple.JSONValue;
def response = [ : ];

response.error = [ message : "Not authorized" ];
def jsonString = JSONValue.toJSONString( response );
def deserializedResponse = JSONValue.parse( jsonString );

I prefer 2) but to be honest I haven't looked into Builders much. There's a simple analogy between Groovy datastructures and JSON. Why complicate it any further?

The Groovy Cookbook

2009-11-05T17:12:00.000-08:00

I just stumbled upon an incredible resource that I wish I knew about sooner. PLEAC, short for Progamming Language Example Alike Cookbook provides the solutions to the problems contained in the respected Perl Cookbook using OTHER langues, including Groovy. This is a terrific idea and if my favourite two languages weren't already complete I'd be eager to contribute.

Here's the Groovy table of Cotents. Learn how to deal with:

1. Strings (100.0%)
2. Numbers (100.0%)
3. Dates and Times (100.0%)
4. Arrays (100.0%)
5. Hashes (100.0%)
6. Pattern Matching (100.0%)
7. File Access (100.0%)
8. File Contents (100.0%)
9. Directories (100.0%)
10. Subroutines (100.0%)
11. References and Records (100.0%)
12. Packages, Libraries, and Modules (100.0%)
13. Classes, Objects, and Ties (100.0%)
14. Database Access (100.0%)
15. User Interfaces (100.0%)
16. Process Management and Communication (100.0%)
17. Sockets (100.0%)
18. Internet Services (100.0%)
19. CGI Programming (100.0%)
20. Web Automation (100.0%)

JAX-RS, Jersey and Groovy

2009-11-03T16:50:00.001-08:00

Groovy makes writing multi-threaded daemon a snap. While searching for a way to control such daemon simply, a co-worker suggested I use JAX-RS. More specifically, he recommended Jersey, sun's reference implementation of JAX-RS or JSR 311.

Jersey made it extremely easy to implement a RESTful web service inside my Groovy based daemon in order to control it while running. Groovy's dynamic class loading nature did case some issues with the resource file. In the end it was easy to overcome though.To get up an running quickly follow the getting started guide. You can create a groovy class which uses the Resource Class annotations they show. Then inside your main application you can spawn the webservice like so:


def resourceConfig = new DefaultResourceConfig( JSONRPCService.class  )
HttpServer myServer = HttpServerFactory.create("http://localhost:9090/", resourceConfig );
myServer.start();

Notice that we have to create resource config object and pass it to the HttpServerFactory. Thats the key, since under groovy the server factory can not seem to find the resource on its own.

That's it. Next post I will how how to create an JSON-RPC interface using Jersey and JSON-lib.

Groovy CLI Parameters Made Easy

2009-10-23T10:37:00.001-07:00

After getting thoroughly frustrated with the CliBuilder ( groovy.util.CliBuilder ), for its lack of documentation and unintuitive usage, I stumbled upon GOP (groovy-option-parser). Its almost exactly what I was looking for.GOP makes processing Groovy command line parameters (args) easy. Checkout the synopsis on its project page.

On a side note, Groovy really needs a CPAN.org equivalent. It has a fanastic runtime dependency loading/resolving system in Grape. It just need a single home/community for people to share modules just like this one.

The quest for a dark Netbeans theme

2009-10-21T17:39:00.000-07:00

Little things in Eclipse have been bugging me enough that I've given up the utility of support from colleagues in favour of advantages of using the Netbeans IDE. The last straw was the difficulty of using a dark colour theme.

Being a long time emacs user I was quite shocked by the lack of dark colour support in both Eclipse and Netbeans. Netbeans clearly has better support for this in the form of text editor colour themes. But what about all the other boxes and dialogues?

During your search for the allusive dark colour theme you'll notice that its the php and ruby people who've done the work to make these IDEs asthetically (and erogomically) pleasing. Some times I wonder about Java people...

I finally managed to get a decent dark themed setup for Netbeans working though:

1) Install the "Extra Ruby Color Themes" plugin via the Plugins->Available Plugins menu.
2) Goto Options->Fonts & Colors and pick the Aloha Theme.
3) Follow the instructions here http://www.mediabits.dk/php/netbeans-the-dark-side/. I think this actually alters things at the java/swing level and works fairly well.

Edit: I'm pleased to say that Netbeans 6.7+ looks terrific on OS X with Aloha. Its not as dark as I'd like but the bright white is gone.

OpenSource Drobo replacement on Mac OS X (Snow Leopard)

2009-09-10T17:57:00.000-07:00

Recently I lost a 320 USB hard drive. What was on it can eventually be replaced, but I've decided that its time to take my personal storage more seriously. I briefly considered getting a drobo. A friend has one and I'm fairly impressed with its abilities. The thing that bothers me the most about it is that its a completely closed system. I'm a bit of a open source evangelist, but I don't consider myself a zealot. However, If there's one part of a computer system that should be free and open, its the code the manages data storage. I can't compromise on this, fsck closed formats.

After a bit of research I settled on using a 4 drive JBOB usb enclosure in a zfs raidz pool. My server is a Mac a circa 2007 Mac Mini running Snow Leopard. The ZFS support is provided by this project. The project still needs some polish but I'm pleased to say it was easy to get my raidz array up and running. The performance is acceptable, I'm sure it would improve if I could use eSATA instead of USB. It's supported by my enclosure but not my mini sadly.

If you're looking to do something similar with zfs here are a few tips, gotchas & advice:

-if you're afraid of using the shell you should probably get a drobo, I'd say my setup was about as easy as formating and mounting any other file system under *nix. Just different and cooler
-esata should improve performance considerably, its already good enough as a media center drive IMHO.
- use raidz with whole drives, I achieved 2/3+ usable space with 4 drives, if one fails the system will continue to work until it can be replaced.
-whenever my mini reboots the drives do not automount and I get 4 nasty messages about not being able to initialize those 4 drives. I hit ignore and then in the shell I type 'zpool import -f '. I'll setup some init scripts for this eventually.
-watch out for the fact that the disk drive names may change since its usb
-watch out for 'GREEN' drives, mine power themselves down frequently and I've had to write a script to prevent this from happening.

RIP: A Remix Manifesto

2009-04-07T18:30:00.000-07:00

I highly recommend you check out RIP: A Remix Manifesto. In particular, there's a segment in which a former member of the Clinton Administration discusses policy regarding globalization and Intellectual Property. The policy was that that the US would open its markets to global competition, particularly manufactured good, in exchange for agreements on Intellectual Property. He discussed the motivation and vision in contrast to the current reality. Fascinating.

A TED A Day Keeps The Blues Away

2008-10-11T18:03:00.000-07:00

If you haven't visited TED yet, then go do it now. TED is yet another example of how the internet is starting to realize its potential as the most powerful educational technology to date. If you're like me, you've had some great lectures but a lot more average and poor ones.

TED allows us to imagine a world where every student can attend the seminars offered by the world's greatest lecturers. It allows us to imagine regular teachers in the class rooms focused on providing personal interaction and one on one tutoring. Of course, its hard to envision what the future of education is. But its clearly time to start dreaming and working towards those dreams.

The End Of Free-Market Fundamentalism

2008-10-11T12:23:00.000-07:00

The focus of this blog is relationship between technology and politics, culture, and education. However, it is difficult not to comment on the incredible paradigm shifts that we are bearing witness to right now. One of the biggest and most under discussed is the end of Free-Market Fundamentalism. It ended when the government began bailing out Fanny Mae and Freddy Mac, AIG, and now the passage of the TARP proposal of $700 Billion for Wallstreet. The American government might even start buying equity stakes in Banks.

When considered as whole there has never been a bigger act of socialism in American history. Tax payer money is being applied to the problem as if were a defibrillator. Will be paid back? How much health care could have been paid for with this money? While those greedy individuals responsible ever be held accountable?

Here's a great video that highlights talks by economists on the end of Free-Market Fundamentalism.

Open Courseware: The Future Of Education

2008-09-18T19:07:00.000-07:00

Standford University has released the Courseware, including lecture videos, for some of its computer science courses under the Creative Commons Attribution 3.0 license. MIT has undertaken a similar intiative, release materials under a similar license. Standford has fewer courses, they all appear to have videos from each lectures.

This is an amazing development. But then, when considered, the reasons not to post educational materials, such as video lectures, seem selfish. Educator time is best spent answering questions, rather than lecturing. Some Lecturers are also fortunate enough to be in a class of their own, and everyone should benefit from their gift.

On the whole, Information technology is starting aspire to its true potential and this should be celebrated. Corporations have been realizing this potential for some time, but now there is also a particularly remarkable transformation in the public space right now. The revolution in education is just getting started, but it will be humanity's greatest accomplishment to date.

Adversity Inspires Innovation

2008-09-13T13:58:00.000-07:00

I just listened to a fascinating interview with Simon Phipps on FLOSS weekly. He's the top open source advocate working at Sun Microsystems. You can check out the interview here. He covers a lot of ground in about hour, including how adversity inspires innovation, the future of open source software, and why sun created Java in the first place. He describes how, in terms of a technology company, creating free software moves the value farther down the chain and how it is still possible to profit greatly from that value.

I've always wondered why sun created Java. There never seemed to be much of a profitability model around it, at least as far as I could see. But Simon explained that the creation of Java was motivated by the fear of an Intel hardware monoculture. Which makes perfect sense, its bad for everyone, especially another chip maker like Sun. Its also pretty clear how open sourcing Java would only further this goal. Now I'm surprised it didn't happen sooner.

I wish the interview had been longer. One point that was quickly covered was his view that the creation of the internet was an accomplishment of even greater historical significance than the creation of the first printing press in Gutenberg. It has me thinking about that printing press and how significant the mass produced written word has been. Fascinating.

Bittorrent For Your Mother

2008-08-25T20:26:00.000-07:00

While searching for a suitable mirror for a the latest Insurgency release, I stumbled upon an innovative service called Bitlet. The service rolls a Bittorrent client and a torrent file into Java Applet. This removes the need to install a standalone Bittorrent client in order to download files.

To be honest, I'm a little suprised I haven't seen something like this a long time ago. I'm also suprised thatservices like fileplanet are even necessary anymore. But I'm glad that Bitlet is picking up some momentum. For the files I tried, it worked very well.

Groovy Is Groovy

2008-08-21T19:24:00.000-07:00

While I appreciate the need for strongly typed languages, I don't find them very enjoyable to program in. I prefer to prototype and refine as needed. With Java in particular this is difficult to do. But its hard to ignore all the rich libraries being written in/for java these days.

Groovy is an agile and dynamic language for the Java VM. A number of different colleagues have recommended Groovy to me and I've finally sat down and played with it. I'm very impressed to say the least. Besides being able to use java libraries easily from within a Groovy script, you can also compile Groovy code into regular java bytecode, or you can embed groovy directly into your Java application.

Personally, I'm looking forward to prototyping using Groovy in some open source projects I've been thinking of. Anything that is too slow can be refactored in Java. Perhaps by someone else ;)

Next-Generation P2P To Change The World

2008-08-21T17:16:00.000-07:00

Those who have already used P2P file sharing software regularly are probably already aware that technology is moving far faster than the business models of large content providers. While the Viacoms of the world are going after the YouTubes, they're completely unaware that the war has already been decided. Next generation P2P software promises to give websites with budgets of tens of dollars the ability to compete with YouTube, Viacom and every other broadcaster.

If you want to see the proof, then go visit the SwarmPlayer trial page. It is a project of the P2P-Next consortium, which recently recieved €19 million in government/corporate funding. While presently only a research proof of concept, SwarmPlayer supports both live broadcast streaming as well as video on demand. Once perfected this technology will radically democratize content distrobution.

If YouTube is giving Viacom a headache, then SwarmPlayer is the red laser dot hovering on its forehead. The trigger has yet to be pulled.