research

here's a screencast of some of the examples coming with the redUniverse quark.
some are just simple processes while others are a little bit more 'intelligent' (boids mainly).

redUniverseExamples from redFrik on Vimeo.

i made a short demo/poster session at the LAM conference 19dec in london. see livealgorithms.org
below is the handout describing the toolkit.

this toolkit is now distributed via supercollider's package system quarks. all open source.
how to install:
download supercollider
Quarks.checkoutAll
Quarks.install("redUniverse")
(if you run OSX and prefer to use SwingOSC over Cocoa gui, you'll need to move the file RedJWindow.sc to the osx folder and recompile.)

RedUniverse - a simple toolkit

Mark d'Inverno & Fredrik Olofsson

This is basically a set of tools for sonification and visualisation of dynamic systems. It lets us build and experiment with systems as they are running. With the help of these tools we can quickly try out ideas around simple audiovisual mappings, as well as code very complex agents with strange behaviours.

The toolkit consists of three basic things... Objects, Worlds and a Universe. Supporting these are additional classes for things like particle systems, genetic algorithms, plotting, audio analysis etc. but preferably many of these functions you will want to code your self as a user.

We have chosen to work in the programming language SuperCollider (www.audiosynth.com) as it provides
tight integration between realtime sound synthesis and graphics. It also allows for minimal classes that are easy to customise and extend. SuperCollider is also open for communication with other programs and it run cross-platform.

i also spent time at UoW learning about genetic algorithms and genetic programming. mainly from john h holland's books and karl sims' papers. i found it all very interesting and inspiring and again i got great help and input from rob saunders.

one of our ideas was to construct synthesis networks from parts of our agents' genomes i.e. to have the phenomes be actual synths that would synthesise sound in realtime. the first problem to tackle was a really hard one. how to translate the genome - in form of an array of floats - into a valid supercollider synth definition?
of course there are millions of ways to do this translation. i came up with the RedGAPhenome class which works with only binary operators, control and audio unit generators. unfortunately there can be no effects or modifier units. on the other hand the class is fairy flexible and it can deal with genomes of any length (>=4). one can customise which operators and generators to use and specify ranges for their arguments. you can also opt for topology of the synthesis network (more nested or more flat).
there is no randomness involved in the translation, so each gene should produce the exact same synthdef. of course generators involving noise, chaos and such might make the output sound slightly different each time but the synthesis network should be the same.

taking the intelligent street project further, mark d'inverno wanted me to try to get it online. the idea was to let people surf to a webpage, send commands to a running istreet system and hopefully collaborate with other online users to compose music. just like in the original sms-version of the piece, everybody could make changes to the same music and the result would be streamed back to all the online users.

the first prototype was easy to get up and running. we decided to use processing and write a java applet for the web user-interface and then stream the audio back with shoutcast. the users would listen to the music through itunes, winamp or some similar program. this of course introduced quite a delay before they could actually hear their changes to the music. but that was not all too bad as we had designed the original istreet in a way that latency was part of the user experience :-)
(the commands sent there via sms took approx 30 sec to reach us from the vodafone server and that could not be sped up. the users were given no instant control over the music - rather they could nudge it in some direction with their commands.)
so our internet radio/shoutcast solution worked just fine and we had it up and running for a short while from mark's house in london.

that was a total homebrewn solution of course. we wanted it to handle a lot of visitors, deal with some hopefully high traffic, be more stable, permanent and not run on an adsl connection.
so at UoW we got access to a os x server cluster and i started to plan how to install supercollider, a webserver and istreet on that. little did i know about servers, network and security and i had to learn ssh and emacs to get somewhere. rob saunders helped me a lot here.

mark d'inverno wanted to see the intelligent street installation gain new live online. so for streaming the sound from istreet over internet, i wrote a class for supercollider called ISRecord. it basically records sound to disk in small mp3 segments. so any sound supercollider produces will be spliced into many short mp3 files that could later be sent as small packages over the internet.
the technique is to continuously save the sound into one of two buffers. when one buffer is filled, the recording swaps and continues in the other. the buffer that just got filled is saved to disk and conversion to mp3 is started. this swap-and-write-to-disk cycle should have no problems keeping up realtime recording. but as the mp3 conversion takes a little bit of extra time - depending on quality and segmentation size etc, there is a callback message from the mp3 converter that evaluates a user defined segAction function when the mp3 conversion is finished. thereby one can notify other programs when the mp3 file is ready to be used.
there is also a cycle parameter that controls how many mp3 segments to save before starting to overwrite earlier ones. this is needed to not totally flood the harddrive with mp3s.

the actual mp3 conversion is done using lame and cnmat's sendOSC is also needed for the lame-to-sc communication.
attached is the recorder class plus a helpfile.

at UoW i also rewrote an old installation called the intelligent street. mark d'inverno and i had worked together on this one earlier and we now wanted it to run on a modern computer (os x). we also wanted to redesign it a bit and make it into a standalone application.

the original intelligent street was a transnational sound installation were users could compose music using mobile phones and sms. that in turn was an extended version of a yet older installation called 'the street' by john eacott. this totally reworked 'intelligent' version was premiered in nov'03 and realised as a joined effort between the ambigence group (j.eacott, m.d'inverno, h.lörstad, f.rougier, f.olofsson) and the sonic studio at the interactive institute in piteå (way up in northern sweden).

for the new os x version we dropped sms as the only user interface and also removed the direct video+sound links between uk-se that was part of the old setup. except for that the plan was to move it straight over to supercollider server (sc3) and rather spend time on polishing the overall sound and music.

idea
yet another system mark d'inverno and i worked on but never finished had the working title 'shadowplay'. we had this idea about an audiovisual installation where people's limbs (or outlines of bodies) would represent grid worlds. agents would live in these worlds and evolve differently depending on things like limb size, limb movement over time, limb shape and limb position. the agents would make different music/sounds depending on the world they live in. a limb world could be thought of as a musical part in a score. the worlds would sound simultaneously but panned to different speakers to help interaction.
the visitors would see the outline of their bodies projected on a big screen together with the agents represented visually in this picture as tiny dots. hopefully people could then hear the agents that got caught or breed inside their own limbs. we hoped to active a very direct feeling of caressing and breeding your own sounding agents.
there were plans for multi user interaction: if different limbs/outlines touched (e.g. users shaking hands), agents could migrate from one world to another. there they would inject new genes in the population, inflicting the sound, maybe die or take over totally. though to keep agents within the worlds they were made to bounce off outlines. but one could shake off agents by moving quickly or just leave the area. these 'lost' agents would then starve to death if not adopted by other users.

tech

i spent a lot of time benchmarking the code for the agents in the different versions of the multi-agent frameworks i posted about here earlier. best performance boost was (of course) when i ported some vital parts of the supercollider code to c. so for example one cpu-intensive task that the agents had to do a lot was to find out about their surroundings. and another 'heavy' task was to calculate the distance to other objects. each operation wasn't very demanding on its own, but when hundreds of agents would do this at the same time, we really needed the speed of the c primitives.

below is the c code i came up with. it replaces some of the computational heavy parts in the Surroundings and ALoaction supercollider classes. to try them out you'd need to download the supercollider source code from svn, add my code to the file PyrArrayPrimitives.cpp and then recompile the whole application. edit the file extArrayedCollection.sc in the A4.zip package posted here earlier to use the primitives.
one issue we then had was to distribute this 'hack'. supercollider doesn't have an api for adding extensions like this to the language (but there's a nice plugin architecture for the server). so i had to build dedicated sc applications including this speed hack.

int prArrayAsALocationIndex(struct VMGlobals *g, int numArgsPushed)
{
	PyrSlot *a, *b;
	PyrObject *obj;
	int size, i;
	double asIndex, w, worldSize;
	a = g->sp - 1;
	b = g->sp;
	worldSize = b->ui;
	obj = a->uo;
	size = obj->size;
	asIndex= 0;
	for (i=0; i<size; ++i) {
		getIndexedDouble(obj, i, &w);
		asIndex= asIndex+(pow(worldSize, i)*w);
	}
	SetFloat(a, asIndex);
	return errNone;
}
int prArrayAsALocationRoundedIndex(struct VMGlobals *g, int numArgsPushed)
{
	PyrSlot *a, *b;
	PyrObject *obj;
	int size, i;
	double asIndex, w, worldSize;
	a = g->sp - 1;
	b = g->sp;
	worldSize = b->ui;
	obj = a->uo;

after some time mark d'inverno and i shifted focus and decided to simplify our ideas. we agreed to work more from a bottom-up approach - letting the agents live within a grid world and visualise their behaviours. other people have been doing quite some work in this area before, but not particularly many of them have been incorporating sound and music. so we had literature and examples to study and people to ask. it was of great help and i learned a lot about designing multi-agent systems from analysing for example jon mccormack's nice eden.

so starting out writing our own system, i did a set of classes for handling agents running around in a grid world of 1-3 dimensions. all agents were very simple minded. they were visually represented with just dots and oh, they could bleep too.
setting up simple scenarios for these classes helped to pinpoint different system models. it also showed my biggest problems coding this usually boiled down to in which order to do things. the model i tried in turn to 'model' was suggested by rob saunders in an article called 'smash, bam and cell', in were all agents first sense their surroundings and then act. but i constantly had to restructure the code and design. this was harder than i had thought and i think i never came up with an all around solution.

another thing i played around with while at UoW was cellular automata. here's a simple one-dimensional CA class for supercollider... external link

and here is some more sc code i wrote to come to grips with CA...

/*cellular automata /redFrik*/
(
	var w, u, width= 400, height= 300, cellWidth= 1, cellHeight= 1;
	w= Window("ca - 1", Rect(128, 64, width, height), false);
	u= UserView(w, Rect(0, 0, width, height));
	u.background= Color.white;
	u.drawFunc= {
		var pat, dict, rule, ruleRand, y= 0;
 
		/*
		rule30= 30.asBinaryDigits;		// [0, 0, 0, 1, 1, 1, 1, 0];
		rule90= 90.asBinaryDigits;		// [0, 1, 0, 1, 1, 0, 1, 0];
		rule110= 110.asBinaryDigits;		// [0, 1, 1, 0, 1, 1, 1, 0];
		rule250= 250.asBinaryDigits;		// [1, 1, 1, 1, 1, 0, 1, 0];
		rule254= 254.asBinaryDigits;		// [1, 1, 1, 1, 1, 1, 1, 0];
		*/
		/*-- select rule here --*/
		//rule= 256.rand.postln;
		//rule= 90;
		rule= 30;
 
		pat= 0.dup((width/cellWidth).round);
		pat.put((pat.size/2).round, 1);
		dict= ();
		8.do{|i| dict.put(i.asBinaryDigits(3).join.asSymbol, rule.asBinaryDigits[7-i])};
 
		//--render
		Pen.fillColor= Color.black;
		while({y*cellHeight<height}, {
			pat.do{|c, x|
				if(c==1, {
					Pen.addRect(Rect(x*cellWidth, y*cellHeight, cellWidth, cellHeight));
				});
			};
			pat= [0]++pat.slide(3, 1).clump(3).collect{|c|
				dict.at(c.join.asSymbol);
			}++[0];
			y= y+1;
		});
		Pen.fill;
	};
	w.front;
)

more interesting than these simple examples are of course things like game-of-life.