January 2011 Archives

managing lights and HVAC

| No Comments | No TrackBacks
Marc Merlin controls his house lights and air conditioning with his computer, and he wanted to tell us about it. He wanted to teach his cat some manners. The cat knows it isn't allowed on the kitchen coutner, but also knows he won't get caught at night. So one way is to have a blender on your kitchen counter triggered by a motion sensor. But a motion sensor isn't targetted enough, it could trip at the wrong time. So another way is to use some motion detection from a web camera. 

Misterhouse was written about 10 years ago to allow for complex home automatino. It's based on perl. It supports almost all open protocols. Has a helpful mailing list, misterhouse-users@lists.sf.net. It can turn lights on based on a switch, and a motion sensor, with correct logic (disabling motion sensor time outs if you used the switch).

It supports X10, Insteon, Z-Wave, UPB, 1-wire. x10 has a few issues. X10RF is a radio frequency antenna way. But X10RF still has some problems. There is also X10sec, which I didn't listen to why it's different.

Looks like you use misterhouse by writing perl modules. He showed an example with X10 stuff. He then described Insteon, which is better than just X10. But there is no support in Australia or New Zealand.

1-wire is a bus topology network. You can run your wire runs for about 1000ft. The 1-wire interface is pretty cheap. There is also Oregon Scientific WMR200, which is cheaper wireless weather solution. Then he talked about XPL, a network protocol to broadcast sensor data. I tuned out, even though the talk was pretty interesting. Later on I noticed he was saying that he was able to graph the powe rusage of individual appliances, and correlate various temperatures to when things were switched on, so it looked pretty powerful. 

lightweight messaging

| No Comments | No TrackBacks
Andy Piper has been working in the UK for IBM over the last 10 years. He mentioned how the world is getting more interconnected. IBM's future of a smarter world is geting more interconnection, with lots of sensors, and 'intelligence'.

He showed a big slide of data centres, head office, remote offices, sensors, instruments, etc. 10 years ago someone at IBM came up with MQ Telemetry Transport, a protocol for sensors (very limited capabilities) to publish information.

This protocol minimises on the wire bits. Low bandwidth connections, and high latency connections that could be very expensive. The embedded devices may have very limited processing. It's also published royalty-free for ease of adoption by everyone else. The messages are asynchronous. It copes with loss of contact between client and server. Includes constructs in case the client goes away, such as 'hey server, tell everyone X if I go away'.

He compared MQTT to HTTP. MQTT is more efficient for small amounts of data. It's lightweight on CPU and network traffic. MQTT includes distrubtion more easily (1 to 1, 1 to n, etc). 

Python has libmosquitto-python bindings, which are provided with the mosquitto server. There are also perl, php, ruby, C, C++ (and more) bindings. http://mqtt.org. Chris Yeoh has setup a MQTT broker at his home for home automation. He's got some gnome applets, mobile devices, website graphs, and sensors all connected together. Another example was a system that had a TV android burglar control system. And using MQTT in a system to drive black taxis around a race track using the mind control gear. Setting up a bunch of ruber ducks to light up when a phone rings.

To get the code, try Really Small Message Broker, rsmb, http://wwww.alphaworks.ibm.com/tech/rsmb. License is free for personal use... Andy was required to mention this because he works for IBM. The source isn't available. Or you could get http://mosquitto.org which is open source, and runs as a daemon. It's IPv4/IPv6 capable. There is Arduino support.

He mentioned http://homecamp.org.uk, and he talked about some home projects. Some people are using MQTT for desktop notifications. Web thermometers. Digital to analgoure readouts. CEIT @ UQ http://ceit.uq.edu.au/content/messaging-protocol-applications. LEGO microscope control.Weather stations.

He then moved onto to talking about messaging in the enterprise. Turns out rsmb can act as a bridge to other brokers. They have some software to hook up several brokers together into some kind of super broker.


7 habits of highy ineffective project managers

| No Comments | No TrackBacks
Carol Smith works with Google. I'm sure it was a good talk, but I was trying to do 2 other things at the same time, so I'm going to have to check the video recording of the talk to be sure.

Liberating information from foswiki

| No Comments | No TrackBacks
Paul Harvey got in his turqoise coloured polo shirt, and white shorts, ready to tell us all about some cool stuff he was doing with foswiki. But as the keynote had gone overtime, we had to wait a little until before people came down from morning tea.

He works at the Centre for Plant Biodiversity Research. And at his work place there has been a long history of overusing microsoft excel for grabing information, and reusing it.

Foswiki is written in Perl+Javascript. It's 2.25 years old (since it forked from TWiki). Biodiversity is in a bit of a mess when it comes to loking after their data. He described why he went with foswiki, and it's openness and plugins were important to them. Also the plugins for foswiki is good. It has 262 released, and more in svn. Fos had a good modular architecture. It is a bit of an old perl project, so it has some old nasty code.  They have WYSIWYG. They also have a "Whi-it-up-titude", enabling wiki patterns. 

He described a data form. If you attach a schema to a topic (ie a page), you can add data to the page with forms. But using these facilities are hard for normal users, you have to do lots of reading of the manual.

Paul found that ownership was the key feature that his users kept using excel. They didn't want to go to databases, and databases often aren't really flexible. And you sometimes need to be an expert to use and adapt.

He mentioned google DataWiki, which surprisingly enough do a better job of describing foswiki than foswiki does of describing themselves.

Paul kept talking about foswiki, and where he works, and what they do. But he didn't have a great narrative, and he was jumping around a bit, and not really giving the uninformed user (us in the audience) what he was talking about. Which made keeping these notes difficult.

He said they are trying to use foswiki as a kind of portal to other information, and other systems. So they are trying to link to other data sources (like information on a plant, it's species, it's order, who classified it, or re-classified it). 

He started talking about some Wallace Core thing they had been working on to help the biodiversity area coalesce onto common set of terms. But I got board, as I didn't see how that related to foswiki. 

Paul hadn't tailored his talk to the audience, but was more talking about what they had done, rather than just using examples to help the audience understand the topic. So as far as a talk goes, it was pretty average, even though I thought the underlying material could have been really interesting.

Mark Pesce Friday morning keynote

| No Comments | No TrackBacks
Marks on mpesce@hyperreal.org, and @mpesce on Twitter. His title is 'Smoke Signals'. He's involved in VRML, and doing other 3D stuff at various universities. He's been a regular judge on ABC Inventors, on Triple J and on ABC TV. He's entire talk is all CC, and rated PG-13.

He mentioned that he likes to do talks that don't pay him, because he gets to say what he really thinks. He first learned BASIC programming on a risc CPU. He's written code for pretty much all the CPU families and micro controllers. In the 90's he read Neuromancer, and he went to Siliocon Valley and started a company to start a 3D environment for gaming. The company bailed. Just like all the other VR companies. They all failed because they wanted to win the whole game, but they all locked it up with patents, and they all failed.

17 years ago this week he folded some software together to come up with a 3D browser. This was the start of VRML. 

A resource shared is a resource squared. It took him 1 billion seconds before he understood the value of open source. He finds the more he gives away the richer he gets. But he doesn't measure richness in just his banking statement.

He talked about how he gave up smoking. He described how behaviour imitation leads to the spread of smoking. If you are surrounded by smokers/obese people then the chances are that you will smoke, or be fat. Divorce spreads by memesis. What we choose to imitate is determined by how we choose to associate with.

Marks talk had a lot in it, and came at us very fast, so I'm not going to be able to write this blog post just now, perhaps I'll be able to come back to it.

extracting requirements from flame wars

| No Comments | No TrackBacks
Paul McKenney said he'd been happily ignoring the flame wars about the android platform on the linux kernel mailing lists. He described a parallel of blind penguins building an enclosure for an elephant. Sometimes the blind penguins talk, and achieve concensus, and come up with a good solution.

But with the android stuff they haven't coalesced onto the right way. So he tried to work out some requirements from the android flamewars. He asked the question of why people flame on the mailing lists. There were some answers from teh audience.

You could calm a long flamewar by simply writing the combatants positions in a neutral tone. If it's a short flamewar, you have to already know a lot to help out, because you have only a small amount of time to try and help. If you do try and help, but people may attack you if you try to help.

Paul says if you're going to engage in a flamewar, you need to know what your goals are. If you just want to have fun, you could be just another flamer. You might want to educate yourself, or understand the positions of the flamers. You might want to present a neutral view of the positions, or advocate for a position, or present a neutral view of all positions and even add a critique of each. You may even want to propose a solution designed to meet all participants need.

To do your homework, you should review textbooks, datasheets and technical web sites (sometimes flamewars are because some are misunderstanding some hard facts. Be aware of any axe-grinding that might be taking place.

When you read the flamewar messages, it will probably take you longer to read and analyse than someone posted. He said you need to take notes as you go, to help you build up a picture of what is going on, or what is inside peoples minds (beyond their flameish language). 

He got to talking about power management, and how in the embedded world they work really hard to save power. They work much harder than you do if you are working on servers or laptops. He showed that power efficiency is a system wide issue, not an application specific application. You want to maximise your time that the CPU is switched off.

Paul showed some flames, there was some discussion, and he pointed out some hidden assumptions. So he found it wasn't enough to listen to what people say, you have to strive to understand what they are thinking. To do that you have to understand the technology. People who are strongly opposed, or strongly wedded to a solution are unlikely to welcome use cases, or alternate solutions.

Paul finished by talking about the current state with this flamewar about android wakelocks, what they want, what other people object about, and what might come along in the future as patches to solve some of these problems, or get some of the android patches into mainline.

perl best practices

| No Comments | No TrackBacks
Jacinta Richardson first referred to Damin Conways 2005 book. The obvious things she said was use warnings, use strict. Have a coding standard. 

She said to use 5.8.9, 5.10.1 or 5.12.2. She said 'use v5.10.1;', to let you use all the new features. She mentioned say(). Also defined or, which is //.
 $price //= $item->cost
So if the price is not defined it gets set to the item cost.

She mentioned state variables. "state $images_allowed = 100;". So the variable hangs around after the function call, and when you return to the function it is still initialised.

She mentioned the given when construct, which is like a C switch statement, but more powerful. You can do a foreach when loop as well, which is pretty cool. The when looks work because of smart matching, ~~. Smart match does magic comparison, which handles if you are comparing to an array, or a string, or a hash, etc.

From 5.10.1+ you have autodie in perl. So you just 'use autodie;', and system calls automatically throw exceptions. To catch exceptions you no longer use an eval block. Now you can 'use Try::Tiny;', and then we can use try{} catch{}; blocks. Within modules you shouldn't call 'die', because the error message isn't so helpful to the user. So instead 'use carp;', and then do 'croak "we broke";'.

She highligthed using perlbrew to easily get versions of perl installed into your home directory. There is local::lib to make it easy to test local versions of modules. It sets up perl environment to use local module. cpanm can be used to get install into your homedir the perl modules.

module-starter is a great way to get started writing a module. It has templates to create module stubs, stubs of documentation, initial tests, a Makefile.

Jacinta talked about Smart Commenting. Then she talked about tests, and said that it is really easy, if you 'use Test::Most tests => 4;'. She talked about using perltidy to make your code more pretty. Can use perlcritic to review your code (in ways that are easy to automate). It refers to Damines book when it explains why things. Like saying you should use strict.

She reminded the audience to use CPAN. She highlighted some modules to remember, like List::Util shuffle. She said to use an ORM rather than SQL statements directly. DBIx::Class and some other thing were the favourite ORMS. There are some frameworks, such as Catalyst and Moose.

Moose makes Object Orientation much better than non Moose OO in perl.

Method::Signatures allow you to do
 func nom ($biscuit, $cookie) {}
 method get ($key) {
  return $self->{ $key };

Path::Class abstracts away interacting with files and directories, which also makes the code more portable to other operating systems.

Regular expressions are a language within perl. She said to use \A and \Z meta characters. So use \A instead of ^ to match the start of a string, in case /m gets added to the regexp later on. Similarly \Z instead of $. You can use alternate delimiters. You don't have to write '/\//\//', you can write 'm{//}'. Try to use qr{} fragments, if that make sense, qr stands for quoted regular expression. Since v5.10.0+ you can do named captures from regular expressions.

She says you should avoid regexp that have already been written. Use Regexp::Common.

Perl 5.10.0 added grammars to the language, and sometimes you should use a grammar instead of a regular expression. This is in Regexp::Grammars.

She referred to Michael Schwerns talk on Monday, about perl5i. SHe said if you missed it you should go and watch the video. From her example it appears the perl5i automatically does lots of use statements for the features Jacinta has been talking about.

bat phone

| No Comments | No TrackBacks
The Serval Batphone is what Paul Gardner-Stephen is talking about. The bat phone is part of the serval project. servalproject.org. The gensis for this project was the Haiti earthquake, which was approximately a year ago. He talked also about other areas where installing cellular infrastructure is economically unviable, and difficult.

So they want to try and develop a technology that doesn't rely on big infrastructure. Servals focus is on telephony capability, then data later. THey have a close affinity with villagetelco.org. Villagetelco have found that 80% of phone calls are over less than 4km.

They started out with 3 assumptions. The system should be able to work on the far side of the moon (ie, in complete isolation of existing infrastructure, not even satellites). The second assumption is that the users are not expected to need to know stuff, it has to work like an ordinary phone. The third assumption is that the telephone directory is fixed, and that you must be able to use existing phone numbers.

So their solution has to be wireless, as to be self organising ad-hoc mesh. Has to be self-claiming of telephone numbers, or even self issuing of phone numbers in a greenfields area. There needs to be authentication, so you know phone numbers get hijacked.

So for authentication they came up with voice signatures. Users record themselves saying their own names. When you call someone, if the call can't be authenticated you are prompted to listen to their voice tag, and continue if you recognize the voice. They do have some PKI in there though, with the hope that the local postoffice could participate in building a web of trust. Or the web of trust could be built up after a few phone calls (users are prompted after the call if they want to sign the key).

They got some seed funding from The Awesome Foundation. He spent 6 minutes to fill out the application, and had $1000 within 6 days. They had a successful tech demo in Arkaroola Sanctuary in the Australian Outback, and have proved the technology works. They have had lots of enquries from NGOs, comapnies, local communities, emergency services. They did a test to for making calls deep underground. They have got working their 3G PSTN gateway working.

He described with some maps scenarios of how it would work. His maps showed it becomes really simple to re-establish communications, and the people can do it themselves, they have the technology in their existing phones.

Paul did a demo, using G1, the original android phone, as they are easy to program. He also had a 'mesh potato', I think from the villagetelco project plugged into a telstra handset. THey have just bought the phone from ebay, it's got no special hardware, only software. The demo started out between a mesh phone and a GSM phone via another mesh+GSM phone. He tried a mesh to mesh phone call, but it failed, because both phones were configured with the same phone number.

Paul said that SIP is a very chatty protocol, and is not great for low bandwidth stuff. SO they have created Air-Clutch, their own protocol. They are doing a demo outside at lunch. THey have a technology road map, and are applying for grants and commercial funding. He says if he gets 20 good people he can change the world. He wants to make sure this technology helps all people, especially the poor and vulnerable, and can help bridge the digital divide.

Paul says the technology is not going to cost any money to put into all phones that do wifi.

Eric Allmans keynote

| No Comments | No TrackBacks
Eric was the creator of sendmail. He started off with a review of the history of sendmail. Sendmail is an old program, that has survived well, in a very changing well, with no initial commercial support. It got kicked of in 1980 at U.C. Berkeley. Eric was meant to be working on RDBMS. Back when it started CPUS were 8 or16 bit running at less than 1 MIPS. Disks were much less than 1 GByte. Network was less than 56 Kbps.

At Berkley they got an arpanet connection. As they had so few ports, there was congestion on usage (people were getting into fights over who could use it). Eric worked out that the killer app was email, the professors weren't really trying to use all of arpanet, but were trying to use mail. So he wrote a store and forward mail system.

He showed some diagrams of the different components, and a code sample. He started off with some design principles. He wasn't going to change the local mail store, he wasn't going to redesign user agents. He had to make delivermail adapt to the world, and one programmer is finite.

His quick hack (delivermail) had some issues. He showed a table of email addresses in different systems, and where it was going. Berkeley got awarded DARPA contract to write 4.2BSD, and Eric got approaced by Bill (don't know who he is) to write the mail thing. So Eric agreed and sendmail was first released in 1982.

In the years 1981 to 1990 the unix was were waged, and most unix grabbed sendmail and lots of different versions of sendmail sprung up. Eric came back to Berkley and (for various reasons) decided to do a sendmail re-write, which pulled in a bunch of these versions. So he made sendmail 8, and integrated new SMTP protocols, new Protocols like DSNs and LDAP. The sendmail 'Bat' book on sendmail got written.

Then in 1998 he started sendmail inc, so he could get back doing coding and other people could do support. But he found out that starting a company means more about money, marketing and sales than coding. This was one of the first commercial/open source hybrid companies.

So Sendmail inc implemented encryptiong and authentication, and milter, virtual hosting, multiple logical queues, LDAP and lots more checking. They were driven by commercial needs, though opensource needs those things as well.

Eric then moved onto the second part of his talk, what lessons has he learned from the history of sendmail. Firstly he said that requirements always change. It started with reliability (mail had to get through, or see an error response). Then there was functionality and performance needed. Then it became about protection (against spam and viruses). Then legal and regulatory compliance, and keeping costs down. He highlighted that the Waterfall model of development doesn't work for all of this.

Eric has been critizised for some of his design decisions. Though some of his decisions were probably the right one at the time. Send mail is 'overly general'. But the mail problem is in flux, so sendmail is a tool for the mail problem, not a solution.

He said using tabs as active characters was the worst decision in his life. But use of rewriting rules was the right decision, even though it could have been implemented a bit better in hindsight. He says that message munging was absolutely essential for interoperability at the time. It's still heavily used, but not always necessary.

He agree's that the syntax of the configuration file is pretty ugly, but not fundamentally flawed. Today he would choose something different.

The queue keeps two files per message, one for envelope and header and one for body. In retrospect these days he'd use enevelopes in a database. But it was the right decision for the time (remember embedded databases didn't exist back then).

He agree'd that the syntax of m4 was painful. But he needed a macro language. There really wasn't anything else he could use, and he wasn't going to write his own. He said the main problem is 'dnl' statements, which aren't even necessary.

He found that with masquerading he made the wrong decision for the extending versus changing features debate. He did masquerading the wrong time the first time. Then he added new features around it to try and fix it, but he should have just changed it. He didn't want to break existing sites, but this created a larger problem for the future.

Some people believe that sendmail has allowed broken software to persist, following Postel's Law (liberal in what you accept and conservative in what you generate). At the time every thing was broken, so it was the right decision. But the long term effect is that broken software has no incentive to get fixed.

If he was doing it again today he would have not used the v7 mailbox syntax (those mboxes). he's used a bunch of modern tools that didn't exist back then. He'd use more privilege separation. He'd create a string abstraction. He'd use separate mailbox names from unix user ids, and a cleaner configuration file.

But he'd still use C for the implementation. He'd still use syslog and rewriting rules. He thinks that OO languages are a mistake. He says OO languages do too much under the covers. He'd continue to do things in small(ish) chunks. He still would have written syslog (he wrote syslog as a side project). He wouldn't rely too heavily on outside tools, as tools have their own cost.

He wrapped up with some takeaways. KISS principle actually works. Even though sendmail isn't simple today, each step was a simple progression. If you don't know what you're doing, advance designs don't help much. That was certainly the case when sendmail was being started, because the mail systems weren't designed yet. Flexibility beats performance when the world changes. He suggests you fix things early, because the installed base only gets bigger. Documentation is key to broad acceptance. The design space is always changing.

latest and coolest html5 media

| No Comments | No TrackBacks
This is a talk by Silvia Pfeiffer. She was introduced as an internationally renowned expert on web video. Looks like she has written at least one book (as she's giving one copy away).

http://caniuse.com. It's a website that tells you which features are supported in which browsers. She includes more than just the new HTML5 markup tags, but CSS3, web workers API, etc. The whole HTML5 platform. So SVG, Canvas, audio, video.

She said her book is already out of date, she's running http://html5guide.net/ for updates to the book content (but I must ahve the wrong domain, because that site doesn't look right). She showed a html <video> tag, with 3 source elements for different movies, and a little bit of css. CSS3 allows for transitions, she showed how a video can expand when you hover over it using that. You can also use a rotate tranformations. In safari she showed a 3D cube that has videows on all the sides.

She showed using some javascript to manage a video element. Not only that you were able to catch these media events and report on them, and display all the media properties. She also showed using SVG filters ontop of videos, which was interesting.

Apparently canvas is better supported across the browsers. She demonstrated catching some events from the video element, and then drawing the video image onto your canvas frame. She showed some javascript programming where she grabbed the pixels from the video frame and used that to do some cool stuff, like change the colour of the border, or grabbed the white writing off a video and draw it onto another picture, or add a reflection to the bottom of a video.

She showed a demo where she had a web page that we all connected to, and we all watched the video together. When she clicked play on the presentation laptop, it started to play on all of our browsers.

She showed a demo by mozilla, where they started playing an audio clip. Then showed the audio samples. Then displayed them, then filtered them, then visualized them, including a 3D visualization.

She showed some subtitle stuff, called WEBVTT. These are used as <track> elements within a <video> element that reference this .vtt files. These can be styled with CSS using ::cue. At the moment these vtt are implemented in Javascript.

So it was a very fancy presentation, and it looks like at some point that future websites will get even more annoying. <video> isn't really the new <blink>, but I'm sure some people are going to use it like that.


| No Comments | No TrackBacks
Lennart Poettering talked about systemd, which is a replacement for the sys-v init.d scripts (which start software on Linux computers when you boot up). systemd is a system and session manager for Linux. It looks after login sessions. systemd is compatible with sysV and LSB init scripts. It can use paralleilzation of service starts up. It uses socket and D-Bus actifavation for starting services. Can do on-demand starting of daemons. Uses CGroups to track processes. Can snapshot and restore system state. Maintains mount and automount points. Can look after services considering dependancies, and does transactions. (All of that is a re-phrasing of the short description they provide on their website).

The kernel starts one process when it boots up. This is process is init. sysv is an init process. Process 1 is special. All other programs are children of the init process. If the init process dies the kernel throws a kernel fault.

In systemd everything is started in parallel, despite the interesting feature that (for the example given) d-bus depends on syslog, and avahi and bluetooth both depends on syslog and d-bus. Apple solved this in their launchd approach by ripping out the socket binding of all the daemons (ie the syslog socket and the dbus daemon) and binds them on behalf the daemons. This is called socket activation. When the init process detects that something tries to connect to a socket you then start the daemon.

This is actually not a new idea. The inetd has been doing 'start on connection' type of work for awhile. So some daemons don't need to be modified to work with systemd. Someone asked if you can use LD_PRELOAD to trick proprietary software (that you can't patch), but apparently it's really tricky, and the systemd people haven't done that.

systemd can automatically restart services that crash. Not only that, but it will get the same socket back again, and so it will not drop lots of connections because systemd kept a duplicate of the socket, so those client connections haven't been dropped.

systemd can also do bus-based activation. So systemd adds some bus names (or whatever it's called) for the services onto the bus. Then when something asks the bus for that bus name, systemd starts up the service to satisfy the request. They also use hardware based activation (like when a network interface comes up) to start things. By default they don't shut down services when hardware things go away.

They extend this idea of when-needed-start-service to file systems, to parallelize file system jobs. This is used for mounting sysfs, the filesystem for binfmt. 

in systemd they try to avoid shell programming, because shell programming is quite slow, because you do lots of forking as you run grep, awk, cp, mv, etc commands. In systemd they don't want to use the shell in the boot process. Often the init.d shell scripts are all the same. The have a case statement, they pick some configuration files. In the programming world when you have template code you try to avoid that by refactoring it out, so that is what they did.

Of course there are other things like setting the hostname, or doing modprobe, or starting udev, or removing a .pid file if it already exists. And systemd does that. They have also built in 'proper' debugging facilities. For instance they put in tracing, graphical dependencies, and an interactive bootup.

systemd is meant to be a good babysitter. They use control groups to supervise services. Control groups allow you to group processes into a hierarchical tree. Control Groups were originally about managing resources. However the implementation is abstract, so systemd can use this.


| No Comments | No TrackBacks
Jeff Waugh talking about node.js. node.js is not a javascript file. Node is a javascript environment. You can get a command line when you just run ig, much like python. The mission for javascript is meant to be interesting.

According to the creator, 'To provide a purely evented non blocking infrastructure to script highly concurrent programs.'.

Jeff said asyncronous is important, by showing an example of bunnies and hamsters. He moved on to talk about how nginx and apache compare with memory usage against concurrent connections. Apache uses more threads for more connections, so uses more memory. The graph comparing requests per second with concurrent connections show's apache goes down compared to nginx as well.

Jeffs first experience with nginx was putting it in front of apache when working for crikey.com. Apparently javascript was originally designed to be a hosted scripting language, or embedded language, orways designed to be single threaded and based on an event loop. The event loop is just part of the language. Javascript makes event programming natural, it's just how you do it.

People are raving about node.js because it's built on top of some cool stuff like libev, libeio, c-ares (async dns library), http_parser (really fast http parser) and OpenSSL. Then the v8 javascript engine is bounded onto that via some slim C bindings. Ryan Dahl is the maintainer of node.js. Jeff said he is very quiet and humble.

In node.js the entire stack is evented. There is no tricky lurking exceptions. There are a couple of blocking calls that you don't have to use if you don't want to. CommonJS modules in the last few years has been part of what has made this easier. Also the browser performance war lead to v8, and hence a fast javascript engine.

node.js only really runs on x86, x86_64 and arm CPUs. v8 hasn't succesfully been ported to other places. The 0.2.x has crappy SSL/TLS support. There is an attempt at a windows port, but no one is really working too hard on it yet. v8 has a heap limit, at the moment 1.9Gb. But if you're using node.js for a glue service thing then you probably won't run into this.

There are alternatives for other languages. Like Twisted for Python, or EventMachine for Ruby, AnyEvent for Perl and NIO2 for Java.

scaling programs

| No Comments | No TrackBacks
Theodore Ts'o began saying how we used to write code so it could scale up to a lot of CPUs, and then we stopped worrying about it. Back in 2001 the linux kernel started having SMP, so support for multiple CPUs. Used to have computers with a lot of sockets for a lot of CPUs, and those systems were very expensive.

SMP is difficult, because if computers share memory, they need to make sure they coordinate about the memory (ie, make sure a cache in one CPU doesn't have old information about whats in memory because another CPU wrote to it). Then they tried to implement NUMA (so CPUs prefer memory that is closer to them), but that's complicated, and can cause performance hits.

So a machine with 4 sockets costs a lot more than a machine with 1 socket. There are a bunch of benchmarks for single CPU systems to work out how fast a system works. But all the different benchmarks test all sorts of different things. To mention SMP scalability S = (score on N cpus / score on 1 CPU). Ie, Linux 2.6 scales to 12 out of 16 CPUs on the blah benchmark. 12 out of 16 is considered pretty good. But far short of the 100% scalling up of processing that you'd desire.

In early 2000's theere was a linux scalability effort. They had weekly conference calls. There were a lot of companies involved. They had weekly/monthly bnchmark measurements by a performance team. So this occurred for 2 to 3 years, then decided that it was good enough. At the start of 2001 we had really poor benchmarks upto 4 CPUs. After this we got upto 6-7 out of 8, or 12 out of 16, and an acceptable number of CPU's on 32 CPU systems.

But this effort died down for several reasons. One of those reasons was that people spending big money on high end gear preferred to run the high end legacy OS's. And linux succeeded wildly on x86, which had very few servers that had more than 8 to 16 CPUs. Linux was used for scale-out computing (running lots of separate systems). And for approximately the next 4 to 5 years nothing was really done for scalability. During this time we saw the rise of Linux on embedded and mobile equipment.

But then CPU frequencies haev stopped doubleing every 12 months, and now we are seeing in mainstream CPUs that have multiple cores. So scalability is starting to matter again. So Theodore suggests that it's time for kernel programmers to think about scalability testing, and for application programmers to think about multiprocessor programming.

He moved on to talking about ext3. Historically most workloads don't really stress the file system, other bottlenecks are hit first. In Enterprise databases are tending to use Direct I/O to preallocated files. And ext3 was pretty good for these direct I/O. But ext3 doesn't perform well in benchmark competitions. But as it wasn't the bottle neck, so system administrators didn't care because it worked, and it was easy to service if things went wrong.

For ext4, in April 2010 IBM's real time team was trying to make it better. They found they were spending 90% of their time on spinlocks. This was in the journal start and stop functions. Theodore said you need to document which lock is used to protect a variable, and what order the locks need to be applied.

As transactions are expensive, in jbd2 (the journalling code for April 2010 ext4) they gruop multiple file system operations into a single transaction. Transaction commits happen every 5 seconds, or when the journal is full. So each file system opration is bracketed by some jbd2_journal start or stop function call. And those calls need to grab some locks to manage some meta data. He found the j_state_lock spinlock was apparently not being used to protect any data. So removing that lock immediately improved performance.

He showed some graphs of how the patch improved performance. The graphs included xfs, which showed xfs was heaps better. He moved onto a benchmark/tool that shows information about how locks were used. But I didn't understand which locks were being referred to. He made some changes to the jbd2 locks, and performance got better.

Then the next problem was that the ext4 layer was submitting writes to the block layer in a 4k block at a time. The block layer would merge them together, but that work meant there was more CPU work doing that. To fix this required quite a large overhald and cleanup.

He finished up by mentioning a bug that needs fixing, and that we need to start thinking about multithreaded programming again. A lot of what h talked about can be used in application programming. He suggested having a look at valgrind's drd tool to find data races, and Lennart Poettering's mutrace tool.

Behaviour Driven Infrastructure

| No Comments | No TrackBacks
Lindsay Holmwood is talking about how to add behavioural driven development to infrastructure. He described test driven development. Starting with unit tests. Behaviour driven development is a reaction to thos tests, like can a user perform a particular task. So it's more system level tests. So it's more about testing the business needs, verifying functional requirements. The business doesn't care so much about how the solution is implemented.

So the business type of tests are outside-in tests. Possibly automated, possibly manual. There has been a movement to specifying theses tests in a language businesses can under stand (they can read it), but in a syntax the programmer can automate.

Andrew Schafer (a founder of puppet labs) suggests infrastructure is a application (this is an abstration). The daemons (such as database or web server) are your libraries, the configuration management is your code. And Cod without tests is bad. So we're bringing tools that developers have been using, and bringing them over to testing your infrastructure.

Cucumber is a tool to write tests (in high level language), and a tool to execute that specification. Lindsay popped up an example. It had a bunch of components, each of which was like a unit test. Then combining them all together was like a system test.

He showed an example of doing cucumber stuff, using his cucumber-nagios shortcuts/tool. Except he had some problems with using the rubygems bundle program, and stuff didn't work for him.

So once you've got these tests, you can do continuous integration. You can re-run the tests when you commit changes to your manifests. He speculates whether you want to run the tests on Production or Staging, or UAT. He asked how you do destructive tests. He asked how you apply the setup/teardown pattern from development testing to infrastructure. He suggested you have A/B testing (but didn't really say what it was). He doesn't have any answers for how you do that, without breaking production.

He moved onto the topic of how do you migrate to a configuration management environment from a system that was hand built. He suggested that you can write a bunch of tests to model the behaviour of the existing systems, and when they all pass, you run the same behavioural tests on a new configuration managed environment.

He suggests we've been running the wrong questions in our monitoring. The standard checks are ping checks, or connect checks. We normally do 'unit tests' of the infrastructure. He says cucumber lowers the barrier of entry to writing good system level checks. He's also saying that cucumber provides a common specification format that dev and ops can share, or/and that IT and business can share.

He talked about some movements within the cucumber community to put together a forge of common specifications.

LCA2011 Wednesday morning keynote

| No Comments | No TrackBacks
Geoff Huston is here for this mornings keynote. But first Someone (didn't get their name) got up to show a video of a balloon launch. It was cute. Had a penguin in the middle of the screen, and the baloon went a long way up. Apparently it was related to the fund raising for the queensland floods somehow. Oh, they have a high res photo that they are going to auction off on Thursday to raise money.

Shirda got up, he's the technical manager for OLPC in Australia, and he spruked their project for a bit, and gave out an OLPC laptop. Ben Sandsfeld got up to talk about his campaign against software patents in Australia. He wants us to sign a petition, which is out at reception.

Geoff Huston got up, he's from APNIC (Asia Pacfic organisation for allocating IP addressess). He described our Unix was a by product of how AT&T couldn't commercialize Unix, so they kind of gave it away for free. And how TCP/IP also got started at around the same time, and how both these projects were early open projects, a pre-decessor to all of our opensource software. Openness is really good. In the 70's, if you had stuff from one company, your entire technology stack was from the one vendor. In the 80's the openness kicked off by unix and TCP/IP mean that vendor lock in started to fall apart, and the industry fundamentally benefitted. Geoff said that IPv4 didn't succeed because it was good, it was just as good as any other technology. But it was open, which is why it succeeded and gave us the Internet today.

Geoff says that the problem with the Internet is that sustaining openness is really hard. Geoff isn't sure that an open Internet is going to still be around in 5 years time. There is a constant pressure by comercial companies trying to develop the technology, who want to reduce the openess.

Things like Net Neutrality, Nex Gen networks, mobile internet, Triple/Quad play technology all apparently rely on closed technologies.

He showed some graphs of IPv4 address allocation. The graphs showed the addresses running out. In 1990 they started working on the address problem. Back then they had the idea that they wouldn't run out of IPv4 addresses, the transition to IPv6 would have occured early enough that this wouldn't be a problem. The idea was that the industry would act rationally to do IPv6 deployment. But it hasn't really happened. We're looking like the entire Internet is going to have to implement IPv6 in 7 months.

At the moment IPv6 sucks. There is a lot of stuff that is broken. So we are going towards an Internet that relies on NAT. ISPs are going to have to deploy large NAT systems, that will be expensive. NAT is going to eat into the tcp and udp port space, so things are going to start breaking, as that address space is only 16 bits.

IPv6 isn't backwards compatible with IPv4. So you still need IPv4 as well as IPv6. But a dual stack transition would only work if we hadn't run out of IPv4 addresses. So Geoff predicts there is going to be a market for IPv4 addresses. 

He mentioned how carriers lost huge market share (from complete control of the telephony stack to computers using packet technology). But these carriers are who we've been asking to invest lots of money to get IPv6 working. Groups like Google and Amazon probably don't need IPv4. Consumers aren't keen on spending more money to pay for IPv6. So he thinks the economic points show that we aren't heading to an open network. That we're heading to a closed network.

He says we need to fix this by altering our environment, so that Telstra, ebay, etc see why they should continue with open networks.

En-Visage-ing for collectd data

| No Comments | No TrackBacks
Lindsay Holmwood is the lead developer behind Visage. He started off with a use case that a web server is having some intermittent problems. It's about collecting metrics of the servers. He mentioned sysstat, dstat,sar, top, etc. But it's hard to correlate the data.

He uses collectd, a lightweight collection of data. It's plugin based. There is a network plugin that makes collectd network aware, that sprays the collected data over UDP to another server. Collectd uses RRDtool to store data. He pointed out it's hard to merge the data from RRD files to view stuff. Visage exposes the RRD data to expose it as JSON, and uses javascript and SVG to render it on the screen.

Visage has Profiles (saved pages of graphs). ANd it has a builder. It's distrubted as a ruby gem. He did a demo. You can change stuff all around in live view, and remove some things, and view, and do a live view.

It looks very cool!

Varnish and HTTP acceleration

| No Comments | No TrackBacks
Simon Lyall said with a typical LAMP stack you can get 5 to 10 hits a second. He suggested if you add a data cache (caching of DB queries) you can get upto 100 h/s. If you add a page cache you can get upto about 300 h/s. To get way up from there you need big infrastructure with load balancing.

If you add a web accelerator in front of the web server you can get 1k to 5k h/s on commodity hardware. You can get upto 20k+ h/s with some tuning. Varnish isn't the only one. There is also squid, lighttpd, nginx, apache traffic server, and more.

You need to balance between keeping hits off the web application server, and showing fresh content.

CGroups for resource management

| No Comments | No TrackBacks
Steven Ellis is talking about cgroups. CGroups are about an in kernel feature and userspace tools and librarys to manage how resources are allocated to users and processes. Resources are disk, network and CPU.

His first example was resource management for virtual guests, but said you could do the same for running a server with multiple databases. You can uses memory, cpuset, cpuacct, cpu and other controllers (devices, network, etc). It's a heirarchy, so you can allocate a restriction to all daemons, or all http instances, and child processes inherit the restrictions. So you could set http to only have 30% of the memory. Or you can measure the number of cpu cycles. 

THere are a bunch of command line tools, that all start with 'cg'. File is in /etc/cgconfig.conf. Redhat have a way of setting CGROUP_DAEMON variables under /etc/sysconfig so that as the daemon is started it is immediately limited by that CGroup setting.

There is some kind of cgroups daemon for managing stuff based on which userid or groupid is used.


tuning postgresql

| No Comments | No TrackBacks
Shoaib Mir is talking about performance of postgresql, and how to tune it. He started with application analysis. How do you work out if it's reading large amounts of data, or writing, or doing complicated queries. 

You need to find which queries are taking a long time to run. Install a analysis tool, such as EPQA or pgfouine (Shoaib prefers EPGA). THen you need to use 'EXPLAIN ANALYZE' to debug them. However the output of EXPAIN ANALYZE gives hard to understand output. Sucks to be you. Don't use EXPLAIN ANALYZE with DML query in production (insert update and delete).

You need to lok at memory. Is the size of the database small enough to fit in memory, or do you need faster disks. What is the Buffer cache usage (pg_buffercache or OS cache).

Storage setup. He recommends RAID 10 over RAID 5 for write heavy activity. Suggests dedicated storage. Run iostat output and look for raits and queue sizes. Perhaps consider trending iostat output. Check if RAID controller is using battery backed cache, and monitor the health of the battery. Turn on the write cache.

Use tablespaces to put indexes and tables on separate storage areas, and transaction logs on separate storage.

He recommends XFS for the filesystem, as it has better journalling than ext3. He says to start with shared_buggers set to 25% of your RAM to start with, then use pg_buffercache to find the optimal value. He talked about a work_mem parameter that is used for sorting operations. If the a disk sort is being used you need a larger work_mem. It's a session variable that can be used for different sessions, so you don't have to set this at server start.

maintenance_work_mem is another session based parameter. You can raise it when you want to create an index, and then lower. 

He mentioned that bonniee++ has some database orientated checks. Suggests using check_postgres.pl for monitoring database health monitoring.

secure gateway

| No Comments | No TrackBacks
Mark Suter talked about their implementation of a secure gateway. They protect networks that have servers hat have low security efforts. They do firewall, IDS/IPS, router and switch administration, Linux adminstration, external hosting infrastructure, Web XML messaging gateways. His company has some kind of certification with their managed stuff (and if they loose certification they'll loose some of their big contracts).

He said documentation is good. They let clients have access to their nagios and cactus interface. He said un-needed secrecy hurts Security. They have some documentation that the client can view that details some of the Business 2 Business links.

lightning sysadmin miniconfs

| No Comments | No TrackBacks
Alistair talked about password security with samba and pam.  He installed libpam-samba. This allows samba client machines to do password changes. For password verification you can use sambas policies, but they default to only requiring 5 characters. Basically he just wanted to annouce he's going to file a bug report in debian. Yipeey.

Steve is talking about Enterprise File System, openefs. openefs.org. This is an approach to deploying applications to thousands of hosts, instead of doing package development. It sets up a global /efs name space, and appears to deploy software into a managed namespace underneath that.

Stuart Low is talking about iseek communicatinos, building an ISP using OSS. 99% of their server environments is Linux based. Termination of customers (ie PPP trafic, or DSLAMs into a LAC) is done via cisco gear, such as the CIsco 720 PPP. They have come up with a solution that uses a linux cluster with L2TPNS. http://l2tpns.sf.net/ It was originally written by two optus engineers in mid 2004, and was stable in 2006. iseek is now the developer for it. L2TPNS includes Walled-Garden support, and multicast radius blah.

Craig is talking about defending voip. He says they are always trying to attack you all the time. It's a big business. He suggests don't let udp port 5060 through, as you probably don't need to be allowing SIP on the internet. Consider requiring SIP phones to VPN in. If you do need to be on the internet you must use strong passwords. You should use fail2ban or similar to react to multiple failed attempts. (You should whitelist your office network). There is a iptables rate-limiting module, such as http://bit.ly/sipdos. The script works well for other protocols. Consider limiting the number of simultaneous calls on your trunks. Asking your telco to block all international calls (and get that in writing/email, so you don't wear the cost if they forget). Craig showed a picture of a very large stack of paper, which was an itemized bill for a one month of sip attack. Craig works for netsip.com.au.

DNSSEC at Mozilla

| No Comments | No TrackBacks
Shyam Mani spoke about implementing DNSSEC. He was involved in implementing DNSSEC at Mozilla. DNSSEC is a bunch of changes to make sure the DNS information hasn't been changed in transit, and that you can verify that a DNS name does not exist.

In DNSSEC you have your public key in a DNS record, I think that's in a DS code type. A DNSKEY record sits in the parents zone. This helps setup a trust chain. You have a KSK (Key SIgning Key), and a ZSK (Zone SIgning Key). You could do this with one key, but for operational reasons it's better to have two. So the KSK becomes the more sensitive key that can have longer life times and higher encryption. You can change over your ZSK key more often. Shyam said RFC4641 is a good read for implementing rollover of keys.

He showed http://dnssec-debugger.verisignlabs.com/mozilla.org, a tool for debugging DNSSEC issues.

He suggests that you check your TLD is signed before you get started. Some registrars may not have implemented DNSSEC, so check with them. Also see if your dns software supports DNSSEC. Apart from doing the key management and signing of zones, the changes to bind configuration are minimal.

He showed http://dnsviz.net/ to visualize the train of trust. He says you should plan in advance of what happens if the keys to compromised. Are you going to allow your dns zones to effectively to go offline. You should sign and publish your zones before you push your DS upto the parent zone.

He had some issues. Pushed DS before signing the zones. Bind defaults to debug logging of DNSSEC stuff, which is a lot.

A question from the flaw asked what the user see's if a DNSSEC shows there has been some DNS tampering has occured. They haven't yet got that in mozilla yet, though there is apparently a firefox plugin. Someone on the floor mentioned freebird, a dnssec proxy that does all the key management for you.

Backing up network gear

| No Comments | No TrackBacks
Julien spoke about backing up network gear. He said you should have a file repository of the OS running on each device. You should have a copy of the current and the new firmwares in your devices. It might also be a great idea to have some of your archive on a usb stick, and to have printed instructions on how to do a recovery.

Julien says that it's best to have an automated pull technique for managing the configuration of your devices. So something that regularly downloads the configuration. He suggests a tool called RANCID, which is a mix of perl and expect.

So once you've got rancid going you get a history of configuration changes. You get hardware inventories, and you get filesystem details (maybe not the contents of all the files, but at least if they have changed). With this information you can extract interesting stuff, such as the firmware versions you are running.

Julien generates a network diagram automatically from the information. Apparently rancid can also be used to push configuration up from templates. Julien mentioned netomata. He says it's a bit slow to start writing your templates for it.

He mentioned 'notch', which is a CLI abstraction layer written in python. Built ontop of notch is PUNC, a rancid replacement, and Mr. CLI, wihch is clusterssh for routers. http://code.google.com/p/punc/. There are some NANOG talks on rancid and netomata. Rancid http://bit.ly/hDSEaL and netomata http://bit.ly/f3Vpwe.

opengear wants to monitor anything

| No Comments | No TrackBacks
Ken Wilson from Opengear spoke about central monitoring. He talked about why you want to monitor stuff. To measure a bunch of physical things he suggests you use a small embedded linux appliance, and hook it up to nagios. He had a picture of an opengear applicance, but didn't mention what model it was.

Their devices can generate nagios configuration for serial events, temp/humidity UPS checks (via nut). They use passive checks, and find it scales better, though it is more complicated to configure (doing passive setup on nagios host, and setting up remote machine to run the checks).

Someone from the floor commented that you really shouldn't rely on an SMS gateway, you should have a GSM device with a SIM directly attached to your monitoring server, and you should also send a test sms every day so you know it's working.

Peters issues with IPv6

| No Comments | No TrackBacks
Peter Chubb spoke at the first talk after lunch in the sysadmin miniconf. He's from some sort of research group within NICTA. They decided in their implementation to add a third computer to act as a ipv6 router.

They had a problem with assymetric routing, so they added some static routes on their routers. Peter was using a laser pointer to highlight which machines that he was referring to, but it wasn't very easy to see. So I couldn't follow some of what he was talking about.

They don't have PXE booting over IPv6 yet, that is going to need manufacturers to fix their firmwares. They haven't got provisioning of things that DHCPv4 provides working yet, such as NTP servers, name servers, etc. He also hasn't got dynamic updates of DNS servers.

On his network he's just using the radvd auto configuration, but he's looking at evaluating DHCPv6 soon.


| No Comments | No TrackBacks
Turns out my laptop overheats if I use it on my lap. I should have realized this, as I'd taken to putting the laptop cover between me and the laptop as it was so hot. I can tell when it overheats, the screen turns off, and the screen saver comes on. Or the keyboard stops responding. I've added some temperature displays to my panel, so I can keep an eye on the temperature.

HA clusters in 20 minutes

| No Comments | No TrackBacks
Sander van Vugt sounds like that he's done this talk before. Normally he tries to get 3 hours for this talk, but he's only got 20 minutes. He wrote a book ('A Practical Guide to Xen HA', he's happy to email a pdf of the book. He's written a bunch of other books as well.

He mentioned corosync is used for the cluster communication layer. They communicate using multicast address. He mentioned a crm gui and crm shell interface. The bulk of his presentation was actually a demo.

He's using a syse, and hence the yast2 setup. He picked a random address, which is a private address space for multicast.

I wasn't really out to fit this idea of clustering into my brain of where you'd use it for, and what. So I was struggling to follow this talk. I wasn't going to ask for any background, as I'm sure other people were right upto speed on this. It sounds like the cluster can be used for a bunch of different things. He mentioned different modes the nodes in te cluster can operate. He talked about a cluster aware file system that could be used, or a master mode where nodes could be setup as a master.

Another example he used was a cluster where both machines could mount an ext3 filesystem, but they had a problem where both hosts tried to open the filesystem at the same time, and once they were both writing the filesystem it all got broken.

He mentinoed that in a cluster you need STONITH (Shoot the other node in the head). So that when the cluster detects a node has stopped functioning, the cluster can force the shutdown of a host, by either a rack PDU force powering down, or triggering a kernel panic.

You need resource monitoring, so you know if your resources are working. You need to remove the resources from your runlevels, so the cluster can start the resource, instead of the resource starting on bootup.

Resource stickineess is about how bad the resource wants to stay on the node it currently running on. So he suggests setting a positive value to ask the cluster to leave the resource on the node it's running.

samba 4 in the real world

| No Comments | No TrackBacks
Andrew Bartlett talking about Samba 4. They have implemented read only Domain Controller support. Apparently it is harder than read-write, but they are thinking it is a safe way to introduce samba4 into an existing Active Directory network.

They have real world deployment of multi-master replication. They have included role transfer. They are supporting extended schema support, such as the MS Exchange schema and custom schema.

One of the biggest problems they have is DNS integration. They almost started to implement DNS. Microsoft don't save DNS data into zonefiles onto disk, but in the directory and replicating it around. So Samba now provide a plugin infrastructure for Bind 9.8.0, and in the samba distribution they can provide a plugin to connect to the bind server.

They now have a new build system. It's a python based build system, using WAF. They can do smaller builds, parallel builds, and a lot of their samba-tool command is almost all python. They are doing regular alpha releases. the 4.0 release is a while off, until they have merged the fileserver and domain controller stuff. The fileserver stuff (and printserver, and old style domain) is in samba 3.6.

Samba4 uses a ldb backend, that writes to gdbm (?) files. Tridge said one of the main reasons they can't use an openLDAP backend is that it doesn't support transactions, which samba4 relies on.

tick tock

| No Comments | No TrackBacks
Julien Goodwin spoke about a brief history of time sync. He started off by mentioning some mechanical clocks, and moved onto quartz crystal oscillator chippy things. Apparently a server keeps pretty good time because they are in a temperature controlled environment, and those chips are fairly temperature sensitive.

Apparently you can buy quartz crystal clocks that are packaged in little ovens. Then he mentioned iridium clocks, and they aren't very big. Then he mentioned a clock built by HP (and now agilent), a cesium based clock. Then he showed a lab hdyrogen maesar clock which is more accurate over the short term, but less accurate over the long term.

Last week a 'quantum' chip scale atomoic clock came out. It's $1500 US each, but is very small. It's not a caesum standard, but it does use caesum, so it's pretty accurate.

He then moved on to talk about syncronziation. Apparently there used to be a clock in each city that was used to sync from. I didn't really understand the syncro stuff he was talking about, it seemed the application for some of this were particular applications. He got onto NTP, the Network Time Protocol.

In NTP parlance a stratum 0 clock is the master clock, and stratum 1 clock is the computer connected to that clock, stratum 2 is a computer connected to the stratum 1 computer. PTP was dveloped to 'replace' ntp. v1 in 2002 was LAN only. v2 can be routed, but isn't much better than NTP.

There are several time standards. GMT, UTC, UT0, UT1, T1R, UT2, UT2R, TAI. There are several time zone databases. DST changes several times a year. He pointed out some silly timezones (Liberia used to be 43 minutes and 08 seconds off an hour shift).

To get accurate time, he says don't use virtual machines as master servers, and he had a few other recommendations (using a GPS clock, using 3 to 5 NTP servers, etc).

Vint Cerfs Keynote

| No Comments | No TrackBacks
Vint started out by saying he wants to persuade us that the Internet today needs to have some evolution, and that it isn't to late to do so. He did a brief review of the history of the Internet, from it's arpanet days of 4 nodes, to somewhere around 1 Billion nodes today.

He mentioned Bob Kahn's open architecture idea. Global addressing. Routers not keeping state. Recovering from lost packets. Various operating systems. No particular application to be supported. Non-national IP address structure. Layering of protocol. IP packets don't know what they are carrying, it's just bits.

Vint says that openness has been really important for the Internet story. Having open standards, without any IP claims.

Apparently Vint decided that 32bits was enough for IPv4. He decided in 1977, and thought it was for an experiment. But IPv6 is coming, along with DNSSEC, RPKI, some other stuff. Thinks that IANA is going to run out of IPv4 allocations very soon, possibly next month. We just need to get on with IPv6.

Vint says that the worst problems of security on the net are often due to human error, choosing weak passwords, configuration errors. He talked about privacy problems, due to companies not deciding to protect privacy related information.

He mentioned clouds. How there are multiple cloud implementations that are built in different ways, by different people. He thinks that these independant clouds are similar to the isolated and independant networks of the 70's. So he thinks we should have ways of having clouds co-operate, and share data, and have access control regime's.

He moved onto Internet Research Problems. Problems that haven't been solved yet, but that these problems are solvable, even though the Internet has been for awhile. Things like Erlang forumlas (Erlang measured phone conversations) for the edge of the network. In the core things average out and you can do some predictions. QoS, Multihoming, Mobility, Broadcast utilization.

He mentioned that not having an address identifier in the TCP protocol, and using the IP address to bind it together was a mistake. It causes problems for mobility. Broadcasting of packets. Things we can do a better job of authentication and Identity. Governance is a quagmire, such as censorship, and other societal issues will continue to come up as the Internet continues to spread through our lives.

He thinks we need to work on how we monitor and expose performance issues in the network. He mentioned the Bit Rot problem, of old file formats no longer being supported by old applications, or old hardware.

Vint finished off talking about the interplanetary network, where they have been developing some protocols and deploying it on various equipment out in space. He's hoping that as some gear finishes their mission they become a node on the interplanetary network.

Squeeze Early February

| No Comments | No TrackBacks
It's been announced that Debian is probably going to release their next stable release of the Debian Linux distribution on the first weekend of February. Except I'm probably going to be up at Bright, to ride up Hotham. I'll have to take the laptop with me.

Linux input from bottom to top

| No Comments | No TrackBacks
Daniel Stone (currently working with Collabora) is talking about the linux input stack. I don't expect I'm going to understand most of the talk. He was going to talk about multi-touch, but Peter is doing a talk on Wednesday about that.

A keycode is a number for the key pressed. A keysym is an identifier for a character. Shift bumps level up by 1, AltGr up by 2. He mentioned pointers, and how some trackpads send you absolute pointers, but what you really want is relative, as you don't want to map your trackpad co-ordinates to your entire screen.

He mentioned that with the raw input they can do some transformations, such as bouncing keys, slow keys, acceleration, multifinger stuff, or palm detection. Daniel described how complicated X event delivery can be. Deliverying hotkeys to specific places, grabbing pointers for dragging windows. But grabs apparently come with some limitations (you can't hit brightness key during a popup menu apparently).

XKB extends the X core keyboard map, but according to Daniel most of it is staggeringly pointless. You can remap your VT switch scripts. He is slowly working on an XKB2 implementation.

Norins ideas on Saving the world

| No Comments | No TrackBacks
Noirin Shirley talked about some of the open source projects she has been involved in. After the 2004 Tsunami they turned to OSS for some software to help with Sahana, a disaster management system. She pointed out that within a week thousands of hours of FLOSS programming time put together the first release of Sahana, possibly not a feat that a commercial company could have achieved.

She defined Hmanitarian Asistance as the long term response to problem areas. Disaster Response as the immediate concerns after a disaster, and normally only lasts a few weeks later. Emergency management occurs over the time frame of several weeks.

She referred to the ushahidi.com software, that was developed in response to the Kenya troubles after an election and violence broke out. But now it's used by lots of places, and was used for the Queensland flood crisis map by the ABC. People send in reports of problems (flooding, fires, violence, volunteers needed), and then volunteers work to verify the reports. ABC had already been experimenting with Ushahidi before the floods, on a site for feral animals. So they were well placed to use the same technology for the Queensland floods. 

Noirin talked about a talking book. In developing areas this is used to send information out to places that don't have an IT infrastructure. A order of 1000 costs $5k. Two books can be connected together to share the audio. It's all open source, uses a battery that is locally available. A test deployment showed in one study that crop yields by 48%, where-as people without the book suffered a 5% drop in crop yield. (In that case the information wasn't complicated, it was planting the crop in little hills, with a moat around each plant, to maximize the benefit of rain).

Noirin mentioned the apache httpd webserver. This software powers millions of websites around the world. This is an example of how open source is a common good. 

lca2011 - silver lining

| No Comments | No TrackBacks
Pia Waugh is talking about the silve lining of cloud computing. This is a bit of an anti-cloud rant. It's based on her blog post. She was in a group of cloud vendors chatting, and someone made the claim that 'no privacy data will be stored in the cloud'.

She highlighted a lot of issues that she knows that clients (government departments, businesses) need to consider before jumping on the cloud bandwagon, but perhaps don't think about up front (like being able to extract your data out from the cloud if you want to move on to another vendor). Privacy off the data, jurisdiction of things. The blog post I mentioned above is a good overview of what she talked about.

lca2011 - deltacloud

| No Comments | No TrackBacks
David Jorm and Stephen Gordon spoke about deltacloud, a project Redhat is running to provide an abstraction of managing cloud instances, which comes with adapters to different cloud providers (such as amazon, redhats cloud services).

deltacloud provides a REST API. It is implemented in Ruby on Rails. They have confirmed backward compatibility across API versions. They support Amazon EC2, GoGrid, OpenNebula, Rackspace, etc. They are trying to support all the cloud providers, so you can move your infrastructure around.

The Aeolus project is an umbrella project for a bunch of things. I'm not sure how they relate to each other.

Their demo started off with firing a browser at the REST API. We saw some instances being started and stopped, and whether or not instances were running or not. 

LCA2011 - devops by Lindsay Holmwood

| No Comments | No TrackBacks
I've decided to start of with the 'Freedom in the cloud' mini-conf. First talk is by Lindsay Holmwood, talking about how developers and operations staff are working more closely together. He is talking about a case study of a high profile fundraising site, that runs during november and requires 100% uptime.

To get consistency they using puppet for configuration management. In his case case study they had about 130+ manifests, so they had a lot of complexity. They had a problem with configuration drift, it becomes complicated to get the right configuration across it. They developed 'roles' (looked like puppet defines) that you pass in some parameters. So instead of a node having a list of classes, or inheritance each node just uses one define.

They use capistrano to trigger stuff, not replace puppet, and for deploying applications. He described capistrano as a Ruby DSL around ssh-in-a-for loop.

His metric tool of choice is collectd. It's plugin based (has 100 plugins by default). Is network aware, so can collect locally and forward onto other places. Has a well defined API and has bindings in perl and python. 

He talked about his environment, some problems he had, etc. He spoke very fast, and a lot of the talk was of a fairly general nature. He talked about such a broad range of issues that he simply didn't have time to go into a great deal of detail.

kicking off day one of lca2011

| No Comments | No TrackBacks
I've registered, and am now sitting in the lecture hall for the opening keynote. So far the conference team have done a great job of relocating the conference to the Kelvin Grove campus of QUT, after the Garden Point campus was flooded a week and a half ago. I'm still not sure which talks I want to go to today, as there isn't a clear mini-conf that suit me. Tomorrow there is a sysadmin mini-conf, so that will be more straight forward.

black swan

| No Comments | No TrackBacks
Just saw Black Swan. It was an incredible movie, but very dark, quite disturbing in places. Natalie Portman did a great job. A friend had warned me that Aronofsky makes very dark films, and he was spot on. But the intensity of the dance, the pressure of the performance, and how we're never quite sure when Nina is hallucinating or not makes for a powerful visceral experience.

About this Archive

This page is an archive of entries from January 2011 listed from newest to oldest.

December 2010 is the previous archive.

February 2011 is the next archive.

Find recent content on the main index or look in the archives to find all content.



Powered by Movable Type 4.35-en