Why do Twitter clients use Adobe AIR?

21/06/2009 – 12:56 pm

@tkramar raised the question on Twitter, why every desktop client for that service seems to be built on top of Adobe Air (here). That’s something, I also was wondering about. And in fact you often hear people complain: “Why AIR? Why not a native application?” Let me try to give the answer while having breakfast. ;-)

Read the rest of this entry »

Managing Web 2.0

10/06/2009 – 06:23 pm

Somehow, all the features and increasingly popular services become more and more confusing.

Ok, everyone has Facebook (at least in English speaking countries people would rather ask if you are on Facebook than what your phone number is). And then there’s Twitter, another way of keeping friends or customers or whoever informed about what’s going on. Plus, you need to share you favourite URLs with digg, del.icio.us, or any other social bookmarking service of your choice. And of course, you need your own blog! And a Flickr account for your photos. Did I cover all of the services a modern Web 2.0 person has to have? Probably not. But now the core question: How do you manage to keep everything up 2 date? I mean as someone who’s already got a full-time job… :D

Comments, suggestions, URL appreciated!

Octopus — Logging Server in C#

06/06/2009 – 07:21 pm

I have to admit that I really like C#. The last 4 or 5 months, I’ve hardly touched any other programming language. And so it happened to be my first choice when I was asked to develop a “reasonably fast” logging server.

So what is this Octopus thing about? In a nutshell it’s a server, which takes HTTP requests with a defined set of URL parameters, confirms receipt the client, and then asynchronously stores the request into a database. The interesting bit is the part between accepting the request and storing it to the DB. 

Basically, we’ve got three levels of storage: a in-memory queue, a flat file buffer, and the database. The tricky thing was to optimise processing, while still being fault-tolerant. I kind of managed that, I think. Octopus can (depending on the hardware it runs on) easily accept up to 2,000 requests per second, and will store them into either a MySQL or SQLite database at the moment. Please read more about it on the Octopus Website.

Waking up from hibernation

26/09/2008 – 12:01 am

After years of Systems Administration I have completely forgotten how much I used to like developing in Java. My current employer gave me that opportunity recently. So I took ownership and responsibility to de-hibernate myself (my Java knowledge became a bit rusty) and to start working on a project which separates the website’s business logic layer from the database layer. That becomes necessary as we want to re-design our infrastructure in a way that enables us to be as platform independent and as scalable as possible.  

On the one hand, we are not even entirely sure if we’ll stick to MySQL or switch to any other DB server type, hence we must not use SQL dialect specific code within the application. On the other hand, we want to provide a SOAP endpoint internally, which allows us to access data from all sorts of applications and clearly separates the business logic of all sorts of clients from our actual data store.

In addition, caching of object structures which are mapped against data structures (or tables) would be very neat. Consequently, I suggested and started using Java with Hibernate and JAX-WS on a Glassfish Java Application Server. We might integrate an additional caching layer later. Terracotta is one of the candidates, which integrates seamlessly and can increase speed by factor 2 to 10. 

After one week of researching, developing proofs of concept and presenting a first prototype, I am sooo excited about all that. :-)  I always loved Java, and again I remember why.

PowerDNS w/ MySQL — should have tried earlier

14/09/2008 – 10:23 pm

I was quite fed up with BIND (zonefile editing, serial number increasing, master/slave setups), so I was searching for alternatives to run Master/Slave DNS services. More or less randomly, I again came across PowerDNS. Again, because I have heard a couple of times that big providers quite often use it. So I decided to give it a go. On FreeBSD it’s quite easy. You just install it together with its MySQL modules, activate the modules in the configuration files, import the database schemata, and fill it with content. A script to import BIND zone files directly to the DB is also included. Couldn’t be easier. 

Once you’ve set it up and got it running, you of course need a slave server as well. So all you got to do is either to install another PowerDNS/MySQL instance on another machine and connect it to the same database (you wouldn’t really want to do that), or connect it to a second MySQL server, which ideally is a replication slave of the first one. Then you have a fully redundant setup and don’t need to worry about failures of either of those. All DNS record changes go directly to the master DB server, which will be picked up by the slave in almost real time. Plus, you don’t need to worry about serial numbers. Change a record in the DB and this change will be published instantly. 

If you fancy web interfaces, you can either go for an open source one, or you can write one on your own. As the DB structure of PowerDNS is kept very simple, it’s easy to add/modify records via script also. Do whatever you want and don’t worry about the stupid restrictions and configuration hurdles you used to run into when going for a BIND server.

Distributed Data Centre in-a-box

16/08/2008 – 01:02 am

Weird title, right? Well, it’s also a weird project, which I am doing just for fun and as a kind of proof-of-concept. The idea is to build a “micro data centre” on a single VMware ESXi based machine and to fully replicate all parts of it onto another single ESXi based machine, which is located in another real data centre. That sounds easy, right? But hold on a second.

Read the rest of this entry »

Why should you bother to use virtualisation?

10/08/2008 – 11:35 am

Be honest: You are running at least one dedicated server, and you certainly have asked yourself whether you should use virtualisation. You might have found “no” to be the answer, as you have one server for each purpose and do not plan to migrate to other hardware machines or to “sub let” your system. That’s ok. But on the other hand, everybody likes to reduce hardware costs, or make more of the hardware they have.

It’s a prejudice that virtualisation is only interesting for so-called Virtual Private Server providers or for big companies who need to run loads of tests for their software releases on different platforms and configurations. Also, you do not need a bunch of servers or a blade-center to take an advantage on virtualisation. In this article I’d like to give an example of what can be achieved with virtualisation apart from those typical and well-known scenarios.

Read the rest of this entry »

Free Virtualisation at its best!

09/08/2008 – 11:47 pm

Did you realise that VMware released ESXi for nothing recently? If not, get it and try it! All you need is a server with full KVM access (either over IP or locally) including the opportunity to install CD images (remotely). In my case, I went for a neat SuperMicro machine from SoftLayer, which offers full KVM-over-IP with remote CD mounts (ISO images on network storages or on the client computer). But that’s not important here, although they actually offer Double-RAM/Double-Drive deals, so that I am paying only US$ 311 for a Quad-Core Xeon, 6GB of RAM and 2×146 GB SAS at an Adaptec RAID controller. It’s an awesome offer, especially because they have an incredible user interface and are extremely flexible when it comes to updates/changes. Okay, okay. Enough advertising :-)

This article is about virtualisation. Caution: Long article!

Read the rest of this entry »

ZFS on FreeBSD 7 — experimental for a reason!

28/05/2008 – 07:04 pm

Yes, it has a reason that ZFS is not yet marked stable on FreeBSD! I had to learn the hard lesson today.

Under very high load and many concurrent read requests (I set up the company’s mail server with ZFS and root from ZFS), the two disks in the Raid array repeatedly lost sync, forcing an automatic re-silvering (auto healing) process to be started, which blocked the system as everything (except /boot) was running from that ZFS arrray. As far as I figured out, the system halted entirely as there was another inconsistency occuring while the re-silvering was still in progress.

I would have investigated further, if it wasn’t a crucial production machine. And that kind of traffic is very difficult to simulate under laboratory situations (maybe I can do that when I have more time). So I had to revert back to UFS as the downtime had to be minimized. It’s a shame, really, because I love the features ZFS offers. On my private server it runs very smoothly, but traffic, load and I/O are not comparable to the mail server in question.

SPF — Sender Policy Framework

25/05/2008 – 04:31 pm

Did you ever receive spam mails which seem to originate from your own mail address? Or did anybody else complain about you sending those emails? Then you should take a look at SPF. In a nutshell, it plugs the holes in the SMTP protocoll, which does not allow to verify if a sender (or anybody who pretends to be that sender) really may use a particular mail server to transmit their mails. Unfortunately, SPF is not yet very wide-spread. Almost everybody (including me) has come across this abbreviation and/or heard that it might protect misuse of mail addresses. But most people (including me) cannot be bothered to implement it.

Read the rest of this entry »