Thursday, February 21, 2008

500 build hours per day

You might not know but here at Jetbrains we are using our own TeamCity server to build almost all of our projects (including such giants as IntelliJ IDEA and Resharper). This is a part of our company strategy proudly named "eat your own dog food". And it works quite well.

Big projects usually require much time to build and run tests, for example, to compile IntelliJ IDEA sources you will need about 16min on a relatively modern PC. While to run a full test suite you'll need almost two hours. The same picture is in Resharper project: the whole build with compilation and tests takes more than 2 hours to complete. Given such a long builds it is critical to be able to run them in parallel. That is why TeamCity has such feature as build grid from the beginning, otherwise we won't be able to use TeamCity at all.

But how much agents will be enough for us? Well, when we released TeamCity 1.0 in October 2006 we had a pool of about 20 agents:

Things did not change much when we released 2.0 version in April 2007 (except that now we had a so-called "glass" indicating current server load :)

Then as more Jetbrains projects moved to TeamCity we started to realize that we need more power. In December 2007 when TeamCity 3.0 released we had more than 30 agents:

Interestingly even with 33 agents we have almost as many builds waiting in the queue :)

So did we stop? Nope! Thanks to our selfless administrators we increased our pool again:

Now we are able to build more than 700 builds per day (all of them took >500 hours, or about 20 build days) and publish more than 10Gb of artifacts. I think it is serious power. BTW our server is installed on a PC with dual core CPU 3.2GHz / 4Gb RAM of that TeamCity uses 750Mb only.

Do you think so much agents will be enough for us? Will see :)

1 comment:

Sergey said...

I would add one more important fact. The server load is still quite low despite those 50 agents continuously doing their jobs. :)