TeamCity has two interesting features: pre-tested commit and remote run. Remote run feature allows you to send your changes to TeamCity server and run a build with them without performing actual commit to a version control system. Pre-tested commit feature just adds automated commit to version control system if build was successful. In TeamCity such builds are called personal builds. There are some things that user of these features should be aware of.
Under the hood
First of all TeamCity currently does not allow you to run personal builds with your changes only. When a build starts your changes will always be integrated with current trunk. Sometimes this causes build failures when you and other developers modified the same files, but your colleague has already committed his changes. However from my experience this rarely causes problems.
Second you can safely modify your files while personal build is running. You can even modify same files that you just sent to TeamCity via remote run. Automated commit will check-in those versions of files that were sent to TeamCity when your personal build started, not the last changes you've made. Moreover starting from TeamCity 3.0 you can manually check-in these versions when personal build finishes. This works because before starting a personal build content of modified files corresponding to this build stored on disk elsewhere.
Third, automated commit actually is performed from IDE. So if you start a personal build with automated commit and then close IDE the commit will not be done until you start your IDE again.
Returning to practical usage...
So when these features are useful? Below I tried to summarize our practical experience of working with these features.
Personal builds are useful when you have a lot of functional tests (which are usually slow), or when you need to run your changes on several platforms with different software. In other words, they are useful when you are not able to run full suite of tests on your developer PC. With help of TeamCity build agents you can run your personal builds on different platforms in parallel and thus save a lot of time.
Personal builds are very useful for testing your code during refactoring. Often refactorings can be split in parts. For example, you've rewritten some component, your code compiles again and you feel that change should work fine. This is a good time for personal build. Usually in this case I am starting a personal build and continue my refactoring which could change the same component again. TeamCity IDE plugin will notify me when build finishes or when a first failure occures in the build (I prefer to have such notification for builds with my changes). And do not forget that TeamCity works hard to provide me with information about failures at the same time when they occur, so if I've broken something I'll soon be informed about that. If results are looking good I may decide to commit changes that were run in this personal build. Personal builds allow you to keep good quality in your code base during long refactoring tasks.
Not all of the functionality can be covered with tests. Sometimes you need to test your fix manually. Or you need to send a patched version of your distribution to a client. Personal builds can be useful here in case if your distribution cannot be created on your PC. For example, TeamCity contains Java / .NET and C++ code. I work mostly on Java components and I do not have Visual Studio installed so I cannot easily build a TeamCity distribution on my PC. In this case I would prefer to start a personal build. There were several cases when a personal build of TeamCity was installed to our internal server (where all Jetbrains projects are built) to check whether performance improvement or other critical fix we've made actually works.
Usually project build scripts and make files should be modified very carefully since a little error in these files can brake all of your builds. Personal builds are good choice when you need to check modifications in such files.
TeamCity does not have functionality explicitly designed for code review, but it is still possible to do a pre-commit code review with help of personal builds. For example, a developer could send you a link to a successfully finished personal build. First of all build results can tell you much about the change. Second you can review changes with help of TeamCity diff window and then make a final decision about it.
For those who never used TeamCity I hope this summary helps you better understand how personal builds work and in which cases they can be useful. If you are using TeamCity for day-to-day work feel free to share your experience with these features.
Thursday, March 6, 2008
Sunday, March 2, 2008
Puzzle Answer
See the previous post for the puzzle. Here comes the answer.
So, what's the strange part in the screenshot? A-ha, you must've noticed that the build number of the server (the one in the footer of the page) is the same as... one of the running builds! How that can be? Is it a fake? Is it a bug?
Nope, thanks to the two TeamCity features: during-the-build artifacts publishing and not loosing the running builds on server restart.
One of the features of TeamCity that was present since 1.0, but is not always noticed at first glance is on-the-fly tests reporting. With it you do not have to wait till the build finishes to know what tests have passed and what have failed - you simply get the information in the real time. (The feature on its own deserves a separate post or two.)
Getting used to this real-time support users kept asking for more: why not get an artifact as soon as it is ready, not waiting for the build to finish? There were some speculations on how this can be implemented: Should we watch for the files and start uploading them as soon as they are ready? But how do we know the files are ready and written fully? Under Windows, it seems we can detect file is open for writing, but what about other platforms?
For now we ended up with a simple but working solution: build script writer should notify TeamCity that the artifacts are ready to be published.
In 3.1 we introduced TeamCity "service messages": specially formatted strings that, when printed during the build, will be detected and parsed by TeamCity to perform an action. And one of messages implemented in the first place was publishArtifacts. You just print ##teamcity[publishArtifacts 'myValuedFile.zip'] into standard output and voilà: myValuedFile.zip is being uploaded to the server in parallel to the main build process. Once we deployed the build with the feature to our TeamCity server we configured our main TeamCity build to upload installers as soon as they are ready (we are creating packages before running the tests: that proved to be useful on a number of occasions).
That's the first part of the story: we can get TeamCity distribution package while the build is still on its way. What we do next is just upgrade the server (shutdown, place the new application files, startup). And here comes another great feature: the server sate is fully restored as if there were no server restart at all. This is way more easy to describe then it was to implement, but without the feature each server restart would be a pain, which we certainly strive to eliminate.
So, here we are: a TeamCity build that builds itself. Isn't it cool? :)
So, what's the strange part in the screenshot? A-ha, you must've noticed that the build number of the server (the one in the footer of the page) is the same as... one of the running builds! How that can be? Is it a fake? Is it a bug?
Nope, thanks to the two TeamCity features: during-the-build artifacts publishing and not loosing the running builds on server restart.
One of the features of TeamCity that was present since 1.0, but is not always noticed at first glance is on-the-fly tests reporting. With it you do not have to wait till the build finishes to know what tests have passed and what have failed - you simply get the information in the real time. (The feature on its own deserves a separate post or two.)
Getting used to this real-time support users kept asking for more: why not get an artifact as soon as it is ready, not waiting for the build to finish? There were some speculations on how this can be implemented: Should we watch for the files and start uploading them as soon as they are ready? But how do we know the files are ready and written fully? Under Windows, it seems we can detect file is open for writing, but what about other platforms?
For now we ended up with a simple but working solution: build script writer should notify TeamCity that the artifacts are ready to be published.
In 3.1 we introduced TeamCity "service messages": specially formatted strings that, when printed during the build, will be detected and parsed by TeamCity to perform an action. And one of messages implemented in the first place was publishArtifacts. You just print ##teamcity[publishArtifacts 'myValuedFile.zip'] into standard output and voilà: myValuedFile.zip is being uploaded to the server in parallel to the main build process. Once we deployed the build with the feature to our TeamCity server we configured our main TeamCity build to upload installers as soon as they are ready (we are creating packages before running the tests: that proved to be useful on a number of occasions).
That's the first part of the story: we can get TeamCity distribution package while the build is still on its way. What we do next is just upgrade the server (shutdown, place the new application files, startup). And here comes another great feature: the server sate is fully restored as if there were no server restart at all. This is way more easy to describe then it was to implement, but without the feature each server restart would be a pain, which we certainly strive to eliminate.
So, here we are: a TeamCity build that builds itself. Isn't it cool? :)
Sunday, February 24, 2008
New feature: search tests by name
The upcoming version of TeamCity 3.1 will contain a possibility to search tests with a particular name in build results. It allows to find out if a particular test has been run in the build. If you have a project with several thousand tests, using a filter is much more convenient than loading the full test list.
Right now, you can see ordinal number of the test, its status, and duration (with possibility to view a graph of the test duration history). All this is described in the corresponding part of the TeamCity UI documentation.
In the future, we plan to add a possibility to view the output of a particular test right from this interface (watch/vote for TW-3553).
Saturday, February 23, 2008
Incidental Field Test: Email Notifications Necessity
Yesterday we noticed an error sending email notification in the logs of our internal TeamCity installation. A quick investigation revealed that the server email sending settings are misconfigured. What was more interesting (and bitter) is that the settings were spoiled 2 weeks ago. Two weeks. All that time nobody was receiving email notifications from our internal TeamCity server! That might seem a disaster, but wait: why we had no complaints about it?
Of course we have other notifiers like Jabber, Windows tray, IDE and RSS, but still email should be the most popular. So why?
My guess is that advanced TeamCity users (who JetBrainers certainly are) do not want to be notified, they want to monitor. Web UI gives you a continuous, real-time feedback on what is going on. There are the whole bunch of additional data like the changes, commits, build running steps, own changes status and more. People may open TeamCity's web in the morning and leave it open for the whole day.
So most people either turned off the notifications at all or just did not notice them missing.
I remember our CruiseControl days when almost the only interface to the server was email. Web UI did not actually produce much more information and after all it was unbearably slow because of the build running on the same machine. Breaking email notifications in those days would mean no CI results - everyone would notice the problem in a couple of hours (or minutes, depending on the length of the builds). But not now. Things change.
Good notifications should not be underestimated. And we do improve notifications and will continue to do so. But who knows, with the tools advances, in a couple of years we may look down on email notifications like we do now on snail mail.
Getting down to earth, what we can do to prevent silly misconfiguration mistakes from going unnoticed? A possible solution is to display a warning to administrators in the web UI if any alike server error occur. Filed as TW-4552.
Of course we have other notifiers like Jabber, Windows tray, IDE and RSS, but still email should be the most popular. So why?
My guess is that advanced TeamCity users (who JetBrainers certainly are) do not want to be notified, they want to monitor. Web UI gives you a continuous, real-time feedback on what is going on. There are the whole bunch of additional data like the changes, commits, build running steps, own changes status and more. People may open TeamCity's web in the morning and leave it open for the whole day.
So most people either turned off the notifications at all or just did not notice them missing.
I remember our CruiseControl days when almost the only interface to the server was email. Web UI did not actually produce much more information and after all it was unbearably slow because of the build running on the same machine. Breaking email notifications in those days would mean no CI results - everyone would notice the problem in a couple of hours (or minutes, depending on the length of the builds). But not now. Things change.
Good notifications should not be underestimated. And we do improve notifications and will continue to do so. But who knows, with the tools advances, in a couple of years we may look down on email notifications like we do now on snail mail.
Getting down to earth, what we can do to prevent silly misconfiguration mistakes from going unnoticed? A possible solution is to display a warning to administrators in the web UI if any alike server error occur. Filed as TW-4552.
Thursday, February 21, 2008
500 build hours per day
You might not know but here at Jetbrains we are using our own TeamCity server to build almost all of our projects (including such giants as IntelliJ IDEA and Resharper). This is a part of our company strategy proudly named "eat your own dog food". And it works quite well.
Big projects usually require much time to build and run tests, for example, to compile IntelliJ IDEA sources you will need about 16min on a relatively modern PC. While to run a full test suite you'll need almost two hours. The same picture is in Resharper project: the whole build with compilation and tests takes more than 2 hours to complete. Given such a long builds it is critical to be able to run them in parallel. That is why TeamCity has such feature as build grid from the beginning, otherwise we won't be able to use TeamCity at all.
But how much agents will be enough for us? Well, when we released TeamCity 1.0 in October 2006 we had a pool of about 20 agents:
Things did not change much when we released 2.0 version in April 2007 (except that now we had a so-called "glass" indicating current server load :)
Then as more Jetbrains projects moved to TeamCity we started to realize that we need more power. In December 2007 when TeamCity 3.0 released we had more than 30 agents:
Interestingly even with 33 agents we have almost as many builds waiting in the queue :)
So did we stop? Nope! Thanks to our selfless administrators we increased our pool again:
Now we are able to build more than 700 builds per day (all of them took >500 hours, or about 20 build days) and publish more than 10Gb of artifacts. I think it is serious power. BTW our server is installed on a PC with dual core CPU 3.2GHz / 4Gb RAM of that TeamCity uses 750Mb only.
Do you think so much agents will be enough for us? Will see :)
Big projects usually require much time to build and run tests, for example, to compile IntelliJ IDEA sources you will need about 16min on a relatively modern PC. While to run a full test suite you'll need almost two hours. The same picture is in Resharper project: the whole build with compilation and tests takes more than 2 hours to complete. Given such a long builds it is critical to be able to run them in parallel. That is why TeamCity has such feature as build grid from the beginning, otherwise we won't be able to use TeamCity at all.
But how much agents will be enough for us? Well, when we released TeamCity 1.0 in October 2006 we had a pool of about 20 agents:
Things did not change much when we released 2.0 version in April 2007 (except that now we had a so-called "glass" indicating current server load :)
Then as more Jetbrains projects moved to TeamCity we started to realize that we need more power. In December 2007 when TeamCity 3.0 released we had more than 30 agents:
Interestingly even with 33 agents we have almost as many builds waiting in the queue :)
So did we stop? Nope! Thanks to our selfless administrators we increased our pool again:
Now we are able to build more than 700 builds per day (all of them took >500 hours, or about 20 build days) and publish more than 10Gb of artifacts. I think it is serious power. BTW our server is installed on a PC with dual core CPU 3.2GHz / 4Gb RAM of that TeamCity uses 750Mb only.
Do you think so much agents will be enough for us? Will see :)
Subscribe to:
Posts (Atom)
