TeamCity NUnit support features

TeamCity supports .NET as well as Java. For .NET there is couple of build runners. To run a build it is possible to use NAnt, MSBuild, Sln2003, Sln2005, Sln2008 build runner.

Actually, there is no matter if you use NUnit or something else. TeamCity is able to track NUnit tests and report all tests information on-the-fly. For that feature you do not need to change the build.

TeamCity automatically replaces NUnit calls from NAnt with <nunit2> task or MSBuild with <nunit> task from MSBuild Community tasks. For Visual Studio solutions there are UI options to enable unit tests. In anyway if you want to run NUnit tests you may use mentioned tasks in you build scripts or even call an .exe to run the tests.

TeamCity supports NUnit 2.2.10, 2.4.1, 2.4.6, 2.4.7. Actually, there is really a small difference what NUnit test runner to use. There is no strong dependency between NUnit test runner and NUnit.Framework.dll that is linked to the application.

Tests may require to be started under exact NET 1.1 (or .NET 2.0). In case .NET 2.0 there is an option x64 or x86. Under TeamCity it is easy to specify it.

For NAnt you have to specify target framework to .NET 1.1 and that would start your tests under .NET 1.1

If you use command line for your tests it would look like this:

$(teamcity.dotnet.nunitlauncher) v1.1 MSIL NUnit-2.4.7 <list of assemblies>

To select .NET 2.0 x86 or .NET 2.0 x64. For NAnt you will have to define property named 'teamcity.dotnet.nant.nunit2.platform'. Values could be x64,x86,MSIL. For command line runner it would look like: 

  $(teamcity.dotnet.nunitlauncher) v2.0 x86 NUnit-2.4.7 <list of assemblies>

All mentioned settings are available on Sln2005/Sln2008 runners from TeamCity Web interface.

For MSBuild TeamCity defines task called NUnitTeamCity. It is possible to define any behavior of the tests under TeamCity. Using Platform attribute it is easy to select platform (x64 or x86) to run your tests. NUnitVersion attribute allows to select NUnit test runner version.

And finally, if some test has failed you will be notified at the exact time of that failure. You do not need to wait for all tests to finish.

TeamCity Plugins Repository is Emerging

We are starting to build TeamCity plugin repository. Well, "repository" is too big a word for it at this time, but more entries are under way.

The idea is not only to list a plugin, but provide plugin documentation space, host plugin repository and bug tracker, and (the most interesting part) to build the plugin against the latest released/current trunk TeamCity versions.

We've set this up for several our plugins (build with TeamCity trunk version is a pending task), and plan to provide the same services to the third-party plugins.

So if you want to write a plugin, or already have one that can be of interest to other users, just let us know.

Getting test results faster

Have you ever paid attention on how fast you can get results of building your changes from TeamCity? I mean if a test failed you do not need to wait while the whole build finishes, TeamCity will notify you immediately. You can say "So what? It's quite natural to expect this feature from an automated build system". I agree, but show me any other continuous integration server which offers similar functionality. Most of them (if not all) require you to specify paths to test results files (usually in JUnit format) and in turn it means that you have to wait while build finishes (what if tests in your build run 40 minutes?). Not to mention that sometimes you have to modify your build script to produce test results in required format.

This is a great feature and we are going to improve it further. First of all we are going to force your build to run recently failed tests before any other tests. This should help you if you are fixing broken tests. TeamCity will try to do this transparently, but in some cases (for example, if you have your own test runner) it can't be done. For these cases we'll provide list of tests via property or via file and you will be able to modify your test runner to support this feature too.

At the moment this feature is already working on our internal server. Currently it is supported for Ant and IPR runners and for JUnit and TestNG frameworks. We hope to add support for other frameworks and runners before the first EAP.

A further improvement would be to run new and modified tests first. This should be especially useful for personal builds. Since TeamCity tracks changes for every build we can determine whether modified files are test cases or not. We do not have a working code for this feature yet but we hope to add it to the first EAP too.

By the way we are going to release first EAP of Calcutta (TeamCity 4.0 codename) at the next week. I hope you'll find time to install it and try these new features.

The fools day TeamCity plugin

We've worked hard to create a new very cool plugin for you (even the 3.1.1 release had not taken so much efforts ;). If you have administrative rights on your TeamCity server then you have unique ability to make fun of your colleagues! Just install this plugin and enable it on the server configuration page. I do not want to describe its abilities here, all I can tell is that "jokes mode" is enabled when browser losts focus... Also if you understand JavaScript take a look at joke.js file, it contains some notes about plugin abilities.

Feel free to extend and improve it :)

Personal builds: practical experience

TeamCity has two interesting features: pre-tested commit and remote run. Remote run feature allows you to send your changes to TeamCity server and run a build with them without performing actual commit to a version control system. Pre-tested commit feature just adds automated commit to version control system if build was successful. In TeamCity such builds are called personal builds. There are some things that user of these features should be aware of.

Under the hood

First of all TeamCity currently does not allow you to run personal builds with your changes only. When a build starts your changes will always be integrated with current trunk. Sometimes this causes build failures when you and other developers modified the same files, but your colleague has already committed his changes. However from my experience this rarely causes problems.

Second you can safely modify your files while personal build is running. You can even modify same files that you just sent to TeamCity via remote run. Automated commit will check-in those versions of files that were sent to TeamCity when your personal build started, not the last changes you've made. Moreover starting from TeamCity 3.0 you can manually check-in these versions when personal build finishes. This works because before starting a personal build content of modified files corresponding to this build stored on disk elsewhere.

Third, automated commit actually is performed from IDE. So if you start a personal build with automated commit and then close IDE the commit will not be done until you start your IDE again.

Returning to practical usage...

So when these features are useful? Below I tried to summarize our practical experience of working with these features.

Personal builds are useful when you have a lot of functional tests (which are usually slow), or when you need to run your changes on several platforms with different software. In other words, they are useful when you are not able to run full suite of tests on your developer PC. With help of TeamCity build agents you can run your personal builds on different platforms in parallel and thus save a lot of time.

Personal builds are very useful for testing your code during refactoring. Often refactorings can be split in parts. For example, you've rewritten some component, your code compiles again and you feel that change should work fine. This is a good time for personal build. Usually in this case I am starting a personal build and continue my refactoring which could change the same component again. TeamCity IDE plugin will notify me when build finishes or when a first failure occures in the build (I prefer to have such notification for builds with my changes). And do not forget that TeamCity works hard to provide me with information about failures at the same time when they occur, so if I've broken something I'll soon be informed about that. If results are looking good I may decide to commit changes that were run in this personal build. Personal builds allow you to keep good quality in your code base during long refactoring tasks.

Not all of the functionality can be covered with tests. Sometimes you need to test your fix manually. Or you need to send a patched version of your distribution to a client. Personal builds can be useful here in case if your distribution cannot be created on your PC. For example, TeamCity contains Java / .NET and C++ code. I work mostly on Java components and I do not have Visual Studio installed so I cannot easily build a TeamCity distribution on my PC. In this case I would prefer to start a personal build. There were several cases when a personal build of TeamCity was installed to our internal server (where all Jetbrains projects are built) to check whether performance improvement or other critical fix we've made actually works.

Usually project build scripts and make files should be modified very carefully since a little error in these files can brake all of your builds. Personal builds are good choice when you need to check modifications in such files.

TeamCity does not have functionality explicitly designed for code review, but it is still possible to do a pre-commit code review with help of personal builds. For example, a developer could send you a link to a successfully finished personal build. First of all build results can tell you much about the change. Second you can review changes with help of TeamCity diff window and then make a final decision about it.

For those who never used TeamCity I hope this summary helps you better understand how personal builds work and in which cases they can be useful. If you are using TeamCity for day-to-day work feel free to share your experience with these features.

Puzzle Answer

See the previous post for the puzzle. Here comes the answer.

So, what's the strange part in the screenshot? A-ha, you must've noticed that the build number of the server (the one in the footer of the page) is the same as... one of the running builds! How that can be? Is it a fake? Is it a bug?

Nope, thanks to the two TeamCity features: during-the-build artifacts publishing and not loosing the running builds on server restart.


One of the features of TeamCity that was present since 1.0, but is not always noticed at first glance is on-the-fly tests reporting. With it you do not have to wait till the build finishes to know what tests have passed and what have failed - you simply get the information in the real time. (The feature on its own deserves a separate post or two.)

Getting used to this real-time support users kept asking for more: why not get an artifact as soon as it is ready, not waiting for the build to finish? There were some speculations on how this can be implemented: Should we watch for the files and start uploading them as soon as they are ready? But how do we know the files are ready and written fully? Under Windows, it seems we can detect file is open for writing, but what about other platforms?
For now we ended up with a simple but working solution: build script writer should notify TeamCity that the artifacts are ready to be published.

In 3.1 we introduced TeamCity "service messages": specially formatted strings that, when printed during the build, will be detected and parsed by TeamCity to perform an action. And one of messages implemented in the first place was publishArtifacts. You just print ##teamcity[publishArtifacts 'myValuedFile.zip'] into standard output and voilà: myValuedFile.zip is being uploaded to the server in parallel to the main build process. Once we deployed the build with the feature to our TeamCity server we configured our main TeamCity build to upload installers as soon as they are ready (we are creating packages before running the tests: that proved to be useful on a number of occasions).

That's the first part of the story: we can get TeamCity distribution package while the build is still on its way. What we do next is just upgrade the server (shutdown, place the new application files, startup). And here comes another great feature: the server sate is fully restored as if there were no server restart at all. This is way more easy to describe then it was to implement, but without the feature each server restart would be a pain, which we certainly strive to eliminate.

So, here we are: a TeamCity build that builds itself. Isn't it cool? :)

A Puzzle

Here is a real screenshot. Can you spot what's not usual in it?



Depicted is our JetBrains TeamCity installation with Benares3.1::MasterBuild build configuration which is the main build for TeamCity 3.1 branch: it creates installers and runs a set of tests.

See the next post for the answer.