4.0.1 EAP, build 8111

Here is a new build to open the EAP program for 4.0.1 bug fix update.

Search results highlighting, build queue and notification calculation speedups, possible (but rare) deadlock fix - these are some of the changes in the build.

Currently we do not make any major changes focusing on important or small changes in the code, so the build should be quite stable.

BTW, did you notice what's special in the build number?

And the winner is...

Build number 8080!
Congratulations, you've just been promoted to the official TeamCity 4.0 release build.

The most advanced continuous integration for Ruby on Rails with TeamCity 4.0

TeamCity 4.0 has a dedicated rake runner, with support for Rake tasks, Test::Unit, and RSpec tests.

Our special thanks goes to the guys behind RubyMine and Ruby plugin for IntelliJ IDEA, who created this runner for TeamCity.

Here is a short (3.5 min, 7.3Mb) demo of some unique TeamCity's features and how they work for a Ruby on Rails project.

This demo includes:

• setting up Rake runner build configuration
• running a build, with progress estimate for subsequent builds
• ongoing test reporting while build is running
• viewing changes in version control, which are included into the build (web diff)
• "first failed in"/"already fixed in" features for failed tests
• full sortable test list for a build
• a graph for test duration time

View demo

Read more about TeamCity features and what's new in 4.0 or download it now and see for yourself!

Regards,

KIR

A chance to try almost TeamCity 4.0 before the official release

We've published build 8018 which is separated from the official release build by only a couple of build number increments.

Don't miss your chance to try it and be fast to let us know if you spot any critical issues with it.

How ReSharper found a bug in IDEA with the help of TeamCity

A history of one bug.

Several days ago, I received a message from one of ReSharper developers:

The link opened our internal TeamCity installation with the ReSharper build failed on sources checkout. The error was produced by SVN unable to update a directory:

svn: Failed to add directory 'test/assemblies': a versioned directory of the same name already exists

After some digging it turned out that the directory was deleted from the repository some time ago but was not deleted form the build agent. Then it was added anew and the build failed trying to add a directory that already existed on disk. The workaround was to invoke "Enforce clean checkout" from the TeamCity UI for the build configuration.

So ReSharper team can continue their current development effort on performance optimization, but why the directory was not deleted in the first place?

TeamCity uses SVNKit library to talk to Subversion repositories and luckily the developer of the library happened to be just a room away these days. Introduced into the issue he reached for the code to check and soon found a bug in the library that might result in not deleted directory in the working copy. It could only reproduce under certain optimization options that only two products are actually using, one of them being TeamCity and the other ... IntelliJ IDEA. But wait, IDEA is preparing for the 8.0 release these days!

No worries. As I am writing this, the library is already fixed, the latest build downloaded from SVNKit TeamCity installation and as I've just checked on our TeamCity server, the fix is already checked into IDEA release branch.

Actually, the bug is a rare one since it can reproduce only under certain circumstances, it was introduced not so long ago and was only included into EAP releases of IDEA and TeamCity.

Calcutta EAP, build 7888

A new EAP build was published on Tuesday.

The main changes are described in our EAP space, but there were many others and I will point out some:

• Connected agents now display the time of last activity rather then time of registration on the server;
• Presentation of Test Details page has been improved;
• "Triggered by" field of the build now displays the build configuration name if triggered by same-sources dependency;
• Storing of duplicate code finder results has been improved. The results are now valid for all the builds. Previously, the duplicate results were only valid for the most recent build and older builds could show incorrect data;
• If you write a build script or tool and need to detect whether it is run under TeamCity or not, you can now use TEAMCITY_VERSION environment variable that holds TeamCity version as shown in the web UI footer;
• IVY library has been updated. We use it to resolve artifact dependencies on agents. The updated version properly handles spaces in the file names, at last;
• Logging of VCS-related errors into the TeamCity logs has been improved to be less verbose but note more details;
• Windows build agent installer no longer defines JAVA_HOME to point to bundled JRE. This will make the administrators make an explicit choice if their builds use JAVA;
• Build stopping logic has been improved under Windows to eliminate wrong process tree building when process ID was reused by the OS;

Grab you copy of the fixes on the download page.

As always, your comments are welcome!

Agents maintenance

If you read this blog from the beginning you probably know that in JetBrains we use quite a big farm of agents. The agents produce a large amount of work. According to our statistics usage of the most of our 45 agents is around 40-50% (there are no builds at night time).

Moreover some of our builds produce a large number of small files on disk. The files are then deleted by build itself or by agent but this increases fragmentation of the file system. Not surprisingly these builds started to run slower, and we decided that we need some kind of agents maintenance policy. Ideally we need to defragment agent disk and probably reboot agent once per week. Well, since we already have agents installed on these PCs why not do maintenance tasks with еру help of the agents themselves?

The solution

Note: we did not experience performance problems with Unix/Linux based agents so all the scripts below are for Windows only.

First of all we created a project in TeamCity and set necessary access rights. Authorization is important since maintenance tasks are potentially harmful and only experienced users should be able to configure and run them.

Then we created three build configurations:
1) Clean & Defragment
2) Reboot
3) Run all

"Clean & Defragment" build configuration removes temporary files from the user home directory and starts disk defragmentation. Currently it is Ant based, and here is the script we use:

<project name="AgentsMaintenance" default="defragment" basedir=".">

  <target name="defragment">
    <echo message="##teamcity[progressMessage 'Defragmenting ...']"/>
    <exec executable="defrag">
      <arg value="C:"/>
      <arg value="-v"/>
    </exec>
  </target>

  <property environment="env"/>
  <property name="userTemp" location="%env.USERPROFILE%/Local Settings/Temp"/>

  <target name="clean">
    <echo message="##teamcity[progressMessage 'Cleaning ${userTemp} directory...']"/>
    <delete quiet="yes" failonerror="false" includeemptydirs="true">
      <fileset dir="${userTemp}" includes="**/*"/>
    </delete>
    <mkdir dir="${userTemp}"/>
  </target>

</project>

"Reboot" build configuration reboots an agent. It uses command line runner and simply invokes the following command:

shutdown -r -f -t 20 -c "Reboot build agent"

And finally the main step is build configuration "Run all". Here is the Ant script:

<project name="AgentsMaintenance" default="runAll" basedir=".">

  <target name="runAll">
    <property name="user" value="username"/>
    <property name="pwd" value="password"/>

    <get src="http://teamcity.server.com/httpAuth/action.html?add2Queue=<clean&defrag conf id>&amp;agentId=allEnabledCompatible"
         dest="response.txt" verbose="yes" username="${user}" password="${pwd}"/>

    <get src="http://teamcity.server.com/httpAuth/action.html?add2Queue=<reboot conf id>&amp;agentId=allEnabledCompatible"
         dest="response.txt" verbose="yes" username="${user}" password="${pwd}"/>
  </target>
</project>

Where <clean&defrag conf id> and <reboot conf id> are identifiers of the corresponding build configurations (buildTypeId from the TeamCity URLs).

"Run all" build configuration is triggered by TeamCity scheduling trigger at Saturday night. The build of this configuration adds "Clean&Defragment" and "Reboot" builds in the queue. It uses not yet released feature of TeamCity allowing to start a build on all compatible and enabled agents (note agentId=allEnabledCompatible in the URL). So if you do not want "Reboot" builds to run on some agent just make this agent incompatible with "Reboot" configuration! Pretty simple, isn't it?

So far the solution works nice. And if your builds started to run slower try to setup something like this, probably the reason is not in the builds itself. BTW the new feature to run build on all compatible agents will be available in the next EAP build.