Feb
7
2012

Starting TeamCity builds from HipChat using Github Hubot

La table du diable depuis la Savane des Pétrifications

After writing about “Running your TeamCity builds from PowerShell for any Git branch” I’d like to talk about another integration which is using Github Hubot so that the build could be started directly form a chat room.

So using the same idea, which is to extend our development environment, we implemented a way to start TeamCity builds directly from our HipChat room. You could do the same from Campfire for sure.

The main idea of this is to have our daily tools right at our disposal where we spend a good part of our days, chat rooms.

First we had to install Github Hubot and we have chosen to use Heroku which offer free hosting for one of their web dyno. They even offer what they call Hubot Factory which is a way to allow people to easily and quickly deploy new instances of Hubot to Heroku. In fact we went with a manual installation described on this page “Hubot for HipChat on Heroku”.

The idea is still the same we need to run TeamCity builds by “Accessing the Server by HTTP”.

This time we want to have Hubot calling the TeamCity server so we need to add a new CoffeeScript to the scripts of Hubot which will launch the HTTP request needed.

Our script is heavily inspired from the TeamCity.coffee script, and here is a first version:

# Starts a build on TeamCity.
#
# You need to set the following variables:
# HUBOT_TEAMCITY_USERNAME = 
# HUBOT_TEAMCITY_PASSWORD = 
# HUBOT_TEAMCITY_HOSTNAME = 
#
# start build buildId -- Starts TeamCiyt build using buildId
module.exports = (robot) ->
  robot.respond /start build (.*)/i, (msg) ->
    username = process.env.HUBOT_TEAMCITY_USERNAME
    password = process.env.HUBOT_TEAMCITY_PASSWORD
    hostname = process.env.HUBOT_TEAMCITY_HOSTNAME
    buildName = msg.match[1]

    msg.http("http://#{hostname}/httpAuth/action.html?add2Queue=#{buildName}")
      .headers(Authorization: "Basic #{new Buffer("#{username}:#{password}").toString("base64")}",
               Accept: "application/json")
      .get() (err, res, body) ->
        if err
          msg.send "Team city says: #{err}"
          return

By the way I started to learn CoffeeScript so this is more hacking then something really productive.

And here is the result in our HipChat room asking hubot to start a build with the command:

hubot start build bt21

TeamCity, GitHub Hubot and HipChat

I will talk in another post about the last part which shows the status of the build in the chat room.

Feb
7
2012

Running your TeamCity builds from PowerShell for any Git branch

Plage le long de la Savane des Pétrifications

I love TeamCity and use it since a while to automate my build/release processes. As human we should never do the work a machine can do, we have certainly better and more interesting things to do.

The habit I saw in the different projects I worked for is to create new TeamCity builds for the branches you work on. It take quite some work to do, even with templates…

So I came with another way of doing it. It is leverage Git, PowerShell and the possibility to run TeamCity builds by “Accessing Server by HTTP”. Basically you just need to make a HTTP request to an Url like this:

http://testuser:testpassword@teamcity.jetbrains.com/httpAuth/action.html?add2Queue=bt10&system.name=<property name1>&system.value=<value1>&system.name=<property name2>&system.value=<value2>&env.name=<environment variable name1>&env.value=<environment variable value1>&env.name=<environment variable name2>&env.value=<environment variable value2>

We will also use the possibility to pass custom parameters (system properties and environment variables) through the HTTP request.

The parameter we want to pass is the Ref name of the Git VCS Root, so we start by setting it using a configuration parameter called BuildRefName, as this:

 

Git, PowerShell, TeamCity

Then we need to add a configuration parameters for our TeamCity project with the name BuildRefName which we currently set to master, so that if the parameter is not defined it will default to this value:

Git, PowerShell, TeamCity 2

Now that we have configured this you can already start a build from your browser by calling the URL!

http://testuser:testpassword@teamcity.jetbrains.com/httpAuth/action.html?add2Queue=bt10

Be sure to replace bt10 with your build id.

Nice ,but we want to get a step further and be able to start the build from PowerShell, which is quite easy!

To achieve this goal you add those two functions to your PowerShell profile:

function Get-Web($url, 
    [switch]$self,
    $credential, 
    $toFile,
    [switch]$bytes)
{
    #.Synopsis
    #    Downloads a file from the web
    #.Description
    #    Uses System.Net.Webclient (not the browser) to download data
    #    from the web.
    #.Parameter self
    #    Uses the default credentials when downloading that page (for downloading intranet pages)
    #.Parameter credential
    #    The credentials to use to download the web data
    #.Parameter url
    #    The page to download (e.g. www.msn.com)    
    #.Parameter toFile
    #    The file to save the web data to
    #.Parameter bytes
    #    Download the data as bytes   
    #.Example
    #    # Downloads www.live.com and outputs it as a string
    #    Get-Web http://www.live.com/
    #.Example
    #    # Downloads www.live.com and saves it to a file
    #    Get-Web http://wwww.msn.com/ -toFile www.msn.com.html
    $webclient = New-Object Net.Webclient
    if ($credential) {
        $webClient.Credentials = $credential
    }
    if ($self) {
        $webClient.UseDefaultCredentials = $true
    }
    if ($toFile) {
        if (-not "$toFile".Contains(":")) {
            $toFile = Join-Path $pwd $toFile
        }
        $webClient.DownloadFile($url, $toFile)
    } else {
        if ($bytes) {
            $webClient.DownloadData($url)
        } else {
            $webClient.DownloadString($url)
        }
    }
}

function tcBuild([string]$branch) {

	$url = "http://YourServerUrl/httpAuth/action.html?add2Queue=YourBuildId"

	if ($branch) {
		$url = $url + "&name=BuildRefName&value=" + $branch
    }
	
	$credentials = New-Object System.Net.NetworkCredential("Username", "Password")

	Get-Web $url -credential $credentials
}

Please adapt the tcBuild script by replacing the YourServerUrl, YourBuildId, Username, Password by your personal values.

The first function Get-Web is from this blog post: “Microcode: PowerShell Scripting Tricks: Scripting The Web (Part 1) (Get-Web)”.

The second, tcBuild, is quite easy and is the one you will use to start a build on the TeamCity server.

My workflow now is the following one:

  1. I commit on my local Git repository then
  2. when I want to run a build I push to the centralized Git origin repository in my branch then
  3. I start my build on the Git Branch from PowerShell by typing :

> tcBuild Jobping-10-NewFeature

where Jobping-10-NewFeature is the name of my Git branch.

This is quite cool and give me also the possibility to run kind of personal builds!

Jan
20
2012
DVCS // TDD // BDD

Improved my development workflow

Fleur d'hibiscus à la villa cannelleFor some time now I slightly modified my development workflow and I have seen a great improvement in my developers life:

  • it started by using Git Svn in front of our central Svn
  • then I introduced NCrunch in my TDD/BDD

NCrunch is an automated parallel continuous testing tool for Visual Studio .NET. It intelligently takes responsibility for running automated tests so that you don't have to, and it gives you a huge amount of useful information about your tests (such as code coverage) inline in your IDE while you work.

 I am a fan of Test Driven Development and Behavior Driven Development, I think there are great tools in a developer toolset in quite some circumstances.

I am also a great fan of JetBrains ReSharper which I use every development day and I couldn’t work as efficiently without it. But…

So what’s the point? NCrunch propose only an automated parallel continuous run of your test. Big deal, it saves you only some keystrokes! Yeah I believed that too, but the difference was quite impacting.

I don’t need to think anymore about running the tests I just run through my TDD thinking only of my tests and code, no distraction just seeing the result of my changes directly aligned with my code.

What I really liked about NCrunch is:

  • the possibility to run only a part of the tests
  • the feedback given in the code window
  • let me focus on the code
  • the risk/progress bar
  • the impact it had on my productivity

I really encourage you to try NCrunch which is free during the beta period.

Supported testing frameworks are:

  • NUnit
  • MS Test
  • Xunit
  • MbUnit
  • MSpec

The Xunit and MbUnit test runners are provided with thanks to The Gallio Project.

Supported languages and .NET versions are:

  • C#
  • VB.NET
  • F#
  • .NET Framework v2.0 and above
  • Visual Studio 2008
  • Visual Studio 2010
  • Visual Studio 11 Developer Preview
Jan
5
2012

Extending existing .NET API to support asynchronous operations

Palmier sur la plage de la grande anse du diamant

The other day I needed a way in a project I am working on to turn a .NET API, RestSharp to name it, so that I could use it in an asynchronous way.

The goal was to have the API returning a Task<TResult> so that I could use it if I want with the Async CTP. I also wanted to have something running both on .NET 4.0 and on Windows Phone 7.5. 

To achieve this goal I used

TaskCompletionSource<TResult> Class

.NET Framework 4

and defined two helper extension methods on the RestClient class of RestSharp as this:

#region using

using System;
using System.Threading;
using System.Threading.Tasks;
using RestSharp;

#endregion

namespace Sharper
{
    public static class RestClientExtensions
    {
        public static Task<TResult> ExecuteTask<TResult>(this RestClient client,
                                                         RestRequest request) where TResult : new()
        {
            var tcs = new TaskCompletionSource<TResult>();

            WaitCallback
                asyncWork = _ =>
                                {
                                    try
                                    {
                                        client.ExecuteAsync<TResult>(request,
                                                                     response => tcs.SetResult(response.Data));
                                    }
                                    catch (Exception exc)
                                    {
                                        tcs.SetException(exc);
                                    }
                                };

            return ExecuteTask(asyncWork, tcs);
        }

        public static Task<TResult> ExecuteTask<T, TResult>(this RestClient client,
                                                            RestRequest request,
                                                            Func<T, TResult> adapter) where T : new()
        {
            var tcs = new TaskCompletionSource<TResult>();

            WaitCallback
                asyncWork = _ =>
                                {
                                    try
                                    {
                                        client.ExecuteAsync<T>(request,
                                                               response =>
                                                               tcs.SetResult(adapter.Invoke(response.Data)));
                                    }
                                    catch (Exception exc)
                                    {
                                        tcs.SetException(exc);
                                    }
                                };

            return ExecuteTask(asyncWork, tcs);
        }

        private static Task<TResult> ExecuteTask<TResult>(WaitCallback asyncWork,
                                                          TaskCompletionSource<TResult> tcs)
        {
            ThreadPool.QueueUserWorkItem(asyncWork);

            return tcs.Task;
        }
    }
}

Now I can write asynchronous code like this:

public Task<List<Project>> GetAllProjects()
{
    var request = new RestRequest { Resource = "/httpAuth/app/rest/projects", RootElement = "project" };
    request.AddHeader("Accept", "application/json");

    return _restClient.ExecuteTask<List<Project>>(request);
}

So TaskCompletionSource<T> is a nice class to know about!

You might also watch this short video by Phil Pennington to get a better idea about it:

Dec
15
2011

Shortening namespace declarations in XAML files

Vue sur la tête de la femme couchée de la terrasse de la villa cannelleThis afternoon I was working on Innoveo Skye Editor which is a WPF application written in C#.

The application is using Telerik RadControls for WPF.

I was facing the issue of having more and more namespace declarations like the following for RadInput and Rad:

 

<Window x:Class="skyeEditor.View.Dialogs.ProductSettingsDialog"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:RadInput="clr-namespace:Telerik.Windows.Controls;assembly=Telerik.Windows.Controls.Input"
        xmlns:Rad="clr-namespace:Telerik.Windows.Controls;assembly=Telerik.Windows.Controls"

Searching for a solution I thought of ASP.NET in which you can declare in the web.config things like that:

<pages pageBaseType="System.Web.Mvc.WebViewPage">
  <namespaces>
    <add namespace="MyNamespace" />

My goal was to have one namespace declaration for example telerik and be able to use it in all my XAML files without having to declare too much.

So I started an online discussion with Laurent Bugnion who has always shown me the good direction, which in this case is the

XmlnsDefinitionAttribute Class

Followed by an article from Sandino Di Mattia about “A Guide to Cleaner XAML with Custom Namespaces and Prefixes (WPF/Silverlight)”.

Using the idea I was able to add the following to the AssemblyInfo.cs of my own project:

[assembly: XmlnsDefinition("telerik",
                           "Telerik.Windows.Controls",
                           AssemblyName = "Telerik.Windows.Controls")]

[assembly: XmlnsDefinition("telerik",
                           "Telerik.Windows.Controls",
                           AssemblyName = "Telerik.Windows.Controls.Input")]

[assembly: XmlnsDefinition("telerik",
                           "Telerik.Windows.Controls",
                           AssemblyName = "Telerik.Windows.Controls.Navigation")]

[assembly: XmlnsDefinition("telerik",
                           "Telerik.Windows.Controls",
                           AssemblyName = "Telerik.Windows.Controls.RibbonView")]

And then my XAML file declaration looks like that:

<Window x:Class="skyeEditor.View.Dialogs.ProductSettingsDialog"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:telerik="http://schemas.telerik.com/2008/xaml/presentation"

And I can use everywhere for all Telerik RadControls for WPF with the prefix telerik

<telerik:RadComboBox ItemsSource="{Binding ProductNodeLabels}"
                        SelectedItem="{Binding SelectedProductNodeLabel}"
                        Margin="0,0,0,5" />

A little step but a nice improvement!

Dec
7
2011

LeWeb’11

Vue sur le rocher du Diamant depuis la grande Anse du DiamantJoining Didier direction LeWeb’11 for the second time we are attending the biggest European Internet conference in Paris.

The topic this year is SOLOMO, Social-Local-Mobile.

I am quite excited about the different people, speakers and companies we will hear, meet there. It was an energizing event last time and I guess it will be again!

Hey Mathieu this would have been a good conference to attend with such a topic, hope you will launch soon!

Nov
12
2011

Fixing ReSharper inspection Results used from XAML

Rocher du diamant depuis la grande anse du diamantFinishing this week sprint I decided to inspect some code using ReSharper 6.1 EAP and I started to give ReSharper a chance to help me find some of broken code.

When I started I had some of the following inspection results. It is clearly showing that some properties wasn’t identified as used in a WPF binding.

So ReSharper proposed to make the property private which for sure was not ok for me has I knew it was used.

Some time ago I started to assign some design DataContext to be able to navigate from View to my ViewModel just by pressing Go to Declaration (CTRL-B in IDEA scheme) in ReSharper.

<UserControl x:Class="skyeEditor.View.StatusBarView"
             xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
             xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
             xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
             xmlns:ViewModel="clr-namespace:skyeEditor.ViewModel"
             mc:Ignorable="d"
             d:DataContext="{d:DesignInstance Type=ViewModel:StatusBarViewModel}">

This for sure helps ReSharper code inspection as you can see on the following screenshot

Now I can navigate from View to ViewModel, can avoid to have developer deleting properties that are sued in XAML bindings, but I can also get an idea of where a ViewModel property is used by using ReSharper Find Usage (Alt-F7). I see right away that the StatusBarView.xaml is using the property ShowActivityProgress.

So don’t forget to add the d:DataContext into your XAML !

Nov
7
2011
Git // DVCS

DVCS and my usage of Git Svn

Sometime ago I moved “away” from subversion has my Version Control System (VCS) because I felt not free of my way of working with it. I found a great was to improve my work experience by using Git Svn in front of our central Subversion repository.

I started my experimentation with Mercurial and Hg Svn because I was already using Mercurial for my personal projects. I hit some much the wall with Hg Svn that I decided quite quickly to go and try Git Svn. I had no experience with Git at that time and I had the feeling that it was more complex to handle than Mercurial (which I still think).

So currently I use Git Svn to work on one of our product at Innoveo Solutions. I also made a presentation to the team during our techno pizza lunch about DVCS, Git, Git Flow and Git Svn.

And here is the list of what I personally gained as a developer:

  1. 2 steps commit
    1. Stage/commit, Push
  2. Local history / branching
    1. No connection to central repository needed to branch/to look at history
  3. Experiment / Refactoring / Spikes
    1. Commit changes on one path, if wrong rollback
  4. Smarter Merging
    1. Git’s focus on content rather file location
    2. Better at resolving merge conflicts for you (e.g. renames)
    3. Branching/Merging is daily workflow not anymore an ‘exceptional case’
  5. Stash changes
  6. Rebase / Rewriting history
    1. Until push you can use interactive rebase

Due to my 4h daily commute (more than 2h in the train), I very much appreciate the offline capabilities of DVCS. It lets me branch, watch the history without having a connection to the central repository. So that a really convenience thing in my day to day job.

But what I even prefer is working in local branches for experiment, refactoring. With that capability I am able again to commit small steps of code change. I particularly appreciate the possibility to experiment some refactoring and to rollback wrong changes when I feel that I went the wrong way with the change. In the past this was not possible and often you would not commit to SVN because that code would be shared with the others or t would break the build. Ok, I know I could do that with Svn working in a feature branch, but we all know the pain it is with the merging back, especially when you refactor and rename files. During refactoring it is important to be able to save/commit small chunk of code change and even more important is to be able to rollback those changes. So in that case DVCS is a perfect match to that problem.

What I currently don’t like in my current way of working with Git in front of Svn is that I have local branches which I don’t sync back to the central server. I don’t like it because if I have an issue with my local machine then those changes might be lost. I will investigate in the coming weeks about possible solutions even if the best one would be a migration of the central repository to Git. But his is another story because it means a wider change in the team.

Nov
6
2011

Unlock your Windows Phone 7 with ChevronWP7 Labs

ChevronWP7 Labs just realized their Windows Phone unlocking tool. This software will let you developer unlock your device for $9.

This is not a Jailbreaking tool, it let you upload your own homebrew applications on your Windows Phone 7 device without going through Microsoft's expensive ($99/year) developer program.

You might read more about it on their blog : ChevronWP7 Labs Availability

Still waiting that the 295 people in front of me get their phones unlocked… !

Update: And here is the result after unlocking !

I also quickly deployed successfully an application to the phone directly from Visual Studio 2010 !

Congrats to the ChevronWP7 team !

Oct
30
2011

Cleaning BlogEngine.NET spam

Martinique 2011

I just migrated my blog to the latest version of BlogEngine.NET 2.5.0.6.

I had a shock when I saw the number of spam that I had on the blog!

447883 Spam! Wow. So I started the cleaning by using BlogEngine tools but it was damn slow, and no way to stop it when you started the delete all.

So I stopped the web site which was a bad idea because then one XML file was damaged. As I always do a backup before doing something like that I was on the safe side, and just reverted the files.

Then I used 7zip to zip the posts folder which is located in the App_Data which was 338 MB, again wow.

Downloaded the zip file on my local machine, installed BlogEngine and imported the post.

I thought it would be faster on my machine because it is a recent one. But still to slow to treat 447833 spam messages.

So as a developer I went on and wrote a little application to do it. And after cleanup the spam which took less than 10 seconds I went to this folder size of the posts

Quite a difference ! And BlogEngine showing me the results

And here is the code, it is using .NET Framework 4 and the parallelization of queries to treat files:

#region using

using System;
using System.IO;
using System.Linq;
using System.Xml;
using System.Xml.Linq;

#endregion

namespace BlogEngineSpamDelete
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            var files = Directory.GetFiles(@"C:\Temp\blogengine\posts", "*.xml");
            foreach (var file in files.AsParallel())
            {
                FixPost(file);
            }
        }

        private static void FixPost(string file)
        {
            XDocument doc;
            using (var stream = File.OpenRead(file))
            {
                doc = XDocument.Load(stream);
            }

            var comments = from comment in doc.Descendants(XName.Get("comment", String.Empty))
                           select comment;

            var spamComments = from comment in comments.ToArray()
                               let data = new CommentState(comment.Attribute("spam").Value,
                                                           comment.Attribute("approved").Value,
                                                           comment.Attribute("deleted").Value) 
                               where ShouldDeleteSpamAndUnApproved(data)
                               select comment;

            foreach (var spamComment in spamComments)
            {
                spamComment.Remove();
            }

            using (var writer = XmlWriter.Create(file, new XmlWriterSettings {Indent = true}))
            {
                doc.WriteTo(writer);
            }
        }

        private static bool ShouldDeleteSpam(CommentState commentState)
        {
            return !commentState.Approved && 
                   (commentState.Spam || commentState.Deleted);
        }
        
        private static bool ShouldDeleteSpamAndUnApproved(CommentState commentState)
        {
            return !commentState.Approved || 
                   commentState.Spam ||
                   commentState.Deleted;
        }

        private class CommentState
        {
            public CommentState(String spam, String approved, String deleted)
            {
                Approved = bool.Parse(approved);
                Spam = bool.Parse(spam);
                Deleted = bool.Parse(deleted);
            }

            public bool Approved { get; private set; }
            public bool Spam { get; private set; }
            public bool Deleted { get; private set; }
        }
    }
}

Update: I also posted the code on bitbucket: https://bitbucket.org/lkempe/blogenginespamdelete

About Laurent

Laurent Kempé

Laurent Kempé is the editor, founder, and primary contributor of Tech Head Brothers, a French portal about Microsoft .NET technologies.

He is currently employed by Innoveo Solutions since 10/2007 as a Senior Solution Architect, certified Scrum Master and Founding Member.

Founder, owner and Managing Partner of Jobping, which provides a unique and efficient platform for connecting Microsoft skilled job seekers with employers using Microsoft technologies.

Laurent was awarded Most Valuable Professional (MVP) by Microsoft from April 2002 to April 2012.

JetBrains Academy Member
Certified ScrumMaster
My status

Twitter

Flickr

www.flickr.com
This is a Flickr badge showing public photos and videos from Laurent Kempé. Make your own badge here.

Month List

Page List