Jenkins and Promoted Builds

I have been revisiting at the automated build system that we use at work. It is a automated representation of the old manual builds system that existed 18-months ago.

For many reasons I have been unhappy with the situation, which represents a compromise on the purity of the system against what we could do in the time.

At the same time we are moving to Subversion, and I want to be able to remove svn  at a later date and replace it with another if required. The use of a different SCM is not the difficulty here!

The situation is that we have a live branch of code, and a development branch of code. Developments go into the 'dev' branch, are tested and QAed and the promoted to the live code. Previously we have used a 'holding' level were code can be checked in at anytime. 

We are going to used Promoted Builds in Jenkins to make this happen. Referring to the blog entry from CloudBees - Another Look at the Jenkins Promoted Builds Plugin, I have setup the standard svn structures for 'live' and 'dev'.

There are eight Jenkins jobs (there could be less, but I like pipelining), breaking down the pipeline of builds, 1 pipeline is the standard CI builds for the 'dev' branch. This consists of Build -> Test -> Deploy pipeline.

CI pipeline

Developers will commit changes to the code repository and that triggers the initial build job - development2. Once this has been successful, it will trigger the development2-tests job, which runs tests, gathers code-coverage statistics. When these pass, it then triggers the development2-deploy task, which copies the builds to the server holding the developer test environments.

A second pipeline (promoting and deploying to QA) is triggered by a manual promotion from the development2 build. This allows the code to be recompiled in release conditions, and deployed to the QA staging area.

The final pipeline (promoting and rebuilding the live system) is triggered by a second manual promotion from the development2 build. This allows the code to be merged into the live branch, and then rebuilt and deployed. The 'live' is not the code that gets delivered to the client, the QAed code goes out the door. It is just there to represent the fully tested code.

Prism 4.0 learnings

I have started to plan to use Prism 4.0 in one of my organisations internal applications. Previously I created a shell application and several modules that implement a standard interface. Having looked at Prism, much of this is already done via the templates that are available.

I'll post some thoughts, problems and examples when I have something concrete.

Extended ListView WPF Custom Control (Part 1)

I am trying to write a WPF control that can be used in my projects to provide a standardised look and feel for data that is presented via grids. I'll put my thoughts down here, as a aid to my fading memory.

First I'll create the control, descended from ContentControl (as it will contain further content)

namespace Controls
{
    using System;
    using System.Windows.Controls;
    using System.Windows;
    using System.Windows.Input;
    using System.ComponentModel;

    public class ListViewExtended  : ContentControl
    {
        static ListViewExtended()
        {
            DefaultStyleKeyProperty.OverrideMetadata(typeof(ListViewExtended), 
                        new FrameworkPropertyMetadata(typeof(ListViewExtended)));
        }
    }
}

This is really simple, and is a standard setup, next the difficult bit is creating the correct XAML template, this looks (in the initial version anyway), like this:

This can be used as follows:

I'll tidy this up and add a screenshot later, and then extend the example to show it doing what I want it actually to do in the next post

MVVM RelayCommands and BackgroundWorker

This is largely cribbed from elsewhere, but I wanted to have a system where a RelayCommand starts off an asynchronous update of the Model.

First we have to have a flag to hold the idle status

public bool IsIdle ( get; set; }

The RelayCommand command needs to bind its CanExecute to this property

_getCommandData = new RelayCommand (
 () => GetTimeSheet(),
 () => IsIdle);

This means that the WPF item bound to this command (in this case a button), is disabled when the idle status is set to false.

So in order to use this, and to not bung up the main UI thread, we need to run the command in a BackgroundWorker thread

protected void DoWork(Action work)
{
 idle = false;
 var worker = new BackgroundWorker();
 worker.DoWork += (sender, e) => work();
 worker.RunWorkerCompleted += (sender, e) => Idle = true;
 worker.RunWorkerAsync();
}

public void GetTimeSheet()
{
 DoWork (delegate() {
   _model.GetTimeSheet();
   TimeSheetData = _model.TimeSheetData;
 });
}

TLBs and Delphi

We register the capabilities of Delphi applications using TLB files. However, from reading MSDN documentation, "Installation package authors are strongly advised against using the TypeLib table. Instead, they should register type libraries by using the Registry table". Does anyone have any advice on how to do this in a 'Delphi' way for Windows 7?

Delphi/Surround/Cruisecontrol.net howto (updated)

This is a further draft of what I had to do to get CruiseControl.Net and Delphi to play nicely - I will fill out further the details later!

1. Configured IIS in Windows (see here)
2. Install CruiseControl.Net 3.5 RC1 (see release notes) - Make sure that the user that the CruiseControl.Net is being run by is able to see and use Delphi (i.e. probably not by default)
   
3. Install CCTray
4. Set the BDS environment setting to point to where Delphi lives (this may have been done when Delphi was installed)
5. Create a definitions.xml file that has all of the required defaults, for accessing Surround, MSBuild, etc.
   
6. Create a project to test. It is difficult to post a real example, as this is code that my company owns, rather than me, but an example could be pulled out and added to this article.
   
7. Setup connection to Seapine Surround. I had a real problem with using Delphi and Surround, as by default the code is checked out as readonly, and Delphi needs to write to the working directory (especially when building typelibraries, etc). I have not spent too much time fiddling with it as I have a patch for the CruiseControl.NET source.
8. Setup the MSBuild section to build the application (in the definitions.xml file)
9. Setup the configuration to be managed by CruiseControl.NET and Surround, so that the files can be just checked out, modified and then checked in again - this avoids having to access the server machine.
10. Each developer now gets emailed whenever a build fails, and we can see the running of tests as well, after each build, see below.
    
11. Modify the DUnit tests to output XML.
    

This I did using the code from the link above, and changed the base UnitTest program code to look like this:

var
res : TTestResult;

...

if IsConsole then
  if ParamCount <> 0 then
    if (copy(ParamStr(1), 0, 5) = '/xml=') then
      res := XMLTestRunner.RunRegisteredTests(copy(ParamStr(1), 6, length(ParamStr(1)) -5))
    else
      res := TextTestRunner.RunRegisteredTests
  else
      res := TextTestRunner.RunRegisteredTests
else
  GUITestRunner.RunRegisteredTests;

if (res.ErrorCount <> 0) then
  halt (1);

Other things I have done ...

• Modify (legally I'm not sure if I can publish it) the code from Run-Time Systems Cyclomatic Complexity Calculator for Delphi so that it outputs the results as an XML file - it actually does this already internally, but doesn't write the results out.
• Create a task to generate developer documentation using doc-o-matic as a task.
  

MVC Framework in Delphi

A real world example can be found here on Joanna Carter's pages.

ListViews

Things I can do with ListViews:

Flat header Vs. 3D header
capture a column resize event in a TListView?
Prevent resizing
Handling Right-Click Events in ListView Column Headers

CI Updated

I now have all of our projects in CCNET, and am using this to control the port of code to Delphi 2009. I am also moving code around, so this helps. I had to patch a few things, but I have DUnit outputing xml that is then picked up by CCNET and displayed graphically (this is cool).

I have also taken the Cyclometric Complexity tool that I use and created a command-line version (this used the OTA interface before), which also outputs to XML. I'll then plug this into the projects and this will allow us to automatically measure complexity!

Getting testing working

OK, so I have the source checked out, and then I have built it. I even have a test application, that outputs XML. However, we have a graphical login. So I need to be able to pass in the signon somehow. OK, so I need to think a bit more about this.

Cruisecontrol.net howto (draft)

This is a draft of what I had to do to get CruiseControl.Net and Delphi to play nicely - I will fill out the details later!

1. Install Apache 2.
2. Install Mod_ASPDotNet.
3. Install CruiseControl.Net.
4. Install CCTray.
5. Create a project to test.
7. Setup connection to Seapine Surround.
8. Setup the MSBuild section to build the application.
9. Modify the DUnit tests to output XML.

This I did using the entry above, and changed the base UnitTest program code to look like this:

var
  res : TTestResult;

...

  if IsConsole then
    if ParamCount <> 0 then
      if (copy(ParamStr(1), 0, 5) = '/xml=') then
        res := XMLTestRunner.RunRegisteredTests(copy(ParamStr(1), 6, length(ParamStr(1)) -5))
      else
        res := TextTestRunner.RunRegisteredTests
    else
        res := TextTestRunner.RunRegisteredTests
  else
    GUITestRunner.RunRegisteredTests;

  if (res.ErrorCount <> 0) then
    halt (1);

More later!

CI Update

Well, after much messing about I now have CI working for Delphi. It checks out the code from Surround (easy to configure), calls MSBuild (again easy), and this then compiles the Delphi source (tricky and required me to mess about and reinstall CruiseControl.NET). It now checks out 5 projects (3 of which compile, 1 fails due to a missed file, and 1 needs an project to be registered), which is about what I expected.

Hurrah!

Cruisecontrol.net update

Well, I have CCNet working and trying to build (but failing) Delphi 2009 code, using MSBuild, good news I found out what was wrong, the bad news was I had to hack the CC.NET source to get this to work.

The cause of the problems was that Delphi 2009 uses the brcc32.exe to build its resources, which is obviously fine, but when the code is downloaded from Surround, it is read-only, so it create lots of .tmp files, all of which are read-only and so can't be written to. I downloaded the CC.NET source and changed the command line that is passed to Surround to checkout the code as read-write, and now the code compilation gets passed this.

However, I am now stuck with MSBuild working from the command line, but not when run from CC.NET. Ho hum, I'm sure I can figure this out as well. Nothing is ever easy!

CI with Delphi

I am trying to get CruiseControl.NET to connect to a Surround SCM system and then check out and build (and then test if possible) projects.

It was simple to install, but it needs as ASP.NET server to work (or so it seems). I've installed Apache and the mod_asp (or whatever it is called), and have got a simple ASP script to work, but can't get the CC.NET dashboard application to run.

More news as it breaks.

Update 1

I now have access to the CC.NET dashboard working. I can't get the command line that CC.NET builds to be correct, the settings seem OK, but I get the following ...

Process command: E:\Program Files\Seapine\Surround SCM\sscm.exe cc * -d20090401153253:20090401153326 -r -bXXXX -p"XXXXXX" -x- -zABCABCDABCDEF:4900 -y"XXX:XXX"

Update 2

Fixed the file locations, and started up the VPN, and now CruiseControl downloads the latest source, and tries to rebuild - the MSBuild process now gives the following error:

E:\Documents\Mark\CCBuildDir\SSCMClone\D2009\XXXX\XXXX\XXXXX.dproj (,): 

errorMSB4057: The target "Rebuild" does not exist in the project.

So we are getting much closer, and this might be working before I go back to work

Continuous Integration and Delphi

Continuous Integration and Delphi using CruiseControl

Actually implemented

I have started to build a proof of concept that actually implments code that does stuff, so can't actually post that code here, but basically it follows the same structures, with a command being sent via the Controller, and then being run across the model (and hence the data). The notification of errors is done by the Notify mechanism, with whatever view knowing what to do with the errors statuses returned for each possible input.

Once a change to client, stock, buy/sell, etc has been entered, then at the the moment the Notify mechanism is directly fired, but what should happen is that the whole order should be validated at this point, allowing for cascadimg errors to be flagged.

We will see on monday!

MVC updated

I have had a small rest over Christmas and have decided that the code below needs to be refactored. I broke down all of the interfaces and implementations of them into different units, and begun to separate out the stuff that is MVC and stuff which used by the model.

Specifically I broke down the command stuff and implemented descendants of TCommand that know what to do to the model - i.e. the model doesn't do anything other than respond to the data changes (or not as is the current state).

This means I have ended up with code like this ..

  TCalculateCommand = class (TCommand)
    procedure Run (model : tObject); override;
  end;
  ..

  { TCalculateCommand }

  procedure TCalculateCommand.Run (model : tObject);
  begin
    (model as TDomainModel).Data.Price := self.Params[0].Value;
    (model as TDomainModel).Data.Consideration := ((model as TDomainModel).Data.Size *
          (model as TDomainModel).Data.Price) + (model as TDomainModel).Data.Charges;
  end;

This is a lot simpler in my view.

However it doesn't solve the problem with how to indicate that a field is invalid?

I might have to register each control with the model, so it knows which fields map to each control. That sounds like a lot to faff, but I don't know how else to do it.

MVC Update

In the real world, which I obviously can't post here, I have a problem where the result of the validation request does three things;

1) Validates the input values
2) Returns other information based on the inputs (i.e. Other client, stock details)
3) Set whether a specific field input is valid or not.

More MVC!

A client search from would look like this ..

type
  TForm3 = class(TForm, IObserver)
    Button1: TButton;
    Edit1: TEdit;
    ListBox1: TListBox;
    Button2: TButton;
    Button3: TButton;
    procedure FormCreate(Sender: TObject);
    procedure Button1Click(Sender: TObject);
    procedure Button3Click(Sender: TObject);
    procedure Button2Click(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
    procedure UpdateView (obj : TObject);
  end;

...

procedure TForm3.Button1Click(Sender: TObject);
var
  c : TCommand;
  o : TCommandParam;

begin
  c := TCommand.Create ('SEARCH');

  o := TCommandParam.Create;
  o.Name := edit1.Text;

  c.AddParam(o);

  Controller.DoCommand(c);
end;

procedure TForm3.Button2Click(Sender: TObject);
begin
  ModalResult := mrOK;
end;

procedure TForm3.Button3Click(Sender: TObject);
begin
  ModalResult := mrCancel;
end;

procedure TForm3.FormCreate(Sender: TObject);
begin
  Controller.AddView (self);
end;

procedure TForm3.UpdateView (obj : TObject);
var
  val : String;

begin
  // Update the view here
  if ((obj as TDomainModel).ClientSearchResult <> nil) then
  begin
    for val in (obj as TDomainModel).ClientSearchResult do
    begin
      ListBox1.AddItem(val, nil);
    end;
  end;
end;

I have merge the View classes with the actual form, in order to make the code relationships a little clearer.

I'm really happy with all of the, and I think it will be applicable for the requirements I have at work.

MVC questions

Having done all of the below, I am still left with a list of questions

1. Should I be using threads to do this ?
2. When the command requires interaction (i.e. if I want a form to search of a client or a stock item), should it be owned by the Model, Controller or View ?

To answer 2., the view owns the form, and and is updated by the model in the same way as before.