HaveComputerWillCode.Com

Welcome!
Life is a Non-Deterministic Finite State Automata
Automation ? (*pGeekiness)++ : Code /eneration;

January 31, 2010

Writing a Macro Recorder: Part 2 – Testing Generated C# Code

Filed under: Programming — Tags: , , , — admin @ 7:29 pm

Writing a Macro Recorder: Part 1 – C#
The source code can be downloaded here (Visual Studio 2008). Download and use at your own risk.
NUnit 2.5.0.9122 can be downloaded here.

I showed you in the last post how to generate C# code by listening to the IBindingList events. That was a small example, but as the number of commands grow and the parameters proliferate, the complexity of the generated code rises too. How to test this generated code?

The answer is obvious: compile the code that was generated. Although a bit of a hassle to set up, once in place, this isn’t just useful for testing generated macro code. It’s useful for testing *ANY* generated code. In the sample code, I will be using NUnit v2.5.0 to drive the tests. If you are using an earlier version, you will probably be OK if you just remove the nunit.framework reference and add your version in.

.Net is cool. The infrastructure comes with its own compiler! Well, technically, it comes with a C#, VB.NET and Managed C++ compiler and there are additional compilers for Cobol.NET, IronPython and pretty much every other language you can think of.

But first a few observations. In the last post, I was driving the binding list via the UI. Automated UI testing is a whole different field so I’m going to pretend I didn’t use a UI last time. Instead, it would be easier to drive the list from code so we know exactly what we expect to be generating. In the TestHarness/TestingBase.DoATest method you will find this:

// CSharp.cs
//
System.ComponentModel.BindingList
 people = new System.ComponentModel.BindingList
();

// We've set up the handler that will forward changes to the macro recorder...
// by driving the list directly, we can always view Recorder.Instance.Text
// to see what has been generated so far.
people.ListChanged += people_ListChanged;

Person woo = new Person();
woo.Name = "WOO";
woo.Age = 33;
people.Add(woo);

Person hoo = new Person();
hoo.Name = "HOO";
hoo.Age = 50;
people.Add(hoo);

people.Remove(woo);

Hardly rocket science: I set up a listener to forward any collection changes to the macro recorder; then I set up the collection. If you step through the code, by the time you get to the end of what has been shown here, you can interrogate the Recorder.Instance.Text field and get hold of this code that was generated by your previous statements:

Before I go on: just think about how awesome that is! That text was generated behind the scenes and (in this case) is practically equivalent to the code that was used to create it. When driving the IBindingList, perhaps in a real application, the developer would have no idea that macros were being recorded behind the scenes as a result of what they did to that data structure. Take this one step further: if you have the context to record a macro, you have the context to record who did what, to what, where and when… in other words, a full Audit Trail. Priceless for troubleshooting and debugging or even to see how your application is really used.

But I digress. Back to the code. Clearly, we can’t compile that. It isn’t even structured correctly. There is no class around it, no using statements and… how will we test it? The best way I’ve found is to create a place holder text file like this:

// Templates/CSharp.TXT
//
using System;

namespace AutoGeneratedCodeTest
{
    /// 
    /// Static wrapper class.
    /// 
    public static class Runner
   {
        /// 
        /// Test method to be invoked. Generated code will be substituted herein.
        /// 
        public static System.ComponentModel.IBindingList Run()
        {
//%CONTENTS%//

            return theBindingList1;
        }
    }
}

You need to substitute the place holder text with the generated code and attempt to compile it – that is step one. Congratulations: your code compiles. But you need to ensure that the operations you have just done, and generated code for, will reconstruct the object into the same state. In this case, it does not matter if the code is equivalent: what matters that the output is equivalent. For example: in the test harness I called .Remove(objectReference) but in the generated code it will generate .RemoveAt(theIndex).

The compilation step is easy. That can be found in the Language/CSharp.cs file but the key line is clearly this:

// CSharp.cs
//
    CompilerResults results = provider.CompileAssemblyFromSource(parms, txt);

Look at the CSharp.txt template again. There is a static method called Run() that returns an IBindingList – the IBindingList that was constructed entirely by the generated macro code that we have substituted. After the substitution, our code looks complete and ready to run like this:

using System;

namespace AutoGeneratedCodeTest
{
    /// 
    /// Static wrapper class.
    /// 
   public static class Runner
   {
        /// 
        /// Test method to be invoked. Generated code will be substituted herein.
        /// 
        public static System.ComponentModel.IBindingList Run()
        {
           System.ComponentModel.BindingList theBindingList1 = new System.ComponentModel.BindingList();

           MacroSample.Person thePerson1 = new MacroSample.Person();

           thePerson1.Name = @"WOO";
           thePerson1.Age = 33;
           theBindingList1.Add(thePerson1);

           MacroSample.Person thePerson2 = new MacroSample.Person();

          thePerson2.Name = @"HOO";
          thePerson2.Age = 50;
          theBindingList1.Add(thePerson2);

          theBindingList1.RemoveAt(0);

          return theBindingList1;
      }
}

So with that infrastructure in mind, we can run the end of our test. The Compile method (in the case of CSharp) generates an Assembly… we can then ‘jump in’ to the generated assembly and obtain the list that was built up using our code:

// TestingBase.cs::DoATest
//
   Assembly assembly = Compile(Recorder.Instance.Text) as Assembly;

   IBindingList result = Execute(assembly) as IBindingList;

   System.ComponentModel.BindingList
 macroPeople = result as System.ComponentModel.BindingList
;

   Assert.AreEqual(true, ListComparer.AreEqual(people, macroPeople));

I wrote a ListComparer class for this test. Recall that ‘people’ is the one we built up in our test. ‘macroPeople’ is what was built up in our generated macro code; the code we compiled and then jumped in to.

To sum up: that is one way to test generated C# code. There’s a million and one ways to do this: with the CodeDOM you could build up the target file programmatically and add the generated code under the method definition. But it’s easier to do it like this: if it doesn’t compile, it is easy for you to add the code you are TRYING to compile into your project, fix everything there, and then put it back into the automated test.

January 30, 2010

Writing a Macro Recorder: Part 1 – C#

Filed under: Programming — Tags: , , — admin @ 9:27 am

The source code can be downloaded here (Visual Studio 2008). Download and use at your own risk.

Why?
When writing Grom, one of the things I wanted as part of the underlying infrastructure was a Macro Recorder. By driving the generated model, I could see – in any language I could be motivated to write a generator for – the code I needed to repeat that operation externally. Without writing a single line of documentation, regression testers, third parties and user interface authors could learn how to drive my model from the generated code.

To show the principles of how to write a Macro Generator, I am going to use this example:

You drive the DataGrid like you drive any DataGrid: adding items, removing them and setting property fields. The text you see in the bottom was generated automatically by the Macro Recorder in response to changes initiated by the user in the grid.

Commands
The idea is to translate every operation you want to generate code for into a discrete ‘Command’ that contains enough context to repeat that operation. In this example, I am using classes derived from System.EventArgs as a command. That command is then ‘Audited’ or ‘Published’. If you provide enough context in the command, that same information is probably enough for an Undo/Redo/Repeat manager to do it’s job, too. The solution to these problems are closely related. See the Command Pattern for more information.

In Grom, I do the macro generation using the Text Template Transformation Toolkit [T4] but for the purposes of this sample I will be using StringBuilder.Append.

A key decision is what ‘Command’ you want to use as a basis for macro generation. Whilst the .Net infrastructure events such as INotifyPropertyChanged.PropertyChangedEventArgs are rich enough for macro code generation, it is unsuitable for Undo/Redo operations because the old value of that property is lost.

If you want something done, …
If you are generating code from a model, the solution is easy: create a rich Command set yourself that is part of the underlying infrastructure your generated code relies on, each containing lots of contextual information for the operations you want to expose from that model. If necessary, use the standard .Net binding interfaces – such as INotifyPropertyChanged – but derive your own context class from PropertyChangedArgs and stash any additional information in there before you fire the event.

For this sample I will use the System.ComponentModel.BindingList implementation provided by .Net.

Code Walkthru
The set up is easy and is done in Form1.cs. All I do is create a collection and set it as the DataSource on the DataGridView. The DataGridView queries the collection for IBindingList and calls methods on that interface when the user performs actions in the DataGrid. For those who don’t know (!), this is called Binding :-)

// Form1.cs
//
People = new BindingList();
People.ListChanged += new ListChangedEventHandler(People_ListChanged);

// Whenever the Recorder gets updated, the text box will be updated too. 
richTextBoxMacroOutput.DataBindings.Add("Text", Recorder.Instance, "Text");

dataGridViewPeople.DataSource = People;

When the Gridview causes a change on the collection, the collection fires the ListChanged event which the Form handles in People_ListChanged. That event is then dispatched – verbatim in this case – to the Recorder. It is the recorder that does the code generation:

// Recorder.cs
//
switch (args.ListChangedType)
{
   case ListChangedType.ItemAdded:
      Buffer.Append(GetTextForListConstruction(sender as IBindingList));
      Buffer.Append(GetTextForObjectConstruction((sender as IBindingList)[args.NewIndex]));
      ...
      break;
   case ListChangedType.ItemDeleted:
      ...
      break;

As you can see, the logic is straight forward: each ‘command’ needs translated into code.

Sometimes, objects are used as parameters in generated code and you will need to use a ‘friendly name’ to make them recognizable elsewhere in that code. That is what the NamingManager is for. Given an object, it will create a name of the form:

the[theObject.GetType().Name][count]

Those object-to-name mappings are obviously cached so that identity can be preserved between commands.

That is*ALMOST* everything! If an object has been added to the list (in this case), it is unknown to the NamingManager. It has no name. What to do in the macro recorder?

1. We could just create a new object of the relevant type in code, but any values already in the object before it is assigned will be lost.

     MacroSample.Person thePerson1 = new MacroSample.Person();

2. We do as (1) above but then generate an assignment operation to set up every property.

      MacroSample.Person thePerson1 = new MacroSample.Person();
      thePerson1.Age = 30;
      thePerson1.Name = "Woo";

I generally prefer the latter but it depends on what you are building up. In Grom, it would generate quite a lot of code to build up an entire application the first time it was used!

Summary
The general sequence of translating any command into code is this. I use the .ItemAdded command as an example:

    // Generate the code the 'source' object - the thing that is having something done to it. 
   Buffer.Append(GetTextForListConstruction(sender as IBindingList));

    // Generate the code for every 'object' parameter. We need to give these a name.
   Buffer.Append(GetTextForObjectConstruction((sender as IBindingList)[args.NewIndex]));

    // Generate code for the command itself. At this point, you have a 'name' for every parameter.
    Buffer.Append(String.Format("    {0}.Add({1});\r\n\r\n", Manager.GetNameFor(sender), 
Manager.GetNameFor((sender as IBindingList)[args.NewIndex])));

The GetTextForXXX methods all behave like follows:

// Recorder.cs
//
        protected string GetTextForListConstruction(IBindingList theList)
        {
            if (null == theList) throw new System.ArgumentNullException("theList");

            // If the list is already known about, no need to construct it...
            if (Manager.Exists(theList)) return "";

       ........

If the object being built is already known about in the NamingManager there is no need to build anything up… it already has identity which means the code to construct it must already exist in the generated code. In which case, it just returns. Otherwise, it establishes a name for itself with the NamingManager and then returns the code to construct itself.

A few things
Generally, the events raised by .Net do not contain enough contextual information for you to do macro generation. For example: with IBindingList, when an object is removed, .ItemDeleted contains only the index of the object that was removed – but not the object itself. Why would you want that reference? Well, the IBindingList.ItemChanged event tells you that the property of an object in the list has changed but does not tell you what property. Which means to get that information you have to subscribe to those events yourself when the object is added… and unsubscribe from them when the object is removed. But you can’t because you don’t know what object has just been removed. AAARGH!

I am *CONVINCED* I am missing something obvious here. The above would seem like something you would so want to do … any ideas?

In Grom, I do not use *ANY* .Net EventArgs at all. As am I generating code from a model, I used a well defined command set that is a part of my infrastructure and I publish those commands instead. The routing of the command, from object to Macro Recorder, is a little complicated and definitely beyond the scope of this post!

How do you (re)construct a complex object? In this sample, Person contains two primitive properties – Age and Name that can be Null. Recall how we build up the object in the macro code when it is assigned to the list:

      MacroSample.Person thePerson1 = new MacroSample.Person();
      thePerson1.Age = 30;
      thePerson1.Name = "Woo";

What if Person contained nested objects? Could (should?) you generate the code to build those objects up programmatically? The fundamental problem is that you cannot create reliable code for a macro recorder unless the object is serializable and can be ‘rehydrated’ from the state that was persisted. This is why it is ideal if you have the original model around at runtime: it contains well defined constructs that you know how to individually map into macro code. You can probably only generate reliable macro code for things you understand and ‘own’.

How to test it?
Generating the macro code is only half of the problem… when you have a complex command set, and the commands are themselves relying on third party data to produce code, you need to find some way of automating the testing of it… this is a snaphot of the Macro Recorder in Grom:

Many things influence the way macro code is generated. All of the classes in my generated model begin with ‘Modeling’. That is the root namespace for Grom. So where does the ‘Grom.’ prefix come from at the start of the type name? How does the Macro Recorder know to use that? That is a property from the code generation job used to generate those classes from my model. The original model – and all parameters to generate that model – are shipped along with the generated code and product. You can see the raw model using the GenMex above.

What if an object is assigned in code that was generated with a different prefix?! I won’t elaborate…you get the point: complexity++. Manual inspection of the generated macro code using a User Interface just isn’t feasible if quality is to be maintained. Automated testing is necessary.

And that’s what part 2 is for :)

How to show code snippets in HTML

Filed under: Programming — Tags: , — admin @ 5:49 am

If you’re writing a programming blog, at some point you’ll need to show source code… which got me thinking: how? Can I preserve the rich text and color  information I get when I copy from Visual Studio and paste into Outlook? Do I need an excuse to play with PowerShell for a while to parse text and belch out HTML code with all the colorization sorted?

No need! Alex Gorbatchev’s superb Syntax Highlighter is what you need:

using WooHoo;

public class : WOO
{
        System.Console.WriteLine("WoOoOoO");
}

He asks for a donation if you use it… but I don’t have PayPAL. When I get it sorted out here in Aus, I’ll be sure to send him a few bucks. The highlighter covers every language I’ll need to post on here – C#, C++, PowerShell, Perl, VBScript, JScript and so on.

January 29, 2010

Hello!

Filed under: Programming — admin @ 11:02 pm

…and welcome to my new blog :-)

I’ve been meaning to register a new domain name for a while so I could keep programming related paraphernalia out of my travel blog. Here it is!

Like my last blog, I am using www.evohosting.co.uk to host it because they have delivered 100% uptime and reliability. Not once in the last seven months have I had issues connecting to my site from Europe or Australia. Superb service!

During the last few years, I’ve learned a lot about code generation, writing macro recorders and hosting PowerShell 2. When I get around to it, I will share some of that knowledge on this blog.

Powered by WordPress