Simulating dynamic variables in C#

January 28, 2008

Well… The C# 3.0 compiler has been out for a couple of months now. I assume plenty of people are using it (albeit calling it “VS 2008”). I also assume, judging by the fate of C# 2.0, that a lot more people are actually programming in C# 1.0/2.0, just under VS 2008. Long live backwards compatibility!

This will be my humble attempt to show some things that you can do in C# 3.0, that you maybe could have done in 2.0 with an uglier syntax and that you surely couldn’t have done in 1.0 (by humane methods).

Disclaimer: you may consider a lot things here an abuse of the new lambda syntax, an abuse of reflection and and downright violation of “traditional C# programming practices”. Good for you. That’s your opinion. Just make sure you don’t end up like this guy that considers these features “subversive to business interests”.

The first thing that popped on my mind are dynamic variables. Present in Lisp, Perl and a whole list of other languages I have no clue about, dynamic variables are basically variables whose value is resolved at runtime by searching up the stack for their definition. Yeah.. that came out weird, let me explain.

There are two types (the perl6 guys might disagree) of variables: lexically scoped and dynamically scoped.

Lexically scoped variables are the regular variables: you define one, you make a closure around it, you return the closure, and when someone calls it, the enclosed variable will have the value that was set in the scope where is was defined. Just for the sake of it (and because it’s pretty in C# 3.0), here’s the classic accumulator function where ‘t’ is a lexically scoped variable:

// You can paste this code inside any C# method. Main() is good.
Func<Func<int>> Accum = () => {
    var t = 0;
    return () => ++t;
}

var acc1 = Accum();
var acc2 = Accum();

acc1().Print(); // prints "1"
acc1().Print(); // prints "2"
acc2().Print(); // prints "1"
acc1().Print(); // prints "3"

Now, for dynamically scoped variables. Fist of all, why do we need a new kind of variables?

Let’s say you’re working on an application for managing.. uhm… cows. Kind of like an ERP for farmers. Suppose you use some logger (I assume you’re not just spitting out exceptions to stderr or something). Let’s call it Logger, with methods like… Logger.Log(Exception ex). Brilliant! Now, what you come to realize after some usage is that the exceptions alone are not helping you that much because pretty much the only useful information they give you is the line number. So what we need is some context.

Let’s say we add the Log(Exception ex, Dictionary context) method to our logger. First of all, a good idea may be to log the currently logged in user and the time when the error happened. Most loggers (log4net, NLog) support specifying things like this in their configuration (albeit it’s a clear violation of the separation-of-concerns principle). But soon you realize that you want different contexts for different (kinds of) actions, like logging the ID/number/name of the current cow if errors occur while viewing information about a specific cow, so you can easily see which cow is responsible for the error. And maybe we want some different context logged when calling a method that processes an order for some cattle from some customers, like the customer’s name, email and the quantity he intends to buy. The logger’s configuration, as bright as it could be, is by no means a solution.
One (dumb) way to achieve this is to actually carry the context through the calls as an additional parameter:

public void ProcessCommand(Command command) {
    //do some stuff
    var context = new Dictionary<string, object> {
        {"client", command.Client.Name},
        {"client", command.Client.Email},
        {"Command quantity", command.Quantity}};

    AddTaxes(command, context);
    //do some other stuff
}

private void AddTaxes(Command command, Dictionary<string, object> context) {
    try {
       //...
    }
    catch(Exception ex) {
       Logger.Log(ex, context);
       throw;
    }
}

For reasons that I won’t discuss here, this is kind of dumb.

Another (maybe dumber) way of keeping the context is by adding a “Context” field to whatever class you’re in. First, this doesn’t work, since obviously it assumes that all the methods which write to the log that might get called somewhere along the way are from the same instance as the “context” field. Secondly, such a field would pollute the model with strange and unnecessary things in a lot of places.

You could, of course, do this in a lot of ways. But, since I mentioned dynamically scoped variables some time ago, we’ll use DynVar. This is a class for that provides static methods for defining and binding dynamically scoped variables of type T. The “Set” method if defined as:

public static void Set(T value, Action<T> scope) {...}

The first parameter is the value you want to set. The second is a scope in which the variable will be defined, which is actually a lambda expression of type Action<T>. When the scope terminates, the variable defined will also unbound (“deactivated”, so to speak). The name of the variable is actually obtained from the name of the parameter of the Action<T> lambda expression passed as the scope parameter.

So, by using an Action<T> as the type of the scope we have obtained three things:

  1. we can easily define the scope for the variable
  2. we provided a name for it (from the parameter’s name)
  3. we actually bind the variable inside (and only inside) the scope

The example above becomes:

public void ProcessCommand(Command command) {
    DynVar<Dictionary<string, object>>.Set(
        new Dictionary<string, object> {
            {"client", command.Client.Name},
            {"client", command.Client.Email},
            {"Command quantity", command.Quantity}},
        loggerContext => {
            //do some stuff
            AddTaxes(command); // no more context, yay!
            //do some other stuff
        });
}

The first parameter is the dictionary with the context. The second is an Action<object> that has the exact code previously contained in the body of our method. The scope’s parameter name, “loggerContext”, indicates the name of the variable bound. Also, the context is bound inside the scope in the parameter, so if we want to modify it we could easily do so.

So if “Set” is only for defining variables, how does one use them? With the “Use” method:

public static Use(Action<T> scope) {...}

This binds the value of a dynamically scoped variable (or the value null if the variable hasn’t been previously defined) to the parameter of the scope. For example, the Logger.Log method now looks like this:

public static void Log(Exception ex) {
    DynVar<Dictioary<string, object>>.Use(loggerContext => {
        //concatenate all the values from the context, the exception and log it somewhere
    });
}

So.. what have we improved? Well, for once there’s no parameter passed around and there is no magic field holding the context. And this works for any call to Logger.Log that occurred somewhere inside the scope received by set, no matter how deep the call stack was or what class the method was defined in.

But this is not all. We can redefine an existing variable, use it inside a scope, and then, when the scope ends, the variable will return to it’s previous value. Kind of like a stack. Here’s a simple example:

void F1(int i) {
    DynVar<Dictionary<string, object>>.Set(
        new Dictionary<string, object> {{"F1's context", i}},
        loggerContext => {
            Logger.Log(null);
            F2(i + 1);
            Logger.Log(null);
        });
}

void F2(int i) {
    DynVar<Dictionary<string, object>>.Set(
        new Dictionary<string, object> {{"F2's context", i}},
        loggerContext => Logger.Log(null));
}

The call F1(1) will log:
F1’s context: 1
F2’s context: 2
F1’s context: 1

This is what happens: The dynamic variable loggerContext is first defined in F1 so Logger.Log logs “F1’s context”. Then, when F2 is called, the variable is re-declared with a new value. This causes the logger to log “F2’s context”. After the declared in F2 terminates (right before F2 itself terminates), the loggerContext variable returns to it’s previous value, the one declared before in F1. Hence, Logger.Log again logs “F1’s context”.

So, let’s try to sum up some properties discussed so far for dynamically scoped variables:

  1. They are defined over a scope.
  2. They exist as long as the scope executes.
  3. They can be accessed from anywhere as long as the defining scope is alive.
  4. A scope can contain yet another self-contained definition for the same variable.
  5. When a scope exits, the variable returns to it’s old value (or null, if none was defined)

What about thread safety?

Well, dynamic variables usually follow the same semantics as the old Lisp-machines: a variables is only accessible from the same thread where it was defined. This can be easily implemented with thread-local storage. As for ASP.NET, the HttpContext.Request.Items makes a perfect host for keeping the values. So yes, they are thread-safe and, with minor adjustments, can be made ASP.NET-safe.

There are plenty of cases where a dynamic variable can prove useful, but the context example is the most common of all. A lot of times you’re gonna need to store some data somewhere and access it down the stack and OOP is going to be awkward.

Here’s DynVar.cs:

using System;
using System.Collections.Generic;
using System.Text;
using System.Threading;

namespace Test
{
    public class DynVar<T>
    {
        public static void Set(T value, Action<T> scope)
        {
            string name = scope.Method.GetParameters()[0].Name;
            Set(value, name, scope);
        }

        public static void Set(T value, string name, Action<T> scope)
        {
            var slot = Thread.GetNamedDataSlot(name);
            Stack<T> values = Thread.GetData(slot) as Stack<T>;
            if (values == null) {
                values = new Stack<T>();
                Thread.SetData(slot, values);
            }
            values.Push(value);
            try {
                scope(value);
            } finally {
                values.Pop();
                Thread.SetData(slot, values);
            }
        }

        public static void Use(Action<T> action)
        {
            string name = action.Method.GetParameters()[0].Name;
            Use(name, action);
        }

        public static void Use(string name, Action<T> action)
        {
            var slot = Thread.GetNamedDataSlot(name);
            Stack<T> values = Thread.GetData(slot) as Stack<T>;
            if (values == null || values.Count == 0)
                action(default(T));
            else
                action(values.Peek());
        }
    }
}

ccnet and svn revision

August 6, 2007

UPDATE: This is old, obsolete and incomplete stuff that I’ve “ported” from my old blog for (in)completeness. If you’re still interested in this please send me an email or leave a comment.

In the previous post about CruiseControl.NET I’ve showed how to make the list of modifications into links to the trac diff page for each file. If you looked carefuly at the screenshot you could also see that the SVN revision number appears and is a link. How did that get there and where does it lead? No better place than the Trac page for the latest changeset so you can view all the changes at once.

To do it you need to modify an the header.xsl file in the ccnet dashboard. You can find it in the webdashboard\xsl subdirectory where you installed CruiseControl.NET. Open it and insert the following:

<tr>
  <td class="header-label" valign="top"><nobr>SVN Revision:</nobr></td>
  <td class="header-data">
    <a href="http://192.168.0.250/trac/{/cruisecontrol/@project}/changeset/{changeNumber}"><xsl:value-of select="changeNumber" />
    </a>
  </td>
</tr>

right after the last <tr> element in the file. You should get something like this:

<!-- Last Modification template -->
<xsl:template match="/cruisecontrol/modifications/modification">
  <xsl:if test="position() = 1">
    <tr>
      <td class="header-label"><nobr>Last changed:</nobr></td>
      <td class="header-data"><xsl:value-of select="date"/></td>
    </tr>
    <tr>
      <td class="header-label" valign="top"><nobr>Last log entry:</nobr></td>
      <td class="header-data"><pre><xsl:value-of select="comment"/></pre></td>
    </tr>
    <tr>
      <td class="header-label" valign="top"><nobr>SVN Revision:</nobr></td>
      <td class="header-data"><a href="http://192.168.0.250/trac/{/cruisecontrol/@project}/changeset/{changeNumber}"><xsl:value-of select="changeNumber" /></a></td>
    </tr>
  </xsl:if>
</xsl:template>

Not only can you now see the svn revision number for the last build but you can also go the the diff page in its Trac repo. Yay!

Note: This will only work if the project has the same name in both CruiseControl.NET and Trac.


CruiseControl.NET + Trac integration

July 23, 2007

UPDATE: This is old, obsolete and incomplete stuff that I’ve “ported” from my old blog for (in)completeness. If you’re still interested in this please send me an email or leave a comment.

While reading the CruiseControl.NET documentation when I saw that you can integrate WebSVN with subversion. This will create a link to WebSVN for each source modification.

Now, since I’m already using trac , wouldn’t it be great to integrate it instead of WebSVN?

I should be able to specify only the url of the trac enviroment and the base path on the subversion repo (because sharing a is nice), sort of like this:

<webUrlBuilder type="trac">
  <tracProjectUrl>http://192.168.1.240/projects/MyProject</tracProjectUrl>
  <tracRepositoryRoot>/MyProject</tracRepositoryRoot>
</webUrlBuilder>

Thankfully, CruiseControl.NET is open source. By looking at how WebSVNUrlBuilder is implemented I’ve found out that I need to:

  • create a TracWebUrlBuilder class that implements IModificationUrlBuilder
  • add 2 two properties, TracProjectUrl and TracRepositoryRoot
  • decorate the class and methods with attributes so they are correctly recognized by NetReflector

But how can you convince CruiseControl to load an assembly? It turns out it’s quite easy. CruiseControl will load any assembly named ccnet.XXXXXXXX.plugin.dll so we simply need to name the assembly ccnet.TracWebUrlBuilder.plugin.dll and place it in the directory where CCService is located – usually the ‘server’ subdirectory where ccnet is installed.

Now, when I click a link in the modifications I’m redirected to the diff for the specified revision/file:

This is nice….

Download the project file here
To compile add a reference to NetReflector.dll and ThoughtWorks.CruiseControl.Core.dll


Major.Minor.CCLabel.svn revision

June 7, 2007

A few days ago the team decided that we needed versioning for the application we’re working on. We wanted the version to include the build number from the build server and the svn revision number. We figured this would make it a lot easier to track bugs.

Frist off, how can you do this from NAnt ? Actually, getting the build number is easy since CruiseControl.NET passes a bunch of properties to NAnt, including CCNetLabel. If you’re using the default labeler than this is exactly the build number.

Fot the svn version number all the solutions we could find were sort of ugly hacks, so we decided to write a bunch of functions that you could call in nant and get the revision numbers. This again turned out to be fairly easy.

Here’s the code:

using System;
using System.Text;
using System.Diagnostics;
using System.IO;
using System.Text.RegularExpressions;
using System.ComponentModel;

using NAnt.Core;
using NAnt.Core.Types;
using NAnt.Core.Attributes;
using System.Collections.Generic;

namespace Nant.SvnFunctions
{
	///
	/// Provides methods for working with SVN
	///
	[FunctionSet("svn", "SourceControl")]
	public class SvnFunctions : FunctionSetBase
	{
		#region Public Instance Constructors

		public SvnFunctions(Project project, PropertyDictionary properties) : base(project, properties) { }

		#endregion Public Instance Constructors

		#region Public Instance Methods

		///
		/// Gets the revision number for the Subversion repository at the specified path
		///
		/// The path to a svn repository
		/// The revision number of the specified repository
		[Function("get-revision-number")]
		public static int GetRevisionNumber(string path)
		{
			return int.Parse(GetInfo(path)["Revision"]);
		}

		///
		/// Gets an url containing the root of the specified repository
		///
		/// The path to a svn repository
		/// The root url of the repository
		[Function("get-repository-root")]
		public static string GetRepositoryRoot(string path)
		{
			return GetInfo(path)["Repository Root"];
		}

		///
		/// Gets the url of the specified repository
		///
		/// The path to a svn repository
		/// The url of the repository
		[Function("get-repository-url")]
		public static string GetRepositoryUrl(string path)
		{
			return GetInfo(path)["URL"];
		}

		///
		/// Gets the name of the person who made the last change in the repository
		///
		/// The path to a svn repository
		/// The name of the last committer
		[Function("get-last-changed-author")]
		public static string GetLastChangedAuthor(string path)
		{
			return GetInfo(path)["Last Changed Author"];
		}

		///
		/// Gets the number of the latest revision in which the  has changed.
		///
		/// The path to check
		/// The latest revision number in which the file or directory specified in  has changed.
		[Function("get-last-changed-rev")]
		public static int GetLastChangedRev(string path)
		{
			return int.Parse(GetInfo(path)["Last Changed Rev"]);
		}

		#endregion Public Instance Methods

		#region Private Instance Methods

		private static Dictionary GetInfo(string path)
		{
			ProcessStartInfo pinfo = new ProcessStartInfo("svn.exe");
			pinfo.Arguments = "info " + path;
			pinfo.UseShellExecute = false;
			pinfo.RedirectStandardOutput = true;

			Process svnProcess;
			try
			{
				svnProcess = Process.Start(pinfo);
			}
			catch (Win32Exception win32Ex)
			{
				throw new Exception("Exception caught executing process: most probably 'svn.exe' was not found", win32Ex);
			}

			string output = svnProcess.StandardOutput.ReadToEnd();

			if (svnProcess.ExitCode == 9009 &amp;&amp; output.Contains("is not a working copy"))
				throw new ArgumentException("The specified path is not a working copy", "path");
			else if (svnProcess.ExitCode != 0)
				throw new Exception("Failed while running 'svn.exe'. Error code " + svnProcess.ExitCode);

			svnProcess.Dispose();
			return ParseOutput(output);
		}

		private static Dictionary ParseOutput(string output)
		{
			Dictionary values = new Dictionary();
			Regex regex = new Regex("^(?[^:]*):(?.*)$", RegexOptions.Multiline);
			foreach (Match match in regex.Matches(output))
				values[match.Groups["key"].Value] = match.Groups["value"].Value.Trim();

			return values;
		}

		#endregion Private Instance Methods
	}
}

Compile this with a reference to NAnt.Core.dll and then load it in the build file like this:

<loadtasks assembly="../lib/Nant.SvnFunctions.dll"/>

You could alternatively dump it in the tasks folder in NAnt to make it available to all projects.

The functions exposed by it are:

  • svn::get-revision-number(path)
  • svn::get-repository-root(path)
  • svn::get-repository-url(path)
  • svn::get-last-changed-author(path)

Now to actually set the version numbers we use the asminfo nant task. This generates all the AssemblyInfo.cs files with the desired attributes set. In our case:

<asminfo output="src/AssemblyInfo.cs" language="CSharp">
    <attributes>
        <attribute
            type="System.Reflection.AssemblyVersionAttribute"
            value="3.0.${CCNetLabel}.${svn::get-revision-number('..')}"/>
    </attributes>
</asminfo>

So now we have pretty version number on all the dlls.

Note: none of us has a lot of experience with neither NAnt nor CruiseControl.NET. If there’s an easier or better way to do this please tell.

Update: Sean Chambers added username/password authentication support. Download here that repo’s dead


MS Connect

May 28, 2007

First off, I just want to say a few things. I use Firefox with a myriad of extensions on top of it. I like it a lot and I am very annoyed when “someone” makes me use “some other” browser. I have used IETab in the past as a compromise but fortunately a couple of months ago the MSDN has become bearable in Firefox so I uninstalled it.

A while ago Scott Guthrie Scott Guthrie posted about some ASP.NET hotfixes. I ran into some of the issues described myself (especially the recompile issue) so I though I should install them.

First off, these fixes are available only through Microsoft Connect. So they want better statistics on who’s using what patch. No big deal, I’ll just enter my “Passport”. I login, I go to the download page, I press “Download”. A bold text on a red (actually pinkish) background appears that says “File transfer is starting. For Windows users, a separate File Transfer Manager will start.”. WTF. After a second or so a Download File dialog appears in Firefox with the name “downloaddetails.aspx”.. hmmm.. not cool… Still, I save the aspx but to no luck. The file was just 1k in size containing no obvious real URL.

So I start IE. I go to the download page and click download and a friggin’ ActiveX Controls wants to install on my machine. Yes.. the above mentioned download manager. Nice. I install the damn thing and get my update.

To summarize the experience:

  • no Firefox support
  • no Firefox error handling (downloaddetails.aspx !?)
  • must install ActiveX in order to download a 2.15 MB file.
  • the managed download worked with the astounding speed of about 30KB/s

So.. maybe I’ll reinstall IETab.


peeves.1

May 5, 2007

Can anyone find a good reason to why Visual Studio 2005 throws

throw new Exception("The method or operation is not implemented.");

instead of

throw new NotImplementedException();

when generating method stubs ?


dynamic languages at MIX 07

May 1, 2007

When Sun announced support for dynamic languages (more here ) everyone expected Microsoft to come up with something similar. And they sure did. Yesterday at MIX’07 the DLR (Dynamic-Language-Runtime) was announced. But it gets better. Among the list of languages that will make use of this is IronRuby (more on InfoQ ), a Ruby implementation for the CLR (or should I say DLR?). See more on Jim Hugunin’s blog here . If you want to play with the Alpha version of the DLR right now then go download IronPython 2.0 Alpha from here .


Covering it up with a designer

April 29, 2007

The first announcement made about LINQ made a lot of people hope that maybe the guys in Redmond finally realized the benefits of working with strong, type safe CLR objects. Until then the only data abstraction solution available from MS were the Typed Datasets, which not only are a leaky abstraction and buggy as hell (see this for example) but also have a major disadvantage: is you decide to use them you might as well tie ball and chain to you ankle with “Visual Studio 2005” written on it. Then the technologies that were gonna be using LINQ were announced, like LINQ to SQL (DLINQ back then), LINQ to Datasets, LINQ to Entities, etc. (Maybe it’s just me, but I can’t figure out a single reason to why you’d need two solutions to map your data to your objects. Why isn’t one enough? And no, the argument that you need a simple one and a complex one doesn’t cut it for me.)

Unfortunately a lot of people complained about the complexity of the Entity Framework, it’s dependency to VS, it’s limitations and let’s not forget using no less than 3 XMLs to define the mapping, among which Ayende (here and here ) and Jeffrey Palermo (here).

As a response to all the gathered feedback the ADO.NET Team recently announced on their blog :

We’ve collected great feedback from you in these early releases, the most significant being that we need better tools to aid in defining a data model and mapping that model to the database. Although we have tools for generating a direct mapping to the storage schema, the real power of the ADO.NET Entity Framework comes in its ability to flexibly map a variety of relational schema representations to a more appropriate conceptual application model. This has reinforced the need to have a graphical designer experience available for the ADO.NET Entity Framework.

This is not the kind of reaction I was hoping for :( . Why do I have the feeling that instead of focusing on the simplicity of the framework the ADO.NET team tries to cover it up with a fancy designer?

Sam Gentile wrote about why you would choose the Entity Framework over NHibernate, but I don’t do consulting work, I don’t work for a Fortune 500 company and, to my knowledge, I’m not dumbed down by visual development. As a developer in a small, somewhat independent software company I would not like to be forced to use a framework just because of the brand name.

I’m really hoping that Linq-for-NHibernate will stabilize before Entity Framework’s release. This will certainly make NHibernate much more attractive. Either way, I guess we’ll wait until next year to see what happens. Until then, happy hibernating!


Programmers and Developers

March 22, 2007

“A programmer”. If you now build software than this is probably what you wanted to be at the time you started learning Pascal, or Basic, or even hacking some assembler. No matter what language you first used programming seemed cool. At least to me it did. I was hooked. Yet if someone asks me what my job is I unconditionally respond “Software Developer”. The more experienced ones sometimes say “Team Leader” or “Software Architect”.

A heart surgeon with 20 years of experience is still “a doctor” and he has no problem with being referred to that way. Yet software people somehow feel insulted when you call them “programmers”.

This happens because the general idea is that coding is a lower, simpler part of building software; to do it right you need to know design patterns, frameworks, use code generators, validation engines, rules engines, workflow engines, etc. That’s all nice and dandy, but people seem to forget that they still need to write a lot of code no matter how fancy their job title is. All the extra stuff will help make a better product and improve quality (and add business value, and stuff, whatever), but it’s the actual coding that creates the product.

In the big (a.k.a enterprisey) software world the focus has been mainly on improving the big picture. This has reached monstrous proportions in the Java world where the universal answer to any problem is (no, not 42) to build a new framework that does a little better than others at some small and insignificant aspect. But no matter how much stuff you’re gonna use to make the big picture brighter you’re still gonna get down in the mud and write code. Too few efforts have been made to actually fix or improve the language and the base library. .NET and C# have been a wake-up call for Sun and I can only hope the “language wars” will last.

Eric Raymond said:

Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot.

I believe this is true for almost any language. Even learning just a new idiom will sometimes make you want to apply it to the language you use every day. To become a good software developer you not only need to learn how to serialize workflows and stuff like that but you also want to learn to code better, smarter, cleaner, and usually there are better ways to do this than what we’re accustomed to.

So learn a new programming language, even an obscure one will do, see what functional programming is about and how dynamic typing can help and sooner or later this will make you a better developer.

Feed the programmer in you!