Friday, August 12, 2016

MethodImplOptions.AggressiveInlining - When Performance is Critical (1)

Background

As a C# (.NET) developer, I'm constantly being reminded that it's more important to prioritize code maintainability over performance optimizations. The argument is usually that most optimizations we do is usually premature and therefore unnecessary.



But recently I began working on a C# implementation of an algorithm for network motif discovery and all those good programming principles failed me. The first implementation was not only consuming too much memory (over 2GB withing 2 minutes of execution), it was taking too long (over 5 hours for my E. Coli test network) to execute. That was totally unacceptable, so I knew it was time to put Performance where it belongs - the first place.

In this series, I will be sharing some of the more interesting optimizations I did to bring my program to execute within milliseconds with very little memory footprint. This particular piece is dedicated to Method Inlining.

Method Inlining

Method calls come at a cost. This cost can usually be avoided by inlining - writing the code directly instead of factoring it into a method. However, methods make our codes more readable and maintainable. Besides, this so-called cost is totally insignificant in a vast majority of program requirements. In some cases, the compiler does the inlining for you too.

However, from .NET 4.5 Microsoft provided us a means to explicitly ask the compiler to inline a method. That means we can still write our codes normally and simply instruct the compiler to inline it. It's achieved by adding the following attribute to the concerned method.

[MethodImpl(MethodImplOptions.AggressiveInlining)]


You'll need to add the following using statement:

using System.Runtime.CompilerServices;


I added this on a few of the methods that are frequently called and my program execution became twice as fast. However, it doesn't seem to work as well on extension methods.

Try it out and let me know your experience.

Thursday, August 4, 2016

A New Multi-Tenancy Framework for Building Software-as-a-Service (SaaS)

The goal of this project is to provide the framework needed to build multi-tenant .NET applications while using your preferred choice of ORM (NHibernate, EntityFramework, Dapper etc), IoC Container (SimpleInjector, AutoFac, StructureMap, Unity, etc) and web frameworks (MVC, NancyFx). We provide the basic and generic functions needed in typical SaaS solutions, like authentication, authorization, custom URL, data separation etc., thus freeing you up to focus solely on building your product.

The diagram below shows the framework architecture and will help in understanding how things (ought to) connect to each other.
Architecture Diagram for MultiTenancyFramework



A full documentation is provided through the project's Wiki page and by downloading the sample projects provided.

The source code is hosted on Github

Thursday, July 7, 2016

Cannot call action method on controller because the action method is a generic method

I have been living my life writing web apps with MVC5 until I ran into this weird problem. It happened because I put a generic method in my controller (I had a base controller, but that's by the way). I implemented some functionalities using Command Pattern and thought I could retrieve my command handler within a controller using the simple call:

var handler = CommandHandler<TCommand>();
handler.Handle(command);

Well, the error above happened and I ended up with a solution where I now call

var processor = MyIoCContainer.GetInstance<ICommandProcessor>();
processor.Process(command);

It's an extra level of indirection, but sire, it worked perfectly! Here is how I arrived at that:

Suppose, following the Command Pattern, you have this handler interface:

public interface ICommandHandler<TCommand> where TCommand : ICommand //ICommand is a marker interface
{
    void Handle(TCommand command);
}

You define the ICommandProcessor interface:

public interface ICommandProcessor
{
    void Process(ICommand command);
}

with a corresponding implementation:

public sealed class CommandProcessor : ICommandProcessor
{
    private IServiceProvider _serviceProvider;
    public CommandProcessor(IServiceProvider serviceProvider)
    {
        _serviceProvider = serviceProvider;
    }

    public void Process(ICommand command)
    {
        var handlerType =
            typeof(ICommandHandler<>).MakeGenericType(command.GetType());

        dynamic handler = _serviceProvider.GetService(handlerType);

        handler.Handle((dynamic)command);
    }
}

IServiceProvider will normally be supplied using Constructor Injection using your favorite IoC container. Using SimpleInjector, the registration of IServiceProvider will look like this:

Container.Register(typeof(IServiceProvider), () => Container);

That's it. All you need do henceforth is implement handlers for each of your commands. Like this:

public class ComputeResultCommandHandler : ICommandHandler<ComputeResultCommand>
{
    // NB: ComputeResultCommand class should implement ICommand marker interface
    public void Handle(ComputeResultCommand command) 
    {
        // Your code here
    }
}

Any good IoC container should be able to register open generics by scanning through assemblies. Thus, once configured appropriately, you do not need to explicitly register the implementations for the handler interface.

Now to something a little more interesting

The handler and processor above can be modified to return a value instead of void. That will basically transform the Command pattern used here to what is normally referred to as Query Object pattern. I'll still retain the name 'Command' and just show the modifications.

First, the marker interface becomes

public interface ICommand<TResult>
{
}

and the handler:

public interface ICommandHandler<TCommand, TResult> where TCommand : ICommand<TResult>
{
    TResult Handle(TCommand command);
}

The processor interface becomes (unsurprisingly):

public interface ICommandProcessor
{
    TResult Process<TResult>(ICommand<TResult> command);
}

and its implementation:

public sealed class CommandProcessor : ICommandProcessor
{
    private IServiceProvider _serviceProvider;
    public CommandProcessor(IServiceProvider serviceProvider)
    {
        _serviceProvider = serviceProvider;
    }

    public TResult Process(ICommand<TResult> command)
    {
        var handlerType =
            typeof(ICommandHandler<,>).MakeGenericType(command.GetType(), typeof(TResult));

        dynamic handler = _serviceProvider.GetService(handlerType);

        return handler.Handle((dynamic)command);
    }
}

That's it!

I will be interested in learning other solutions to this problem. Of course I googled it before honing my own solution. My search landed me to this StackOverflow question which was asked in 2010 and on MVC2. The provided solution suggested either implementing your own ActionDescriptor of writing a new ControllerFactory where some magic code was written to circumvent MVC's default behaviour since there were no MVC methods to override. Besides not being comfortable with messing around with the controller, I wasn't sure how things might have changed between MVC2 and MVC5.

I've tailored the thoughts in this blog post to answer the question there in case some other person runs into a similar problem.

That said, I'll love to hear your thoughts.

Saturday, March 5, 2016

I'm Excited About ASP.NET Core 1.0 (Former ASP.NET 5)

I love C# and .NET Framework - no doubt! If you've ever worked with C#/.NET, you won't want to use any other framework in your life. It's one language I know that was developed with the developer in mind. In fact, with so much abstraction built in, coding becomes something everyone can do with relative ease.

However, the fact that .NET framework is not cross-platform has always given me worries. Not because Microsoft platforms aren't rich enough - in fact, their platforms make life so much easier for you when you want to support your code in production environment; but because Windows hosting costs so much, compared to non-Windows (e.g. Linux) hosting. I understand that the reason is because of license fees for Windows OS, SQL Server, etc., but then, my pocket (and that of some of my clients) don't have the brains to understand that.

My alternative was to code in Java, PHP or other open source cross-platform languages, but having done C# for years, those languages appeared tortuous, verbose and inelegant, with so many disconnected bits and pieces of libraries scattered all around the internet (Well, Maven seems to address problem).

So, when I heard that Microsoft is working on ASP.NET 5 which will be both open source and cross platform, I was overjoyed! Sadly, shortly after announcing ASP.NET 5 RC 1 Update 1, Scott Hanselman declared the ASP.NET 5 project dead, and introduced the new ASP.NET Core 1.0 and .NET Core 1.0. Clearly that has set us a few months behind, but then I believe it's worth the wait.

ASP.NET Core 1.0

This means that in no time, I will be able to build cross-platform apps without having to switch from one language (and IDE) to another! Sure, the first release of .NET Core 1.0 will not be as feature rich as .NET Framework 4.6; it's cool to see the direction Microsoft is going.

As an aside, I now have a good reason to migrate from NHibernate to Entity Framework. NHibernate was so good to me, and I'll surely miss it. But EF7 has become a man, and it's time to consolidate all my development platforms into one.

Friday, March 4, 2016

C# - How To Get Logged In User's Username and Track Activities from Windows Service

If you've ever looked at Windows Event Logs, you must have noticed that it logs too many information, most of which are not useful when you're actually trying to do some kind of forensics. To reduce this noise, I went to build a proof-of-concept Windows Service whose only task is to log users' session activities (Lock, Unlock, Logout, Login, etc) in a different place, thus giving me a much smaller log file to dig through.

Typically, the main class in a Windows Service project is the class that inherits from ServiceBase. Luckily, ServiceBase has an override-able method that can be made to fire when a user's session activity changes. Add the code below in the class to override the method

protected override void OnSessionChange(SessionChangeDescription changeDescription)
{
    //TODO: 
}

Unfortunately, the only information about the logged-on user that the incoming parameter carries is the Session ID, which is just an integer. To be useful, we have to find a way to get the username corresponding to that ID. In earlier versions of Windows, we would have easily written the code to run the powershell quser command to get us what we want; but that command is no longer available in Windows 10. To make matters a little worse, there is no (at least, I wasn't able to find any) .NET API to call to retrieve that info. So, I was left with no other option than to go old-school.

Add this code to the class inheriting from ServiceBase

[DllImport("Wtsapi32.dll")]
private static extern bool WTSQuerySessionInformation(IntPtr hServer, int sessionId, WtsInfoClass wtsInfoClass, out IntPtr ppBuffer, out int pBytesReturned);
[DllImport("Wtsapi32.dll")]
private static extern void WTSFreeMemory(IntPtr pointer);

private enum WtsInfoClass
{
    WTSUserName = 5, 
    WTSDomainName = 7,
}

private static string GetUsername(int sessionId, bool prependDomain = true)
{
    IntPtr buffer;
    int strLen;
    string username = "SYSTEM";
    if (WTSQuerySessionInformation(IntPtr.Zero, sessionId, WtsInfoClass.WTSUserName, out buffer, out strLen) && strLen > 1)
    {
        username = Marshal.PtrToStringAnsi(buffer);
        WTSFreeMemory(buffer);
        if (prependDomain)
        {
            if (WTSQuerySessionInformation(IntPtr.Zero, sessionId, WtsInfoClass.WTSDomainName, out buffer, out strLen) && strLen > 1)
            {
                username = Marshal.PtrToStringAnsi(buffer) + "\\" + username;
                WTSFreeMemory(buffer);
            }
        }
    }
    return username;
}
If you don't have one already, add a constructor to the class; and add this line to it:
CanHandleSessionChangeEvent = true;
Now you're good to go! Access the username from the overridden method like this:
protected override void OnSessionChange(SessionChangeDescription changeDescription)
{
    string username = GetUsername(changeDescription.SessionId);
    //continue with any other thing you wish to do
}
- If you find this useful, let's hear from you.

Friday, January 22, 2016

How to Migrate Your Data From Shared Hosting Server To Your System

Sometime ago, I built an ASP.NET site for a client and hosted it on GoDaddy public shared server. Soon enough, the need arose to back up all data generated periodically. Even though GoDaddy offers database backup, we did not have extra cash for that, so I had to innovate, creating a solution that connects to the server and brings down my data from some or all my tables whenever I want them, at no cost. The rest of the post is a summary of what I did. Use it as a guide if you are in a situation similar to the one I described above.

NB: For this project, I worked with ASP.NET/C#, NHibernate as ORM and SQL Server, the database engine. Consequently, I assume you are already familiar with those tools.
  1. Create a Console Application project.
  2. Add two configuration files to the project. I call them DataFrom.config and DataTo.config. These config files will only contain the usual NHibernate xml configuration settings, the difference in the two files being the value connection string.node. DataFrom.config should point to the database on the shared hosting server, and DataTo.config to the database on your system you wish to store your backup. (You will create it later... or now, if you like)
  3. Create a new Class Library project. I'll call it DataMigration. The reason is you'll typically need to use this solution in other settings, so it's best to build it independent (to a great extent) of any specific app.
  4. Create an Interface, IMigrator and add the following code to it.
    using NHibernate;
    namespace DataMigration
    {
        public interface IMigrator
        {
            /// 
            /// Move data from one DB to another
            /// 
            /// Session for the DB on the shared server
            /// Session for the DB on your system
            void Migrate(ISession fromSession, ISession toSession);
        }
    }
    
  5. Create a class called Initialiser. We'll write the code to initialize session factories for the two databases and return the two sessions we'll use for the job. The code will look like this:
    using System;
    
    namespace DataMigration
    {
        public class Initialiser
        {
            /// 
            /// I have assumed you have a way to create your session factory as the details of that is 
            /// beyond the scope of this post. The important thing is to create them and return their 
            /// corresponding sessions. My own way is represented by the class NHibernateSessionManager
            /// 
            /// 
            /// 
            public static void Init(out ISession fromSession, out ISession toSession)
            {
                var baseDir = AppDomain.CurrentDomain.BaseDirectory;
                var fromConfigFile = baseDir + "DataFrom.config";
                var toConfigFile = baseDir + "DataTo.config";
    
                var fromDatasourceKey = "fromdatasource";
                var toDatasourceKey = "todatasource";
    
                //Sesson Factory - To
                Console.WriteLine("Initializing 'To' session factory...");
                NHibernateSessionManager.Init(toConfigFile, toDatasourceKey);
    
                //Sesson Factory - From
                Console.WriteLine("Initializing 'From' session factory...");
                NHibernateSessionManager.Init(fromConfigFile, fromDatasourceKey);
    
                Console.WriteLine("Initializing 'From' session...");
                fromSession = NHibernateSessionManager.GetSession(fromDatasourceKey);
       
                Console.WriteLine("Initializing 'To' session...");
                toSession = NHibernateSessionManager.GetSession(toDatasourceKey);
    
                Console.WriteLine("Done Initializing!");
            }
        }
    }
    
    
  6. Create a class called Runner. This will host two important methods Run and MoveData. Run initializes our session factories, gets our two sessions for us and then call IMigrator's Migrate method. MoveData is a generic method that does the actual migration. It basically uses fromSession to retrieve all records from a table (represented by the type T) and uses toSession to flush it into the corresponding table in our local database. Here's the code:
    using NHibernate;
    using System;
    using System.Collections;
    using System.Collections.Generic;
    
    namespace DataMigration
    {
        /// 
        /// We use this to move data from one DB to another; PARTICULARLY from public shared server where we have the public connectionstring
        /// to our own system so we can hold a backup of the live data
        /// 
        public class Runner
        {
            /// 
            /// This is the method to be called by our Console Application to start the migration job
            /// 
            public static void Run(IMigrator migrator)
            {
                Console.WriteLine("Commencing Migration!");
                ISession fromSession, toSession;
                try
                {
                    Initialiser.Init(customMappingAssemblies, out fromSession, out toSession, autoPersistenceModel);
    
                    migrator.Migrate(fromSession, toSession);
                    fromSession.Close();
                    fromSession.Dispose();
                    toSession.Close();
                    toSession.Dispose();
                }
                catch (Exception ex)
                {
                    Console.WriteLine(ex);
                    Console.WriteLine();
                }
                Console.WriteLine("Done Migration!");
            }
    
            /// 
            /// This method does the actual migration job
            /// 
            public static void MoveData<T>(ISession fromSession, ISession toSession, string tableName = null, string entityName = null) where T : class
            {
                var typeNameInPlural = typeof(T).Name.ToPlural();
                IList<T> records;
                if (!string.IsNullOrWhiteSpace(entityName))
                {
                    if (string.IsNullOrWhiteSpace(tableName)) throw new ArgumentNullException("tableName", "This should not be null");
                    typeNameInPlural = entityName;
                    Console.WriteLine("Migrating {0}...", typeNameInPlural);
                    records = fromSession.QueryOver<T>(entityName).List();
                }
                else
                {
                    Console.WriteLine("Migrating {0}...", typeNameInPlural);
                    if (string.IsNullOrWhiteSpace(tableName)) tableName = typeNameInPlural;
                    records = fromSession.QueryOver<T>().List();
                }
    
                if (records != null && records.Count > 0)
                {
                    Console.WriteLine("{0} retrieved. About to flush them into the new database...", typeNameInPlural);
                    SqlManipulations.SqlBulkInsert<T>(records, toSession.Connection, tableName, entityName);
                    Console.WriteLine("Done flushing {0} into the new database...", typeNameInPlural);
                }
                else
                {
                    Console.WriteLine("No data found for {0}", typeNameInPlural);
                }
            }
        }
    }
    
    
    Two things to note: (1) As a convention, my table name is the pluralised form of my class name; hence I've "written" an extension method, ToPlural, to convert strings to plural. However, MoveData still allows you to specify custom table names. (2) The implementation of the generic method SqlManipulations.SqlBulkInsert is not shown as it's outside the scope of this post. I may write another post to describe that in detail.
  7. Build this library and reference it in the console app. You will also need to reference the library containing the entity definitions, NHibernate libraries and all other relevant ones you may need.
  8. In the console app, create a class, called Migrator to implement our IMigrator interface. The implementation of the Migrate method will be merely a call to Runner's MoveData method. Here's the code:
    using NHibernate;
    using DataMigration;
    
    namespace DataMigrationConsole
    {
        class Migrator : IMigrator
        {
            public void Migrate(ISession fromSession, ISession toSession)
            {
                // For each of our entities, we'll call MoveData in a similar way
                // Here, we demonstrate with just two entities
                Runner.MoveData<UserRole>(fromSession, toSession); //table name left out
                Console.WriteLine("Done UserRoles \n");
                Runner.MoveData<User>(fromSession, toSession, "Users"); //table name supplied
                Console.WriteLine("Done Users \n");
            }
        }
    }
    
  9. Create an empty database and use your ORM's schema update feature to generate the database tables. This way, you are guaranteed to get the exact schema you have on the server.
  10. Finally we modify the Console App's Main method and run the program. The Main method will look like this:
    static void Main(string[] args)
    {
        Runner.Run(new Migrator()); //This is all we need to call
    
        Console.WriteLine("\n***End of Program***");
        Console.ReadKey();
    }
    

Further Improvements

Suppose you have a large number of entities in your project, using the code in No. 8 will quickly become unpleasant. You can solve this problem by modifying Migrator and Runner classes.

For Migrator class, provide a way to supply the entities you're interested in - either by listing them directly or by supplying the assembly where they reside. Here's the modified version:
using NHibernate;
using DataMigration;
using System.Reflection;
 
namespace DataMigrationConsole
{
    class Migrator : IMigrator
    {
        Type[] _relevantEntities;

        /// 
        /// Constructor
        /// 
        /// The assembly where the Entity Definitions were created
        public Migrator(string assemblyName)
        {
            // This code will load all the classes in this assembly. However, you can filter out your entities by
            // either making them implement an interface, then filter out all types that implement that interface;
            // or by decorating your entity classes with an Attribute, and filtering by that attribute.
            _relevantEntities = Assembly.Load(assemblyName).GetTypes();
        }

        /// 
        /// Constructor
        /// 
        /// The names of the classes  you want to migrate their data
        public Migrator(string[] entityClassNames)
        {
            var theTypes = new List<Type>();
            foreach (var entityName in entityClassNames)
            {
                var type = Type.GetType(entityName, false);
                if (type != null)
                {
                    theTypes.Add(type);
                }
            }
            _relevantEntities = theTypes.ToArray();
        }
  
        public void Migrate(ISession fromSession, ISession toSession)
        {
            foreach (var entityType in _relevantEntities)
            {
                var tableName = entityType.Name.ToPlural();
                Runner.MoveData(entityType, fromSession, toSession, tableName);
                Console.WriteLine("Done {0} \n", tableName);
            }
        }
    }
}

With this, you can now modify Main method to use the constructor that suits you best.

But wait: did you notice that I used a different overload of Runner's MoveData? The implementation for this overload is shown below. Notice that I now used NHibernate session's CreateCriteria instead of QueryOver. Notice also that our SqlBulkInsert method had to change:
public static void MoveData(Type entityType, ISession fromSession, ISession toSession, string tableName = null, string entityName = null) where T : class
{
    var typeNameInPlural = entityType.Name.ToPlural();
    IList records;
    if (!string.IsNullOrWhiteSpace(entityName))
    {
        if (string.IsNullOrWhiteSpace(tableName)) throw new ArgumentNullException("tableName", "This should not be null");
        typeNameInPlural = entityName;
        Console.WriteLine("Migrating {0}...", typeNameInPlural);
        records = fromSession.CreateCriteria(entityName).List();
    }
    else
    {
        Console.WriteLine("Migrating {0}...", typeNameInPlural);
        if (string.IsNullOrWhiteSpace(tableName)) tableName = typeNameInPlural;
        records = fromSession.CreateCriteria(entityType).List();
    }

    if (records != null && records.Count > 0)
    {
        Console.WriteLine("{0} retrieved. About to flush them into the new database...", typeNameInPlural);
        SqlManipulations.SqlBulkInsert(entityType, records, toSession.Connection, tableName, entityName);
        Console.WriteLine("Done flushing {0} into the new database...", typeNameInPlural);
    }
    else
    {
        Console.WriteLine("No data found for {0}", typeNameInPlural);
    }
}
That's all. Quite easy, isn't it?

Share your thoughts and feedback in the Comments section below.