• Michael Lang
• | August 19, 2013
• | Azure, Business
• | 1 Comments
• | azure

Azure SDK extensions – unofficial

Microsoft Azure has a good quality SDK for interfacing with the Azure table storage REST HTTP interface. In the early days of Azure table storage you had to build your own query urls and process raw http rest style responses. Now with the Azure SDK 2.0 library we have .net code to interface with from our application. However, there is still room for improvement.

TableEntity

The Azure SDK can easily persist any object that derives from TableEntity. But that is a problem. I don’t want my persistence layer telling me what my business layer classes need to derive from, and I don’t want to have a separate data layer and business layer class all the time. Persistence libraries need to get over this pattern, and just let developers use their custom business layer entities.

The benefit of TableEntity is that it will serialize/deserialize all your public get/set properties when writing to/reading from the cloud. You can even customize this by overriding the WriteEntity and ReadEntity methods. Just watch out for the types of your properties. I learned the hard way that it doesn’t support some common framework types like decimal. The TableEntity class will silently skip persisting those property types.

public class Microsoft.WindowsAzure.Storage.Table.EntityProperty{
	//..
    internal static EntityProperty CreateEntityPropertyFromObject(object value, Type type)
    {
      if (type == typeof (string))
        return new EntityProperty((string) value);
	  //... snipped for brevity
      if (type == typeof (long?))
        return new EntityProperty((long?) value);
      else
        return (EntityProperty) null; // WTF !!  Where is my property value!
}

Honestly this should either throw an exception or serialize the value to a string. See TableEntityProxy for a solution to all these problems.

Enter TableEntityProxy<T>

You can see the full source code on github.

When I was working on my application I really couldn’t bare to define my business entities and also define them in my persistence layer and use AutoMapper to add pointless mappings. I was also frustrated with properties not having values when I loaded them from table storage.

The first obvious answer was to create a new generic base type that allowed any type that had a default constructor. Then store a property to reference the object to be stored or the object that was retrieved.

    public class TableEntityProxy<T> : TableEntity
        where T : class, new()
    {
        public TableEntityProxy()
        {
            Entity = new T();
        }
        public TableEntityProxy(T entity)
        {
            Entity = entity;
        }

        public T Entity { get; set; }

		// .. snipped
	}

Then the next step was to override the ReadEntity and WriteEntity methods of TableEntity. Parts of the implementation were copied from the Azure Storage SDK. The code is pretty good, and it just had a few failings to fix. Notably how should it write and read data types not supported in Azure. In addition to the TableEntity supported types, I decided to support any type that has a type converter supporting bi-directional conversion with a string. That includes all value types in the framework and many other custom types. This TableEntityProxy also supports nested objects. Just don’t give it an object with circular refernces within itself.

        protected void WriteEntityProperty(IDictionary<string, EntityProperty> properties,
            OperationContext operationContext, object propertyObject,
            PropertyInfo propertyInfo, String expressionChain)
        {
			// ... snipped common code that is Azure SDK
            if (propertyInfo.PropertyType.IsClass && propertyInfo.PropertyType != typeof(String))
            {	//support properties that are a class, by serializing all child properties, recursively
                if (propertyValue == null)
                    return;
                foreach (var subPropertyInfo in propertyInfo.PropertyType.GetProperties())
                {
                    WriteEntityProperty(properties, operationContext, propertyValue, subPropertyInfo,
                                        memberExpression);
                }
                return;
            }
			// ... snipped common code that is Azure SDK
		}
        private static EntityProperty CreateEntityPropertyFromObject(Type type, object value)
        {
            if (type == typeof(string))
                return new EntityProperty((string)value);
			//... snipped for brevity
            if (type == typeof(long?))
                return new EntityProperty((long?)value);

			//Here is the key difference to support other value types
            var typeConverter = TypeDescriptor.GetConverter(type);
            if (typeConverter.CanConvertTo(typeof(string)) && typeConverter.CanConvertFrom(typeof(string)))
                return new EntityProperty((string)typeConverter.ConvertTo(value, typeof(string)));

            throw new InvalidCastException(String.Format("Cannot cast property '{0}' into an EntityProperty to be persisted into Azure. Implement a TypeConverter for conversion to and from a String.", type.FullName));
        }

These changes also have corresponding reverse logic during ReadEntity

        protected void ReadEntityProperty(IDictionary<string, EntityProperty> properties,
            OperationContext operationContext, object propertyObject,
            PropertyInfo propertyInfo, String expressionChain)
        {
			// ... snipped common code that is Azure SDK
            if (propertyInfo.PropertyType.IsClass && propertyInfo.PropertyType != typeof(String))
            {	//support properties that are a class, by deserializing all child properties, recursively
                if (propertyValue == null)
                {
                    var constructor = propertyInfo.PropertyType.GetConstructor(Type.EmptyTypes);
                    if (constructor == null)
                        return;
                    propertyValue = Activator.CreateInstance(propertyInfo.PropertyType, null, null);
                    propertyInfo.SetValue(propertyObject, propertyValue, null);
                }
                if (!properties.Keys.Any(p => p.StartsWith(columnName, StringComparison.InvariantCultureIgnoreCase)))
                    return; //don't recurse if none of the nested properties have values in the record.
                foreach (var subPropertyInfo in propertyInfo.PropertyType.GetProperties())
                {
                    ReadEntityProperty(properties, operationContext, propertyValue, subPropertyInfo, memberExpression);
                }
                return;
            }

			// ... snipped more common code that is Azure SDK
            switch (entityProperty.PropertyType)
            {
                case EdmType.String: //includes if exists: 'PartitionKey', 'RowKey', 'ETag'
                    if (propertyInfo.PropertyType == typeof(string))
                        propertyInfo.SetValue(propertyObject, entityProperty.StringValue, null);
                    else
                    {	//Here is the key difference to support other value types
                        var typeConverter = TypeDescriptor.GetConverter(propertyInfo.PropertyType);
                        if (typeConverter.CanConvertFrom(typeof(string)))
                            propertyInfo.SetValue(propertyObject, typeConverter.ConvertFrom(entityProperty.StringValue));
                    }
                    return;
                case EdmType.Binary:
                    if (propertyInfo.PropertyType == typeof(byte[]))
                        propertyInfo.SetValue(propertyObject, entityProperty.BinaryValue, null);
                    return;
                case EdmType.Boolean:
                    if (propertyInfo.PropertyType == typeof(bool))
                        propertyInfo.SetValue(propertyObject, entityProperty.BooleanValue.HasValue ? entityProperty.BooleanValue.Value : default(bool), null);
                    else if (propertyInfo.PropertyType == typeof(bool?))
                        propertyInfo.SetValue(propertyObject, entityProperty.BooleanValue, null);
                    return;
				// ... snipped more common code that is Azure SDK
                default:
                    throw new InvalidCastException(String.Format("Cannot cast Azure table value into property '{0}'.", propertyInfo.Name));
			}
		}

CloudTable

Every example of using CloudTable on the internet starts the same. You have to get the storage account, get a cloud table client, and then get the cloud table. This doesn’t follow the DRY principle.

// Retrieve the storage account from the connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
    CloudConfigurationManager.GetSetting("StorageConnectionString"));

// Create the table client.
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();

// Create the table if it doesn't exist.
CloudTable table = tableClient.GetTableReference("people");
table.CreateIfNotExists();

Many times you want to do a simple task, like save an item or load an item by a combination of its key(s). The code to do this is rather verbose. Now there are cases when you will need to define your own TableQuery and combine many filters, but that is not the norm in my experience. You do need to know how to use TableQuery when you run into those cases, but that doesn’t mean you should have to use them for all your common use case queries also.

Here is the Azure official documentation to retrieve a single entity by its key. It is rather verbose.

// Retrieve the storage account from the connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
    CloudConfigurationManager.GetSetting("StorageConnectionString"));

// Create the table client.
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();

// Create the CloudTable object that represents the "people" table.
CloudTable table = tableClient.GetTableReference("people");

// Create a retrieve operation that takes a customer entity.
TableOperation retrieveOperation = TableOperation.Retrieve<CustomerEntity>("Smith", "Ben");

// Execute the retrieve operation.
TableResult retrievedResult = table.Execute(retrieveOperation);

// Print the phone number of the result.
if (retrievedResult.Result != null)
   Console.WriteLine(((CustomerEntity)retrievedResult.Result).PhoneNumber);
else
   Console.WriteLine("The phone number could not be retrieved.");

Enter CloudTableProxy<T>

All CloudTableProxy does it help you follow DRY. It wraps up the common opening of your CloudTable and build and execute your table operations. For the complex scenarios it lets you pass in a query as string, as defined with Azure SDK TableQuery. Using it is pretty simple.

using Candor.WindowsAzure.Storage.Table; //required for GetValidPartitionKey() and GetValidRowKey()

	public class AzureOrderRepository : OrderRepository{
		// ... snipped configuration code
		public override Order Get(String orderId)
        {
            var item = TableProxy.Get(orderId.GetValidPartitionKey(), orderId.GetValidRowKey());
            return item == null ? null : item.Entity;
        }
        public override void InsertOrUpdate(Order order)
        {
            TableProxy.InsertOrUpdate(order);
        }
	}

For that to work you have to do a one time setup of the TableProxy property based on a configured connection name. This initialization code tells the CloudTableProxy how to determine your partition and row keys given an instance of your class.

        public string ConnectionName
        {
            get { return _connectionName; }
            set
            {
                _connectionName = value;
                _tableProxy = null;
            }
        }
        private CloudTableProxy<Order> TableProxy
        {
            get
            {
                return _tableProxy ?? (_tableProxy = new CloudTableProxy<Order>
                    {
                        ConnectionName = ConnectionName,
                        PartitionKey = x => x.OrderId.GetValidPartitionKey(),
                        RowKey = x => x.OrderId.GetValidRowKey()
                    });
            }
        }

How do you get it?

You can get the source directly in Candor common, WindowsAzure namespace.

You can also find it on NuGet, using the Visual Studio package console Install-Package Candor.WindowsAzure.

More

Candor common has a similar proxy class for CloudQueue called CloudQueueProxy. It also has supplemental methods for putting record change notifications on a queue, so that you can consistently deserialize those from a worker role to trigger background work. There is a separate project in the solution for building a worker role task that can process those change notifications. Another project has a ‘common logging’ factory adapter to write log entries to an Azure table with some additional role information attached to each entry. You can find all of this in ‘candor-common’ on github.

References

How to use the Table Storage Service
http://www.windowsazure.com/en-us/develop/net/how-to-guides/table-services/

Candor.WindowsAzure source on Github
https://github.com/michael-lang/candor-common/tree/master/Candor.WindowsAzure