Making use of transactions in Dynamics 365 Customer Engagement

I believe that if you have ever written a CRM plugin, you are aware of the event execution pipeline (https://msdn.microsoft.com/en-us/library/gg327941.aspx) and that the operation is run in transaction which means that if there will be any exception, the whole transaction will be rolled back. Pre-operation plugins and Post-Operation plugins are also run in transaction, so if there will be an exception raised by the plugin, changes will not be reflected in CRM. This includes not only chaning attributes for the entity that the plugin is running on – create/update/delete changes on any other entities are also rolled back. This is really convenient – we can perform a bunch of operations and we are sure that even if anything goes wrong during Post-Operation, everything will be changed back to the original state.

The usage of transactions can be embraced even more by using custom Actions and ExecuteTransactionRequest. Custom  Actions can also be run with a rollback option (which can be enabled or disabled on Action configuration page). ExecuteTransactionRequest allows to group different OrganizationRequests and send them in bulk to the server – they will be run in single transaction.

Ok, but how this transactional behaviour works in real world? To make it easier to understand for you I created a custom entity “new_lock” and prepared a very simple CRM plugin:

public void Execute(IServiceProvider serviceProvider)
{
    var factory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
    var service = factory.CreateOrganizationService(null);
    var @lock = new Entity("new_lock");
    @lock.Id = new Guid("590424F1-611F-E711-80D2-00155D010402");
    @lock["new_name"] = "lockme";
    service.Update(@lock);
    Thread.Sleep(30 * 1000);
}

I registered this plugin on Create message of the Contact entity. As you can see it simply retrieves the “new_lock” record and updates its name to some different value and then waits 30 seconds. Let’s open this record in a browser:

Now, let’s create a Contact (using a different browser window). After clicking “Save” you will notice that the loader starts but the data is not getting saved. That’s ok because our plugin is running in the background and it takes about 30 seconds to finish. If you try to save the opened “new_lock” record with some different name, you will notice that… you can’t do this:

This is the principal idea of transactions – by creating a Contact we opened a transaction and “locked” the “new_lock” record by updating it. Before the transaction is finished nothing can write to this record. After 30 seconds, our contact is created and the “new_lock” is updated with the new value. Is this a value from the plugin or from what we have entered on the form? Well it’s the second option, because we updated the value after the transaction has done it (it does not matter that it waited 30 seconds after that).

How can we make use of that transactional behaviour of CRM? I believe that a very cool feature that we can develop is proper auto-numbering. It’s pretty well known problem which is usually handled by some custom counter entity. The algorithm usually works like that:

  1. Run plugin on create of numbered entity
  2. Retrieve Counter
  3. Assign current value of counter to entity record
  4. Update counter with the next value
  5. Plugin ends

If you don’t have many users or some custom apps creating entities, you will probably never run into any concurrency problems. But imagine what will happen if you have two concurrent calls and both retrieved value of the counter before point 3 was fulfiled. This means that both records will have the same number and counter will be increased only by 1. Some may say – ok let’s use standard locking, we can have a static object to lock on and before plugin starts it’s core logic, the running thread must obtain the lock. That would work flawlessly on single-frontend-server environments. But what happens if there are more application servers for CRM? You cannot synchronize threads between different servers…

Knowing what we already know about transactions, we can easily modify this scenario to:

  1. Run plugin on create of numbered entity
  2. Retrieve Counter
  3. Update some dummy counter property to some dummy value (here we will lock the counter for usage only for this transaction)
  4. Retrieve Counter (now we are sure nobody will modify its value)
  5. Assign current value of counter to entity record
  6. Update counter with the next value
  7. Plugin ends (and any concurrent operations start from point 3)

This should work even on multi-server environments, because the transactions are locked on database level. Very simplified code (so pardon the style) can look like this:

public class TransactionPlugin : IPlugin
{
    public void Execute(IServiceProvider serviceProvider)
    {
        var context = (IPluginExecutionContext)serviceProvider.GetService(typeof(IPluginExecutionContext));
        var factory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
        var service = factory.CreateOrganizationService(null);

        var counter = GetCounter(service);
        counter["new_name"] = "lockme";
        service.Update(counter);

        //now counter is locked

        //we have to get it again to prevent race conditions
        counter = GetCounter(service);

        var target = (Entity)context.InputParameters["Target"];
        target["new_number"] = counter["new_nextnumber"];

        counter["new_nextnumber"] = (int)counter["new_nextnumber"] + 1;
        service.Update(counter);
        //after finishing the transaction, concurrent executes can go on from the lock
    }

    private Entity GetCounter(IOrganizationService service)
    {
        //return some counter
    }
}

 

UPDATE 21.07.2017

I prepared more detailed code for autonumbering plugin, which I have successfully used in my projects. The plugin works on Pre-Create of Account.


namespace TransactionPlugin
{
public class Autonumber : IPlugin
{
public void Execute(IServiceProvider serviceProvider)
{
var pluginContext = (IPluginExecutionContext)serviceProvider.GetService(typeof(IPluginExecutionContext));
var factory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
var service = factory.CreateOrganizationService(null);
var target = pluginContext.InputParameters["Target"] as Entity;
var qExpression = new QueryExpression("new_lock");
qExpression.ColumnSet = new ColumnSet("new_name");
qExpression.Criteria.AddCondition("new_name", ConditionOperator.Equal, "Counter");
var results = service.RetrieveMultiple(qExpression);
var counter = results.Entities.First();
var blocker = new Entity("new_lock");
blocker.Id = counter.Id;
blocker["new_name"] = "Counter";
service.Update(blocker); //NOW WE LOCK ALL TRANSACTIONS
var lockedCounter = service.Retrieve("new_lock", blocker.Id, new ColumnSet("new_lastnumber"));
var currentNumber = lockedCounter.GetAttributeValue<int>("new_lastnumber");
target["accountnumber"] = $"{++currentNumber}";
var counterUpdater = new Entity("new_lock");
counterUpdater.Id = counter.Id;
counterUpdater["new_lastnumber"] = currentNumber;
service.Update(counterUpdater);
}
}
}

view raw

Autonumber.cs

hosted with ❤ by GitHub

As you can see I’m retrieving my “new_lock” entity by its name using QueryExpression and after locking it with Update, I’m retrieving it again using Retrieve message. It is very important, because the following code will not work correctly:


namespace TransactionPlugin
{
public class Autonumber : IPlugin
{
public void Execute(IServiceProvider serviceProvider)
{
var pluginContext = (IPluginExecutionContext)serviceProvider.GetService(typeof(IPluginExecutionContext));
var factory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
var service = factory.CreateOrganizationService(null);
var target = pluginContext.InputParameters["Target"] as Entity;
var qExpression = new QueryExpression("new_lock");
qExpression.ColumnSet = new ColumnSet("new_name", "new_lastnumber");
qExpression.Criteria.AddCondition("new_name", ConditionOperator.Equal, "Counter");
var results = service.RetrieveMultiple(qExpression);
var counter = results.Entities.First();
var blocker = new Entity("new_lock");
blocker.Id = counter.Id;
blocker["new_name"] = "Counter";
service.Update(blocker);
var qExpression2 = new QueryExpression("new_lock");
qExpression2.ColumnSet = new ColumnSet("new_lastnumber");
qExpression2.Criteria.AddCondition("new_name", ConditionOperator.Equal, "Counter");
var lockedResults = service.RetrieveMultiple(qExpression2);
var lockedCounter = results.Entities.First();
var currentNumber = lockedCounter.GetAttributeValue<int>("new_lastnumber");
target["accountnumber"] = $"{++currentNumber}";
var counterUpdater = new Entity("new_lock");
counterUpdater.Id = counter.Id;
counterUpdater["new_lastnumber"] = currentNumber;
service.Update(counterUpdater);
}
}
}

If you try to run that, you will discover that you have a lot of duplicates. The reason is that the second RetrieveMultiple gets exactly the same value as the first one, so if we already obtained counter and some other thread will modify its value – this change will be not reflected. This is very important, so make sure you are not falling for this trap – for me this feature is quite surprising, QueryExpression results seem to be cached somehow in plugin context. What is even more interesting the following version will work flawlessly:


namespace TransactionPlugin
{
public class Autonumber : IPlugin
{
public void Execute(IServiceProvider serviceProvider)
{
var pluginContext = (IPluginExecutionContext)serviceProvider.GetService(typeof(IPluginExecutionContext));
var factory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
var service = factory.CreateOrganizationService(null);
var target = pluginContext.InputParameters["Target"] as Entity;
var qExpression = new QueryExpression("new_lock");
qExpression.ColumnSet = new ColumnSet("new_name", "new_lastnumber");
qExpression.Criteria.AddCondition("new_name", ConditionOperator.Equal, "Counter");
var results = service.RetrieveMultiple(qExpression);
var counter = results.Entities.First();
var blocker = new Entity("new_lock");
blocker.Id = counter.Id;
blocker["new_name"] = "Counter";
service.Update(blocker);
var qExpression2 = new QueryExpression("new_lock");
qExpression2.ColumnSet = new ColumnSet("new_lastnumber");
qExpression2.Criteria.AddCondition("new_name", ConditionOperator.Equal, "Counter");
var rmRequest = new RetrieveMultipleRequest();
rmRequest.Query = qExpression2;
var lockedResults = (RetrieveMultipleResponse)service.Execute(rmRequest);
var lockedCounter = lockedResults.EntityCollection.Entities.First();
var currentNumber = lockedCounter.GetAttributeValue<int>("new_lastnumber");
target["accountnumber"] = $"{++currentNumber}";
var counterUpdater = new Entity("new_lock");
counterUpdater.Id = counter.Id;
counterUpdater["new_lastnumber"] = currentNumber;
service.Update(counterUpdater);
}
}
}

So simply replacing service.RetrieveMultiple with RetrieveMultipleRequest causes the results to be retrieved from the database, not from some kind of cache. That is something you should probably be aware of when implementing your own solution.

4 thoughts on “Making use of transactions in Dynamics 365 Customer Engagement

  1. Hi Pawel,

    This approach still does not guarantee 100% uniqueness of “new_number”.

    A simple code which will cause duplicated entries (code simplified for readability):

    Thread thread1 = new Thread(new ThreadStart(doImport));
    Thread thread2 = new Thread(new ThreadStart(doImport));

    thread1.Start();
    thread2.Start();

    Where doImport does the following:

    public void DoImport()
    {
    for (int i = 0; i < 5; i++)
    {
    Account accToCreate = new Account();
    accToCreate.Name = "test";
    service.Create(accToCreate);
    }
    }

    I wasn't able to find the cause of this behviour as it works in most cases (regular system usage). Importing 1000 recods created <10 duplicates.

    Like

    • Hi PBK,

      Thank you for your comment. I investigated this more and updated my post with some more detailed code. I’m 100% sure it works and I tested it even with your application without making a single duplicate. As you can see I don’t even use the standard .NET “lock” statement in my code, so if the transactions would not work properly, I would end up with a lot of duplicates. I also posted some finding about QueryExpressions in plugins, please make sure that you are not a victim of this strange “caching” of RetrieveMultiple results.

      Like

  2. Yup. I can confirm, that using different retrieve works. Furthermore, if you are using XrmContext generated by svcutil, you should also explicitly call service.Retrieve for the second retrieve, otherwise you may get duplicates (even with MergeOption set to NoTracking).
    I find it odd, that using generated context the error does not happen always and is somehow “undeterministic”.

    Anyway, I haven’t tested the solution with load balancing, but for other scenarios it works.

    Like

Leave a comment