Plug-in ServiceEndPoint Integration – Minimize IPluginExecutionContext payload

Integrating Microsoft Dynamics 365 with Azure Service Bus (ASB) can be easily implemented via the Plug-in Registration Tool without writing any code.
With this integration method, each time the target business event occurs, the complete operation IPluginExecutionContext payload is automatically sent to ASB, though often you need only a small portion of the data.

For some integration scenarios you need to apply conditional logic to decide if and which ASB queue to target. For these scenarios, you can write custom code which allows you to programmatically post message to ASB from Plug-in/Custom Workflow Activity components.
This method, also allows minimizing the IPluginExecutionContext payload which can simplify the logic required to parse it and generally improve performance. Unfortunately, I could not find a way to reduce the payload to just the required data, only minimize it.

The following screenshots (ASB Explorer) describes 2 similar messages (Contact creation event) posted to ASB before (4656 bytes) and after minimizing the IPluginExecutionContext payload (1842 bytes).
As you can see, the full payload weighs ~2.5 times the minimized payload.


To minimize the IPluginExecutionContext payload while maintaining the data you need, follow these simple steps in your code:

  1. Extract the required data from IPluginExecutionContext and add it to the SharedVariables collection
  2. Clear the data heavy IPluginExecutionContext collections
    • InputParameters
    • OutputParameters
    • PreEntityImages
    • PostEntityImages

The following sample code extracts the lastname attribute from the IPluginExecutionContext, clears the redundant collections and posts the minimized context to ASB:

        public void Execute(IServiceProvider serviceProvider)
        {
            // Retrieve the execution context.
            IPluginExecutionContext context = 
                (IPluginExecutionContext)serviceProvider.GetService(typeof(IPluginExecutionContext));

            IServiceEndpointNotificationService cloudService = 
                (IServiceEndpointNotificationService)serviceProvider.GetService(typeof(IServiceEndpointNotificationService));

            // Extract the tracing service.
            ITracingService tracingService = (ITracingService)serviceProvider.GetService(typeof(ITracingService));

            //extract target record data from InputParameters collection
            Entity contact = (Entity)context.InputParameters["Target"];
            //extract required data 
            string fullName = contact["lastname"].ToString();

            //add required data to the SharedVariables collection
            context.SharedVariables.Add("lastname", fullName);

            //clear redundant data from context 
            context.PreEntityImages.Clear();
            context.PostEntityImages.Clear();           
            context.InputParameters.Clear();
            context.OutputParameters.Clear();
            
            try
            {
                tracingService.Trace("Posting the execution context.");

                string response = cloudService.Execute(new EntityReference("serviceendpoint", 
                    serviceEndpointId),
                    context);
                
                if (!String.IsNullOrEmpty(response))
                {
                    tracingService.Trace("Response = {0}", response);
                }

                tracingService.Trace("Done.");
            }
            catch (Exception e)
            {
                tracingService.Trace("Exception: {0}", e.ToString());
                throw;
            }
        }
    }

Sandbox Execution Timeout

While reviewing existing Microsoft Dynamics 365 Online implementations, I have noticed a repeating assumption by which Custom Workflow Activity, executing within an a-synchronous process, is not subjected to the sandbox 2 minutes execution timeout limitation.
This false assumption led to wrong architectural designs in which long running processes are assigned to be handled by CWA components.

As this article states, Custom Workflow Activity code registered to work in sandbox mode is subjected to the 2 min. execution timeout limit, just like Plug-in code. If you want to test this declaration yourself, just put a Sleep statement in your CWA code and get the following exception for both sync. and a-sync. processes:

The plug-in execution failed because the operation has timed-out at the Sandbox Host

So here are my tips regarding the Sandbox timeout limitation:

  1. Plan for big scales: You may have hundreds of records to process on day 1, but in a year or two, you might have thousands and your CWA/Plug-in component will start hitting the execution timeout limit.
  2. Use the right tool: Both CWA and Plug-in components are not meant for heavy lifting. For potentially long running processes or complex computations, these mechanisms should be used only as event wrappers, triggered by a business event and passing on execution to a more robust and manageable mechanisms such as Azure App Services, Azure Functions or other external mechanisms.

  3. Handle all exceptions: You can easily hit the the timeout limit, especially when integrating external services which you have no control of and may not response in a timely manner.
    In general, If you don’t handle exceptions, your business logic may disgracefully break and your users/clients might experience some ugly ass raw exceptions.
    While handling exceptions, you can single out timeout exception by testing the exception status:
    catch (WebException exception)
    {
    string str = string.Empty;
    if (exception.Response != null)
    {
    using (StreamReader reader = new StreamReader(exception.Response.GetResponseStream()))
    {
    str = reader.ReadToEnd();
    }
    exception.Response.Close();
    }
    if (exception.Status == WebExceptionStatus.Timeout)
    {
    throw new InvalidPluginExecutionException(
    “The timeout elapsed while attempting to process the request.”, exception);
    }
    throw new InvalidPluginExecutionException(String.Format(CultureInfo.InvariantCulture,
    “A Web exception occurred while attempting to process the request. {0}: {1}”, exception.Message, str), exception);
    }
  4. Diagnose: You can diagnose your CWA/Plugin components by using the Trace service in your code. Tracing can be easily switched to display all tracing, just error or turned off by setting the Plug-in and custom workflow activity tracing in the Organization Settings dialog and then viewed in the Plug-in Trace Log area.
    Tracing all incoming parameters values (as this article suggests) can help you better diagnose your CWA operations.

    Plug-in and custom workflow activity tracing in the Organization Settings dialog
     Plug-in Trace Log area

  5. Monitor: Sandbox Run time statistics can help you monitor your Plug-in/CWA performance over time, failures and running time metrics.
    Run the following FetchXML query to get statistics about your component. Replace the name attribute value with your component name.
    Note the averageexecutetimeinmilliseconds and failurecount results – it might indicate that your business logic is failing or close to failure.

    <fetch>
    <entity name=”plugintypestatistic”>
    <attribute name=”averageexecutetimeinmilliseconds” />
    <attribute name=”failurecount” />
    <attribute name=”failurepercent” />
    <attribute name=”crashpercent” />
    <attribute name=”crashcount” />
    <link-entity name=”plugintype” from=”plugintypeid” alias=”PluginType” to=”plugintypeid”>
    <attribute name=”isworkflowactivity”></attribute>
    <attribute name=”typename”></attribute>
    <filter>
    <condition attribute=”name” operator=”eq” value=”MyProject.Workflows.ProcessSomething“></condition>
    </filter>
    </link-entity>
    </entity>
    </fetch>

Plug-in Configuration Manager Utility

Plug-in components often require external configuration settings. Maybe the Plug-in code consumes a web service which end point alternates between test and production environments or maybe you use the Plug-in configuration to turn logging on and off. Basically, any code setting which depend upon external resources is worth exporting to external configuration as it may prevent code re-compilation, additional testing etc.

There are some common approaches to implement configuration settings for Plug-in components:

  1. Configuration file: each Plug-in instance consumes a configuration file
  2. Configuration record: each Plug-in instance executes a query to retrieve a designated Microsoft Dynamics CRM configuration record
  3. Step configuration: each Plug-in instance constructor receives configuration settings from the Step secure/unsecure configuration

Performance wise, option 3 is preferred as the Step’s secure/unsecure configuration is cached alongside the Plug-in step and no additional operation is required to retrieve it. Changing the Step’s secure/unsecure configuration will automatically un-cache and reload relevant Plug-in with updated configuration. If you are not familiar with this option implementation, read this.
Options 1 and 2 require an expensive IO operation or query as each Plug-in instance explicitly consumes additional resources to retrieve configuration settings. When one of these approaches are applied in large scale (or with poor server resources), it will take a considerable toll on server performance and sometimes, the UX.

While option 3 yields better performance, it is not maintenance friendly. In enterprise scale applications, you may have to go through hundreds of Plug-in steps to update secure/unsecure configuration settings. Now think about doing this while deploying a version to the Production environment a 3 AM.

In this post, I offer a utility which allows you to enjoy Step configuration performance benefit along with simple maintenance for configuration bound Plug-in components.

WIIFM?

An unmanaged solution can be downloaded here. Use this tool as is or change the code to better suit you needs.
Although this tool was designed and built for Microsoft Dynamics CRM 2016 (Online/On-premise), the approach behind it can be implemented with previous versions, probably down to 2011.

As displayed below, the Plug-in Configuration Utility allows you select a registered Plug-in assembly, view related Steps and finally view each Step configuration details.
Clicking the ‘Update Configuration’ while a single or multiple Steps are selected will updated the respective configuration settings.

By default, Custom Work Flow Activities and System owned Plug-in assemblies are hidden, but can be revealed by unchecking the matching checkbox. You can view some interesting configuration settings under the Microsoft.Crm.ObjectModel assembly Steps, which are hidden when using the Plugin Registration Tool.
Although possible, do not change any configuration settings for these Steps.

Plugin Configuration Manager Demo

Bits & Bytes

The entities used behind the scenes here are

  1. pluginAssembly – representing registered Plug-in DLL file
  2. pluginType – representing IPlugin event handler class contained in the Plug-in DLL file
  3. sdkMessageProcessingStep – representing Step details binding the plugintype event handler to specific entity and message. Contains the unsecure configuration string
  4. sdkMessageProcessingStepSecureConfig – representing a secure configuration record related to the sdkmessageprocessingstep. Contains the secure configuration string

Plugin Configruation Entity Model

Although the Plug-in Registration Tool displays the secure configuration string as part of the Step settings (represented by the sdkMessageProcessingStep entity), this attribute actually resides in the related sdkMessageProcessingStepSecureConfig entity. Moving the secure configuration string to a different entity allows setting different privileges, as the secure configuration string may contain sensitive data like password.

This also means the in order to programmatically set new secure configuration string for a Step, it is required to create and relate a new sdkMessageProcessingStepSecureConfig  record.

Plugin Step Secure Configruation Privileges

Why Should You Always Debug with the Plugin Registration Tool

I have written posts on how to debug with the Plugin Registration Tool (PRT) and the PRT features in the past.
Working mainly with Online deployments lately, I gathered some insights I would like to share here.

First, some facts:

  • Debugging Online deployed Custom Workflow Activities (CWA) with the PRT is currently impossible
    This is sad, because there is no method to debug Online CWA with Visual Studio. The PRT is certainly purposed to help debugging CWA, but the functionality just doesn’t work.
    Additionally, current (v7.1.1) and former SDK versions do not describe CWA debugging procedure with the PRT, only the Plugin debugging procedure.
    I have approached MSCRM SDK team regarding these problems a few months ago and notified these issues will be addressed in future SDK versions.
    Until that happens, I develop and debug CWA in an on premise organization and then publish to the target Online organization.
  • The PRT is the only option to debug Online deployed Plugin components with Visual Studio debugger
    Microsoft Online deployments do not expose server processes, so VS Attach to Server Process technique is not viable.
    The fact that the PRT is an only option doesn’t mean it is a bad option. Actually, I find it superior to the Attach to Server Process technique in all aspects when it comes to debugging Plugin components, even in an on premise deployment. Read on to find out why.
  • Debugging CWA/Plugin components in on premise production deployment with Attach to Server Process technique is usually not viable
    Similarly to Online deployment, on premise production deployment do not expose server processes as this is considered a security breach.
    So in this scenario, the PRT is again the only option to debug CWA/Plugin components with the Visual Studio debugger.

Now, let’s compere PRT with the Attach to Server Process technique, which is the common alternative technique to debug Plugin/CWA components with VS:

  • Server Process Blocking:
    Debugging synchronous components is usually performed by attaching the Visual Studio debugger to the W3WP server process, which blocks the process. It means that while you are busy debugging, the entire application is unresponsive for any other users (developers, QA testers etc.). If you are working alone there is no problem but when other users are involved, this seriously affect their productivity, not to mention deeply annoying them. It also means other developers can’t debug synchronous components while you do.

    With the PRT technique, you attach the Visual Studio debugger to the PRT process, which runs on your dev machine. This means no server process is blocked while you debug and no one is planning to assassinate you.

  • DLL updating while debugging:
    While debugging with the Attach to Server Process technique, each code edit & compilation cycle requires updating the Plugin/CWA DLL on the server. While this process involves only a few mouse clicks, when done over and over again (common in debugging scenarios) it becomes a real time consumer and also takes your mental focus off the actual debugging process. Add to this the need to trigger the Plugin/CWA execution by using MSCRM UI (e.g. create or update a record) for each debugging cycle and watch your productivity rate flatline.

    With the PRT technique there is no need to deploy the DLL while debugging, as the PRT process is mapped to the DLL in your project BIN directory.
    No need to execute any UI triggering process since the context is packaged in the PRT debugging Profile.
    This is a MAJOR time saver.

  • Replay debugging scenario:
    The PRT has the magical Replay feature which the Attach to Server Process technique lacks.
    This feature allows you debug Production and inaccessible MSCRM deployments since the debugging context can be extracted (by system administrator) as a text file and used with the PRT on your dev machine assuming you have the relevant DLL.
    Off course, SDK calls in your code will not be executed in this scenario, but in many cases the debugging context is enough to trace the culprit.

Finally, I recommend using the ReAttach extension for Visual Studio coupled with the PRT to maximize the debugging process efficiency.

Debugging Server Side Components with ReAttach

Whether you are using the Plugin Registration Tool or Attach to server process technique to debug Plugin or Custom Workflow Activity components, you probably waste a LOT of time on the Attach To Process procedure in Visual Studio:

  1. Click Debug menu
  2. Click Attach To Process… menu option
  3. Find the target process in the processes list
  4. Click Attach

Preform this procedure over and over again for every debug iteration…not only this procedure is time consuming, it also takes your mental focus off the debugging process.

Attach To Process

 

I recently discovered the ReAttach extension for Visual Studio, which allows you to re-attach Visual Studio debugger to the last process it was attached to in one click of a button or a keyboard shortcut (Ctrl+R, Ctrl+A).
You can also select from a list of recently attached processes.

ReAttach

 

This a MAJOR productivity boost and it’s free. Give it a whirl, you won’t go back.

Debugging Workflow Custom Activities with Plugin Registration Tool

I have been using the Plugin Registration Tool to debug Plugins for sometime now and it has turned out to be a real time saver as it let’s you run and debug your code without having to trigger application events. This shortens debug cycles and allows Plugins debugging by multiple developers simultaneously without halting the W3WP process.

I have just found out the Plugin Registration Tool can also help debug Custom Workflow Activities as well. Since this feature is currently not documented in the SDK or any other official literature, this post will guide you through.

Please note, currently this feature is not functioning well in Online deployments, hope MS will fix this soon.

In order to debug a Custom Workflow Activity, follow these steps:

  1. Activate the Plugin Registration Tool as an administrator. Make sure you are using the latest Plugin Registration Tool version from the latest SDK package.
  2. Register your Custom Workflow Activity using the Plugin Registration Tool.
  3. Create a Workflow Rule which reference your Custom Workflow Activity.
  4. If you don’t have the Profiler installed, click the ‘Install Profiler’ button in the inner menu. After it finishes installing, the Plugin Registration Tool shows the ‘Uninstall Profiler Button’ and a new item ‘Plug-in Profiler’ is added to the registered components

    image

  5. Click the ‘Plug-in Profiler’ item. Note the ‘Profile Workflow’ button appearing in the inner top menu

    image

  6. Click the ‘Profile Workflow’ button to open the Profile Setting window. From the Workflow picklist, select the Workflow Rule which contains the Custom Workflow Activity requires debugging

    image

  7. Check the Custom Workflow Activity requires debugging. Select ‘Persist to Entity’ radio option and click ‘OK’. Copy the value In the Persistence Key field to your clipboard

    image

  8. Notice the Profiled item added under the Plug-in Profiler root

    image

  9. Go to your CRM organization and trigger the relevant Workflow. In my example, this is a manual Workflow rule.

    image

  10. Go to Settings –> Extensions –> Plug-in Profiles. If you can’t locate it, refresh the window.

    image

  11. Open the relevant Plug-in Profile record, make sure the Persistence Key matches the one you have copied earlier

    image

  12. Click the Serialized Profile tab and copy the value in the Serialized Profile filed

    image

  13. Create a new text file and copy the Serialized Profile string into it. Save the file
  14. Go Back to the Plugin Registration Tool, select the profile created earlier and click the Debug button in the inner toolbar

    image

  15. In the opened Debug window, select the newly created text file in the the Profile Location field and the Custom Workflow Activity assembly in the Workflow Activity field

    image

  16. Go to Visual Studio, where the Custom Workflow Activity project is opened. Compile your code and locate the Custom Workflow Activity assembly and PDB file in the “…\Program Files\Microsoft Dynamics CRM\Server\bin\assembly” folder
  17. Attach the debugger to the Plugin Registration Tool Process. Put some break point in your code and click the Start Execution button in the Plugin Registration Tool Debug Window.

You are done. The Breakpoint should be hit and from now on you can edit your code and debug locally against the Plugin Registration Tool.

Now go get productive.