Asynchronous Batch Process Solution Revisited – part 1

I had some free time and decided to review the Asynchronous Batch Process Pattern Solution. On January 2013 I promised to publish a revised version and didn’t manage to do so. To redeem myself, I have decided to publish this version on CodePlex, so it can be customized by anyone who needs to.
If you don’t know what ABPP is all about and what’s in it for you, I suggest you start here.

In the first part of this post I’ll describe the new features, architecture considerations and implementation details. In the next part, I’ll walkthrough the different scenarios and use cases.

You can download an unmanaged solution as well as source code here.
As always, I advise not to publish an external solution on your production environment without testing it first. I tested this solution with Online (2015 RU 0.1) and on-premise 2015 deployments.

 

Following are some of the features included in the revised solution (version 2.0):

  1. Microsoft Dynamics CRM 2015/Online supported solution
  2. Action support – allow mapping to Action/Workflow Rule (sync./a-sync.) process as an Action component
  3. Integration scenarios support – activate Action process without specifying target records. This allows scheduling activation of any external action (like posting/retrieving data to/from an external application or data source) which can be wrapped with Custom Workflow Activity component
  4. Aggregative FetchXML query support – schedule execution of a an aggregative query (multiple aggregation support) and map aggregation result(s) to target Action as an input parameter
  5. Hourly scheduling  frequency support
  6. Diagnostics – update batch process record with last activation result (success/failure) and failure reason.
  7. Fetchxml 5000 records limit override – use conversion to Query Expression in order to handle queries that exceed 5000 records
  8. Activation counter – additional attribute that automatically increment with each successful Batch Process activation. This allows adding a workflow step or Rule that will suspend the Batch Process scheduling after X activations
  9. Zombie process prevention – scheduling activation only for future dates to prevent spawning of Workflow instances for past dates that will never occur

I have skipped some features that I could not design without breaking the ABPP paradigm, which generally dictates a complete separation between the Schedule component, the Action component and the target business records.
These features need some more thinking and if you have any suggested architecture, tell me about it.
The features I skipped:

  1. Centralized error handling – since the Action component is oblivious to its activator, execution errors (especially with a-sync. process Action handlers) cannot be easily related to the process activation
  2. Detailed Tracing – similarly to error handling, recording detailed tracing information such as total number of failure/success cases or processed records ids is a challenge due to the ABPP paradigm

Now, let’s get down to the details.

First, a small addition to the ERD. The new Process Activation entity purpose is to document each activation details.
In the future, I’ll add option to document the ids of the records which were returned by the Target Records Query.
This will allow tracing of the specific records that were affected by each ABP activation.

image

Currently, the Process Activation entity contains these custom attributes:

  1. Parent Batch Process id
  2. Activation start date
  3. Activation end date
  4. Total number of processed business records

Next, the main algorithm diagram:

Algorithm

Asynchronous Batch Process Solution Revisited – part 1

I had some free time and decided to review the Asynchronous Batch Process Pattern Solution. On January 2013 I promised to publish a revised version and didn’t manage to do so. To redeem myself, I have decided to publish this version on CodePlex, so it can be customized by anyone who needs to.
If you don’t know what ABPP is all about and what’s in it for you, I suggest you start here.

In the first part of this post I’ll describe the new features, architecture considerations and implementation details. In the next part, I’ll walkthrough the different scenarios and use cases.

You can download an unmanaged solution as well as source code here.
As always, I advise not to publish an external solution on your production environment without testing it first. I tested this solution with Online (2015 RU 0.1) and on-premise 2015 deployments.

Following are some of the features included in the revised solution (version 2.0):

  1. Microsoft Dynamics CRM 2015/Online supported solution
  2. Action support – allow mapping to Action/Workflow Rule (sync./a-sync.) process as an Action component
  3. Integration scenarios support – activate Action process without specifying target records. This allows scheduling activation of any external action (like posting/retrieving data to/from an external application or data source) which can be wrapped with Custom Workflow Activity component
  4. Aggregative FetchXML query support – schedule execution of a an aggregative query (multiple aggregation support) and map aggregation result(s) to target Action as an input parameter
  5. Hourly scheduling  frequency support
  6. Diagnostics – update batch process record with last activation result (success/failure) and failure reason.
  7. Fetchxml 5000 records limit override – use conversion to Query Expression in order to handle queries that exceed 5000 records
  8. Activation counter – additional attribute that automatically increment with each successful Batch Process activation. This allows adding a workflow step or Rule that will suspend the Batch Process scheduling after X activations
  9. Zombie process prevention – scheduling activation only for future dates to prevent spawning of Workflow instances for past dates that will never occur

I have skipped some features that I could not design without breaking the ABPP paradigm, which generally dictates a complete separation between the Schedule component, the Action component and the target business records.
These features need some more thinking and if you have any suggested architecture, tell me about it.
The features I skipped:

  1. Centralized error handling – since the Action component is oblivious to its activator, execution errors (especially with a-sync. process Action handlers) cannot be easily related to the process activation
  2. Detailed Tracing – similarly to error handling, recording detailed tracing information such as total number of failure/success cases or processed records ids is a challenge due to the ABPP paradigm

Now, let’s get down to the details.

First, a small addition to the ERD. The new Process Activation entity purpose is to document each activation details.
In the future, I’ll add option to document the ids of the records which were returned by the Target Records Query.
This will allow tracing of the specific records that were affected by each ABP activation.

image_thumb.png

Currently, the Process Activation entity contains these custom attributes:

  1. Parent Batch Process id
  2. Activation start date
  3. Activation end date
  4. Total number of processed business records

Next, the main algorithm diagram:

Algorithm

Calling Action with Sdk.Soap.js

Microsoft Dynamics CRM 2013/2015 Action mechanism is a basic building block in the SPoI (Single Point of Implementation) approach:

  1. Define Action business logic using native Workflow Steps
  2. If required, complement the Action business logic with Custom Workflow Activity components
  3. Activate the Action from server and client side code

Activating an Action from Client Side is made easy with Sdk.Soap.js infrastructure. here is a short walkthrough:

  1. Define & Activate Action

    Go to Settings –> Processes and define a new Action. When you are done, activate it.

    My sample Action, named e4d_HandleNewLead, handles new Lead process with the following stages:

    1. Creation of new Lead record
    2. Assignment of the Lead record to the proper Team according to Lead’s number of employees
    3. Sending an acknowledgments message to the Lead’s email

    The Action’s input parameters are:

    1. First name
    2. Last name
    3. Subject
    4. Email address
    5. Number of employees
    6. Industry code

    The Action’s only output parameter is the Lead record URL.

    Action details

    Action details

  2. Generate Action class

    1. Download the Sdk.Soap.js Action Message Generator.zip and extract it
    2. Download the Sdk.Soap.js library.zip and extract it
    2. Open the VS solution and click F5 to run it
    3. Select the target Organization or feed in a new one
    4. Wait for the program to complete and go to the …\bin\Debug\Messages\vsdoc folder to make sure your Action class has been created successfully
  3. Add required references

    1. Add Sdk.Soap.min.js and your Action minified file into your MSCRM Solution. My Solution prefix is ‘e4d’.
    2. Add script reference to the Sdk.Soap.min.js library into your HTML/Entity form libraries from the ..\Sdk.Soap.js\C#\Sdk.Soap.js folder
    2. Add script reference to your minified Action class into your HTML/Entity form libraries 
    3. Add references to .vsdocs versions of  your libraries to the HTML/JS files to get IntelliSense support

                script references

    script references

  4. Consume Action

1. Declare an Action request object and initiate with required parameters values
2. Execute the request into a response variable and access the response parameters

consume action

Make sure you deploy the minified files in your Solution while using the .vsdocs version in your IDE.  

Microsoft Dynamics CRM 2013 Custom Action – A Single Implementation Point

Microsoft Dynamics CRM 2013 introduced a new member in the Processes family: Custom Actions.
I see great potential in this new feature, especially as it can be easily executed (synchronously or a-synchronously) from client side code and return results. While Synchronous Workflow can also be executed from client side code, it can’t return results in an elegant manner.

This Custom Action capability enables both server side and client side to share a single implementation point. Here is an example:

Lets say that whenever a Contact record is created, the government id number must be validated (9 digits that adhere to a specific algorithm). Contacts are created manually by users but also programmatically as part of a scheduled integration with an external application.

User experience considerations dictate a client side validation, as we want to warn the user about an invalid value when he changes the field value rather than when when saving the record.
But what about the records created programmatically? We have to add another point of implementation in the form of a Plugin or a Synchronous Workflow to support that.
So we end up with two implementation points, both JavaScript code and a server side component code. This means more development and maintenance efforts.

Let’s review a different solution approach using Custom Action:

  1. Develop a ‘IDValidation’ Custom Workflow Activity to preform the required validation and return a Boolean (valid/invalid) result.
  2. Wrap the IDValidation’  Custom Workflow Activity with a Custom Action and call the Custom Action from JavaScript onchange event (see Andrii’s useful solution here).
  3. Call the Custom Action directly from your server side integration code or wrap the Custom Workflow Activity with a Synchronous Workflow to support records created programmatically

This approach uses a single custom code component consumed both from server and client side, therefor reducing development and maintenance efforts.