Asynchronous Batch Process Solution Revisited – part 1

I had some free time and decided to review the Asynchronous Batch Process Pattern Solution. On January 2013 I promised to publish a revised version and didn’t manage to do so. To redeem myself, I have decided to publish this version on CodePlex, so it can be customized by anyone who needs to.
If you don’t know what ABPP is all about and what’s in it for you, I suggest you start here.

In the first part of this post I’ll describe the new features, architecture considerations and implementation details. In the next part, I’ll walkthrough the different scenarios and use cases.

You can download an unmanaged solution as well as source code here.
As always, I advise not to publish an external solution on your production environment without testing it first. I tested this solution with Online (2015 RU 0.1) and on-premise 2015 deployments.

 

Following are some of the features included in the revised solution (version 2.0):

  1. Microsoft Dynamics CRM 2015/Online supported solution
  2. Action support – allow mapping to Action/Workflow Rule (sync./a-sync.) process as an Action component
  3. Integration scenarios support – activate Action process without specifying target records. This allows scheduling activation of any external action (like posting/retrieving data to/from an external application or data source) which can be wrapped with Custom Workflow Activity component
  4. Aggregative FetchXML query support – schedule execution of a an aggregative query (multiple aggregation support) and map aggregation result(s) to target Action as an input parameter
  5. Hourly scheduling  frequency support
  6. Diagnostics – update batch process record with last activation result (success/failure) and failure reason.
  7. Fetchxml 5000 records limit override – use conversion to Query Expression in order to handle queries that exceed 5000 records
  8. Activation counter – additional attribute that automatically increment with each successful Batch Process activation. This allows adding a workflow step or Rule that will suspend the Batch Process scheduling after X activations
  9. Zombie process prevention – scheduling activation only for future dates to prevent spawning of Workflow instances for past dates that will never occur

I have skipped some features that I could not design without breaking the ABPP paradigm, which generally dictates a complete separation between the Schedule component, the Action component and the target business records.
These features need some more thinking and if you have any suggested architecture, tell me about it.
The features I skipped:

  1. Centralized error handling – since the Action component is oblivious to its activator, execution errors (especially with a-sync. process Action handlers) cannot be easily related to the process activation
  2. Detailed Tracing – similarly to error handling, recording detailed tracing information such as total number of failure/success cases or processed records ids is a challenge due to the ABPP paradigm

Now, let’s get down to the details.

First, a small addition to the ERD. The new Process Activation entity purpose is to document each activation details.
In the future, I’ll add option to document the ids of the records which were returned by the Target Records Query.
This will allow tracing of the specific records that were affected by each ABP activation.

image

Currently, the Process Activation entity contains these custom attributes:

  1. Parent Batch Process id
  2. Activation start date
  3. Activation end date
  4. Total number of processed business records

Next, the main algorithm diagram:

Algorithm

Asynchronous Batch Process Solution Revisited – part 1

I had some free time and decided to review the Asynchronous Batch Process Pattern Solution. On January 2013 I promised to publish a revised version and didn’t manage to do so. To redeem myself, I have decided to publish this version on CodePlex, so it can be customized by anyone who needs to.
If you don’t know what ABPP is all about and what’s in it for you, I suggest you start here.

In the first part of this post I’ll describe the new features, architecture considerations and implementation details. In the next part, I’ll walkthrough the different scenarios and use cases.

You can download an unmanaged solution as well as source code here.
As always, I advise not to publish an external solution on your production environment without testing it first. I tested this solution with Online (2015 RU 0.1) and on-premise 2015 deployments.

Following are some of the features included in the revised solution (version 2.0):

  1. Microsoft Dynamics CRM 2015/Online supported solution
  2. Action support – allow mapping to Action/Workflow Rule (sync./a-sync.) process as an Action component
  3. Integration scenarios support – activate Action process without specifying target records. This allows scheduling activation of any external action (like posting/retrieving data to/from an external application or data source) which can be wrapped with Custom Workflow Activity component
  4. Aggregative FetchXML query support – schedule execution of a an aggregative query (multiple aggregation support) and map aggregation result(s) to target Action as an input parameter
  5. Hourly scheduling  frequency support
  6. Diagnostics – update batch process record with last activation result (success/failure) and failure reason.
  7. Fetchxml 5000 records limit override – use conversion to Query Expression in order to handle queries that exceed 5000 records
  8. Activation counter – additional attribute that automatically increment with each successful Batch Process activation. This allows adding a workflow step or Rule that will suspend the Batch Process scheduling after X activations
  9. Zombie process prevention – scheduling activation only for future dates to prevent spawning of Workflow instances for past dates that will never occur

I have skipped some features that I could not design without breaking the ABPP paradigm, which generally dictates a complete separation between the Schedule component, the Action component and the target business records.
These features need some more thinking and if you have any suggested architecture, tell me about it.
The features I skipped:

  1. Centralized error handling – since the Action component is oblivious to its activator, execution errors (especially with a-sync. process Action handlers) cannot be easily related to the process activation
  2. Detailed Tracing – similarly to error handling, recording detailed tracing information such as total number of failure/success cases or processed records ids is a challenge due to the ABPP paradigm

Now, let’s get down to the details.

First, a small addition to the ERD. The new Process Activation entity purpose is to document each activation details.
In the future, I’ll add option to document the ids of the records which were returned by the Target Records Query.
This will allow tracing of the specific records that were affected by each ABP activation.

image_thumb.png

Currently, the Process Activation entity contains these custom attributes:

  1. Parent Batch Process id
  2. Activation start date
  3. Activation end date
  4. Total number of processed business records

Next, the main algorithm diagram:

Algorithm

Sharing Business Records via Processes in Microsoft Dynamics CRM

Microsoft Dynamics CRM 2013 Added new Workflow based mechanisms: Custom Action and Real Time (synchronous) Workflow. Along with Dialogs and Background (a-synchronous) Workflow, these mechanisms render the already powerful Processes family into a versatile implementation suite which allows implementing complex business logic in online and on-premise deployments. Some business logic which in version 2011 was only achievable via custom code, can now be achieved via declarative Process.

This versatility increases the ROI on Custom Workflow Activities components development as these components, which complete the capabilities of the Processes family via code, can now be reused in even more business scenarios by non developers.

One functionality is painfully missing in the Processes basic toolbox: the Share functionality. This may derive from the potential negative performance impact related to record Sharing in large scales which must be taken into consideration when planning an implementation.
It is recommended to share records with Teams rather than with single Users, and also to revoke shared privileges when ever possible.

To supply the required Share functionality via Processes, a Custom Workflow Activity can be used. Custom Workflow Activity is a DLL developed in .NET and deployed to Microsoft Dynamics CRM organization. It allows extending the Process capabilities with custom functionality. Once developed and deployed, this component can be used in Dialogs, Workflows (a-synchronous and synchronous) and Custom Actions in various business scenarios for different entities.

In order to Share a business record with a User or Team, some input details are required. These details are mapped by the user defining the Process hence the Custom Workflow Activity component must support matching input parameters:

  • Target record – which business record is to be shared?
  • Target principal – with what User/Team is the target record to be shared?
  • Share privileges – what privileges (read/write/delete etc.) are to be granted to the Target Principal regarding the Target record?

There are some challenges in developing a generic Custom Workflow Activity component, which I will discuss next.

One major challenge is making this component generic enough to handle any entity that supports the Share operation.
This challenge derives from the Custom Workflow Activity input parameter types, which sadly do not include the Entity class. The supported EntityReference parameter type requires the ReferenceTarget attribute, which bind it to a specific entity type. So using EntityReference does not achieve the required generic solution.

My solution is to let the user (who defines the Process) define the target entity schema name as an input parameter of string type. This approach puts the responsibility of getting the entity schema name on the user defining the Process. I trust anyone who is can define a Process is capable of correctly copying the entity schema name from the entity customization page.

Another challenge is getting the id of the target business record. As this component is required to operate on any sharable entity, the user defining the process must be able to point to any business record of any entity. Again, the Custom Workflow Activity input parameter types limit us.

The solution here relies on the Workflow ability to supply the URL of any business entity as part of the attributes exposed in the Workflow editor. As this URL eventually contains the record GUID (unique record id), it can be used to supply the required target business record id.

The rest of the input parameters are more trivial. As records can be shared with either User or Team, two EntityReference input parameters allow specifying the target principal respectively. The privileges input parameter allows the user to specify the required privileges via a simple string.

The same approach can be implemented in version 2011 as well.

Untitled

Debugging Workflow Custom Activities with Plugin Registration Tool

I have been using the Plugin Registration Tool to debug Plugins for sometime now and it has turned out to be a real time saver as it let’s you run and debug your code without having to trigger application events. This shortens debug cycles and allows Plugins debugging by multiple developers simultaneously without halting the W3WP process.

I have just found out the Plugin Registration Tool can also help debug Custom Workflow Activities as well. Since this feature is currently not documented in the SDK or any other official literature, this post will guide you through.

Please note, currently this feature is not functioning well in Online deployments, hope MS will fix this soon.

In order to debug a Custom Workflow Activity, follow these steps:

  1. Activate the Plugin Registration Tool as an administrator. Make sure you are using the latest Plugin Registration Tool version from the latest SDK package.
  2. Register your Custom Workflow Activity using the Plugin Registration Tool.
  3. Create a Workflow Rule which reference your Custom Workflow Activity.
  4. If you don’t have the Profiler installed, click the ‘Install Profiler’ button in the inner menu. After it finishes installing, the Plugin Registration Tool shows the ‘Uninstall Profiler Button’ and a new item ‘Plug-in Profiler’ is added to the registered components

    image

  5. Click the ‘Plug-in Profiler’ item. Note the ‘Profile Workflow’ button appearing in the inner top menu

    image

  6. Click the ‘Profile Workflow’ button to open the Profile Setting window. From the Workflow picklist, select the Workflow Rule which contains the Custom Workflow Activity requires debugging

    image

  7. Check the Custom Workflow Activity requires debugging. Select ‘Persist to Entity’ radio option and click ‘OK’. Copy the value In the Persistence Key field to your clipboard

    image

  8. Notice the Profiled item added under the Plug-in Profiler root

    image

  9. Go to your CRM organization and trigger the relevant Workflow. In my example, this is a manual Workflow rule.

    image

  10. Go to Settings –> Extensions –> Plug-in Profiles. If you can’t locate it, refresh the window.

    image

  11. Open the relevant Plug-in Profile record, make sure the Persistence Key matches the one you have copied earlier

    image

  12. Click the Serialized Profile tab and copy the value in the Serialized Profile filed

    image

  13. Create a new text file and copy the Serialized Profile string into it. Save the file
  14. Go Back to the Plugin Registration Tool, select the profile created earlier and click the Debug button in the inner toolbar

    image

  15. In the opened Debug window, select the newly created text file in the the Profile Location field and the Custom Workflow Activity assembly in the Workflow Activity field

    image

  16. Go to Visual Studio, where the Custom Workflow Activity project is opened. Compile your code and locate the Custom Workflow Activity assembly and PDB file in the “…\Program Files\Microsoft Dynamics CRM\Server\bin\assembly” folder
  17. Attach the debugger to the Plugin Registration Tool Process. Put some break point in your code and click the Start Execution button in the Plugin Registration Tool Debug Window.

You are done. The Breakpoint should be hit and from now on you can edit your code and debug locally against the Plugin Registration Tool.

Now go get productive.

Reviving Microsoft Dynamics CRM 2011 Activities using Process

Microsoft Dynamics CRM 2011 UI does not enable users to reactivate Activities. Once an activity such as Task is completed, it can not be re-opened, although users may add notes and attachments to it.

I can think of a business scenario in which a user is required to reactivate a Task. Maybe it is incomplete although reported as completed, or maybe reported completed by mistake.

In this scenario, a Process can be used as Processes can change completed Task state from ‘Completed’ to ‘Open‘. You can set a manual Workflow rule to do just that.

Task change State Workflow