MSCRM On-Premises – the Rise and Fall of Rollups

MSCRM Rollups per Version

I recently made this diagram for a presentation regarding Microsoft Dynamics CRM 2016. Looking at it, I can’t help wondering about the future of Rollups as a package containing bug fixes and new features for On-premises deployments.

Microsoft wants you to migrate to the cloud. Microsoft Dynamics CRM Online offers many features which are not available for On-premises deployments. As with version 2015, some new features were deployed Online and were never made available for On-premises deployments.  If you wanted these features, you had to migrate Online or upgrade to version 2016.
In the last few years, Microsoft released a major version almost every year while the number of Rollups per version dropped from 18 (version 2011) to 1 (version 2015).

What does this mean for On-premise deployments? Are enterprise organizations expected to upgrade their Microsoft Dynamics CRM applications each year to get new and important features?

I think not. I’m guessing Microsoft will change its method for deploying new features for Microsoft Dynamics CRM.
Assuming Microsoft will keep offering the On-premises deployment type for a few more years, new features will not be packaged in Rollups anymore, but rather as smaller Solutions with longer backward compatibility span. I also expect major versions will become almost transparent for both Online and On-premises deployments. While Online Organization admins will have an option to activate these solutions, On-premises admins will be able to download and install when required.

Anyway, until we are all in cloud, Online deployment will always have a unique set of features.

Retrieving Audit History Records via API

Although the Audit History entity is somewhat degenerated, records for this entity can be extracted via multiple API mechanisms in both Online and on-premise scenarios:

  1. Organization Service query  (QueryExpression, QueryByAttribute, LINQ, FetchXML)
  2. OrganizationData Service query

The Audit History entity lacks a FilteredView element and therefore can not be retrieved from the Database in a supported manner. Also, version 2016 new Web API does not expose the Audit entity and can not be used for that purpose.

In this post I will demonstrate using some of the API mechanisms to extract Audit History records. Note that no API mechanism retrieves all of the Audit History record attributes (as viewed in MSCRM UI), though some retrieve more attributes than others.

Using the OrganizationData Service:

The OrganizationData Service is soon to be depreciated, but it is still available in version 2016. Compared to other APIs, the OrganizationData service query retrieves the least details for an Audit History record.

The following query retrieves Audit History records created on or after 8.1.2016 08:00 AM:

https://thekrustykrab.api.crm.dynamics.com//XRMServices/2011/OrganizationData.svc/AuditSet?$filter=CreatedOn gt datetime’2016-08-01T08:00:00Z’

Following is the result describing one Audit history record regarding an Update action. Note the UserId, AuditId, the audited entity logical name, the audited entity record id attributes. The Audit new and old value are missing.

      <m:properties>
        <d:CreatedOn m:type="Edm.DateTime">2016-01-08T10:03:05Z</d:CreatedOn>
        <d:AttributeMask>8</d:AttributeMask>
        <d:AuditId m:type="Edm.Guid">a9811402-efb5-e511-80e4-6c3be5be6de0</d:AuditId>
        <d:TransactionId m:type="Edm.Guid">d4e30d02-efb5-e511-80da-d89d67645050</d:TransactionId>
        <d:UserId m:type="Microsoft.Crm.Sdk.Data.Services.EntityReference">
          <d:Id m:type="Edm.Guid">2b6b8853-b45a-4d06-b0ad-6621b22bcbde</d:Id>
          <d:LogicalName>systemuser</d:LogicalName>
          <d:Name>Yaniv Arditi</d:Name>
          <d:RowVersion m:null="true" />
        </d:UserId>
        <d:Operation m:type="Microsoft.Crm.Sdk.Data.Services.OptionSetValue">
          <d:Value m:type="Edm.Int32">2</d:Value>
        </d:Operation>
        <d:RegardingObjectId m:type="Microsoft.Crm.Sdk.Data.Services.EntityReference">
          <d:Id m:type="Edm.Guid" m:null="true" />
          <d:LogicalName m:null="true" />
          <d:Name m:null="true" />
          <d:RowVersion m:null="true" />
        </d:RegardingObjectId>
        <d:ObjectId m:type="Microsoft.Crm.Sdk.Data.Services.EntityReference">
          <d:Id m:type="Edm.Guid">eac785da-d3a5-e511-80d8-2c59e542ba68</d:Id>
          <d:LogicalName>opportunity</d:LogicalName>
          <d:Name>6 orders of Product SKU JJ202 changed</d:Name>
          <d:RowVersion m:null="true" />
        </d:ObjectId>
        <d:CallingUserId m:type="Microsoft.Crm.Sdk.Data.Services.EntityReference">
          <d:Id m:type="Edm.Guid" m:null="true" />
          <d:LogicalName m:null="true" />
          <d:Name m:null="true" />
          <d:RowVersion m:null="true" />
        </d:CallingUserId>
        <d:Action m:type="Microsoft.Crm.Sdk.Data.Services.OptionSetValue">
          <d:Value m:type="Edm.Int32">2</d:Value>
        </d:Action>
      </m:properties>

Using the Organization Service: 

The OrganizationService can be accessed from both server and client side code.

While the OrganizationService has the RetrieveMultiple method, which returns Audit History records collection with general Audit attributes, it also supports the RetrieveAuditDetailsRequest message which returns full details for a specific Audit record. So you can use any type of querying method (QueryExpression, QueryByAttribute, LINQ, FetchXML) to retrieve a matching History records collection and complete the data with the RetrieveAuditDetailsRequest.

The following FetchXML query retrieves Audit History records created on or after 8.1.2016 08:00 AM:

<fetch count=”25″>
    <entity name=”audit”>
         <filter type=”and”>
              <condition attribute=”createdon” operator=”gt” value=”2016-08-01T08:00:00Z” />
         </filter>
    </entity>
</fetch>

Following is the result describing one Audit history record. Note that the FetchXML result contains the Operation & Action type (Update) as well as AuditId, the audited entity logical name and the audited entity record id attributes.
Note that the Audit old and new values are still missing.

[
  {
    "formattedValues": {
      "operation": "Update",
      "createdon": "1/8/2016 12:03 PM",
      "action": "Update",
      "objecttypecode": "Opportunity"
    },
    "objectid": {
      "Id": "eac785da-d3a5-e511-80d8-2c59e542ba68",
      "LogicalName": "opportunity",
      "Name": "6 orders of Product SKU JJ202 changed"
    },
    "userid": {
      "Id": "2b6b8853-b45a-4d06-b0ad-6621b22bcbde",
      "LogicalName": "systemuser",
      "Name": "Yaniv Arditi"
    },
    "operation": {
      "Value": 2
    },
    "createdon": {},
    "auditid": "a9811402-efb5-e511-80e4-6c3be5be6de0",
    "attributemask": "8",
    "action": {
      "Value": 2
    },
    "objecttypecode": "opportunity",
    "transactionid": "d4e30d02-efb5-e511-80da-d89d67645050"
  }
]

Using the RetrieveAuditDetailsRequest coupled with the FetchXML result above, the Audit old & new values can be retrieved:

var audit DetailsRequest = new RetrieveAuditDetailsRequest
{
    AuditId = new Guid(“41674669-e2b5-e511-80e4-6c3be5be6de0”)
};

var auditDetailsResponse = (RetrieveAuditDetailsResponse)organizationService.Execute(auditDetailsRequest);

Following is the RetrieveAuditDetailsResponse object viewed in QuickWatch:

Quickwatch: Audit new and old value

Outlook Tip – Defer Outgoing Email Messages

This post is not related to Microsoft Dynamics CRM, but read on because this tip can save your ass.

How many times did you regret sending an email while actually clicking the Send button? Poor choice of words, spotted typo, wrong recipients or just sending sensitive data to your customer?
Did you try to ‘recall’ the message? Did that trick ever work for anyone?

I can honestly say I hadn’t any of these horrific accidents since I started using a simple Outlook Rule: Defer Outgoing Email Messages by 3 minutes.

I discovered that 3 minutes is the ideal time frame for me to identify a mistake and cancel the pending email sending. Opening the email message waiting in the Outbox changes the message status to Draft and cancel sending until you click the Send button again.

Follow the next steps to define this rule in Outlook 2013 (I assume any decent email client allow setting similar rule). I believe Google plan (or already did?) to add similar option to Gmail accounts.

1. Make sure no email is selected (you can select the empty outbox folder) and Click the ‘Create Rule’ option under the Rules menu (Move section, Home tab)

Manage rules & alerts

2. Click the ‘New Rule…’ button to open the Rule editor

3. Select the ‘Apply rule on messages I send’ option and click Next

apply rule on messages I send

4. Click Next again and approve applying the rule to every outgoing message

apply rule on messages I send

5. Check the ‘defer delivery by a number of minutes’. In the bottom pane, click the ‘a number of’ clickable text and enter the number of minutes suitable for you. Click Next

defer delivery by a number of minutes

6. Optional step: check the ‘Except if it is marked as importance’. This will allow you to send urgent messages without deferring by setting the high importance flag on

except if it is marked as importance

7. In the lower pane click the importance clickable text and select High. Click Next

select high importance

8. Name the step with some meaningful name. Click Finish

name rule

You are done! Test the rule by sending an email. The new rule will delay the outgoing email in the outbox for 3 minutes unless you reopen it.

Try this rule, you will never go back.

Asynchronous Batch Process Solution Revisited – part 1

I had some free time and decided to review the Asynchronous Batch Process Pattern Solution. On January 2013 I promised to publish a revised version and didn’t manage to do so. To redeem myself, I have decided to publish this version on CodePlex, so it can be customized by anyone who needs to.
If you don’t know what ABPP is all about and what’s in it for you, I suggest you start here.

In the first part of this post I’ll describe the new features, architecture considerations and implementation details. In the next part, I’ll walkthrough the different scenarios and use cases.

You can download an unmanaged solution as well as source code here.
As always, I advise not to publish an external solution on your production environment without testing it first. I tested this solution with Online (2015 RU 0.1) and on-premise 2015 deployments.

 

Following are some of the features included in the revised solution (version 2.0):

  1. Microsoft Dynamics CRM 2015/Online supported solution
  2. Action support – allow mapping to Action/Workflow Rule (sync./a-sync.) process as an Action component
  3. Integration scenarios support – activate Action process without specifying target records. This allows scheduling activation of any external action (like posting/retrieving data to/from an external application or data source) which can be wrapped with Custom Workflow Activity component
  4. Aggregative FetchXML query support – schedule execution of a an aggregative query (multiple aggregation support) and map aggregation result(s) to target Action as an input parameter
  5. Hourly scheduling  frequency support
  6. Diagnostics – update batch process record with last activation result (success/failure) and failure reason.
  7. Fetchxml 5000 records limit override – use conversion to Query Expression in order to handle queries that exceed 5000 records
  8. Activation counter – additional attribute that automatically increment with each successful Batch Process activation. This allows adding a workflow step or Rule that will suspend the Batch Process scheduling after X activations
  9. Zombie process prevention – scheduling activation only for future dates to prevent spawning of Workflow instances for past dates that will never occur

I have skipped some features that I could not design without breaking the ABPP paradigm, which generally dictates a complete separation between the Schedule component, the Action component and the target business records.
These features need some more thinking and if you have any suggested architecture, tell me about it.
The features I skipped:

  1. Centralized error handling – since the Action component is oblivious to its activator, execution errors (especially with a-sync. process Action handlers) cannot be easily related to the process activation
  2. Detailed Tracing – similarly to error handling, recording detailed tracing information such as total number of failure/success cases or processed records ids is a challenge due to the ABPP paradigm

Now, let’s get down to the details.

First, a small addition to the ERD. The new Process Activation entity purpose is to document each activation details.
In the future, I’ll add option to document the ids of the records which were returned by the Target Records Query.
This will allow tracing of the specific records that were affected by each ABP activation.

image

Currently, the Process Activation entity contains these custom attributes:

  1. Parent Batch Process id
  2. Activation start date
  3. Activation end date
  4. Total number of processed business records

Next, the main algorithm diagram:

Algorithm

Color Your Entity

I have noticed a new attribute named ‘Color’ in the new entity form (Online, 2015 RU 0.1).
It is not documented in the online help yet, but guessing this attribute purpose is not difficult: customize the entity panel color in the navigation menu for custom and existing (OOTB) entities. Nice!

The Color attribute input text length is limited to 7 characters, to match a color Hex code such as #FF6600 (Orange). Setting a Hex code will let you preview the represented color in the preview box beside the attribute text box.

You can find any color Hex code in web site like this.

Setting entity color

Custom entity color in navigation menu

Microsoft – First signs of humor

I often spin up Microsoft Dynamics CRM Online trials for demos, tests etc. Along with the 2014 Spring Update, some welcomed changes were made to the Online Trial set up process.

One insignificant addition to the process made me smile. This subtle humor is quite rare in Microsoft over whole user experience and more often found in Google applications.

It makes the process more friendly while keeping it user friendly.

Keep it up, Microsoft.

 

clip_image001

Logging and Handling Microsoft Dynamics CRM 2013 Exceptions – Part 3

In the previous post I described a solution to the business problem of logging & handling implementation level exceptions (presented in the first post of this series).
In this post, I will supply an actual solution, demonstrate common usage scenarios and other solution features.

Before I walkthrough usage scenarios, some implementation notes:

In order to support exceptions raised from transactional components (such as Plugin registered to pre/post operation stages), the LogException method in ExceptionManagement.cs file is using the ExecuteMultipleRequest class to execute the e4d_LogExceptionRequest request. As the ExecuteMultipleRequest instance is external to the Plugin transaction, it manages to create the Exception record without the Plugin transaction rolling it back. Using the Execute method directly with the e4d_LogExceptionRequest request instead, would have executed the LogException Action, but the created Exception record would have been deleted by the Plugin transaction rollback, leaving no trace.

The provided Exception Management solution is unmanaged. This allows you to modify the solution freely but note that unmanaged solution can not be uninstalled.

The provided Exception Management solution is exported as MSCRM2013 6.0 compatible.

As an Exception record is created on behalf of a user, all users must have the Create, Read and Write privileges for the Exception entity.
The solution contains a custom Security Role named Exception Logging Authorized which includes these privileges for the Exception entity. Make sure you add this role to your users or add the privilege to their existing roles.

Ok, let’s get some work done:

Step 1: import and configure the Exception Management solution:

1. Download and extract the E4D MSCRM2013 Excreption Management.zip package.
2. Import the solution ExceptionManagement_1_0_0_0_target_CRM_6.0.zip.  Leave the ‘Enable any SDK message…’ checkbox checked.
3. Add the Exception entity to the Settings area.
4. Publish the solution.
5. Add the Exception Logging Authorized Security Role to all users.

Step 2: Follow the relevant scenario below to use the Exception Management functionality from your custom code:

Scenario 1: Logging Exceptions from JavaScript code running in Entity form

1. Open the target entity Form Properties window.
2. In the Events tab, under Form Libraries, add the ExceptionManagement.js library to the form libraries.
3. If the JavaScript Web Resource that contains your custom code does not appear in the form libraries, add it. Make sure it appears below the e4d_ExceptionManagement.js library.

image_thumb5_thumb

4. In your custom JavaScript code (someFunction in my example), wrap the target code with Try-Catch blocks.
5. In the Catch block, add the following line of code. You can add your own error handling code in the Catch block.

Sdk.logException(error, Sdk.e4d_LogExceptionRequest.PriorityLevel.LOW);

Your code should look similar to this:

function someFunction()
{
try
{
//your custom business logic here
}
catch(error)
{
//your custom exception handling here
//log exception. This is the only line of code required in your custom code
//You can set the Priority level to MEDIUM, HIGH or CRITICAL instead of LOW
Sdk.logException(error, Sdk.e4d_LogExceptionRequest.PriorityLevel.LOW);
}
}

6. Save and publish the entity form.

Scenario 2: Logging Exceptions from JavaScript code running in HTML Web Resource

1. In your HTML Web Resource, add the following references in that order:

    <script src=”ClientGlobalContext.js.aspx” type=”text/javascript”></script>
<script src=”e4d_exceptionManagement.js” type=”text/javascript”></script>

2. Follow steps 4,5 in the Scenario 1. 3.
3. Save and publish the Web Resource

Scenario 3: Logging Exceptions from c# code running in Plugin/Custom Workflow Activity/.net application

1. Add the ExceptionManagement.cs file to your Visual Studio project
2. In your code, add the following using statement

   using ExceptionMgmtServices;

3. wrap the target code with Try-Catch blocks
4. In the Catch block, add the following line of code. You can add your own error handling code in the Catch block

string friendlyMessage = ExceptionManagement.LogException(ex, service, PriorityLevel.LOW);

Your code should look something like this:

try
{
//your custom business logic here
}
catch (Exception ex)
{
//your custom exception handling here
//You can set the Priority level to MEDIUM, HIGH, CRITICAL instead of LOW
string friendlyMessage = ExceptionManagement.LogException(ex, service, PriorityLevel.LOW);
//Optionally, throw out the exception to return friendly message to the client
throw new Exception(friendlyMessage);
}

I advise you test the relevant scenarios by generating an intentional exception, such as calling a non-existing function. Make sure an error message is displayed, an Exception record is created and that the Exception details are correct.

The ExceptionManagement_1_0_0_0_target_CRM_6.0.zip contains a Web Resource named ClientSideExceptionDemo.htm which demonstrate scenario 2.

The E4D MSCRM2013 Excreption Management package contains a sample Plugin project called E4D.CRM.ExceptionMgmtServices.ClientPlugin aimed to demonstrate scenario 3.

Managing Exceptions tools:

Navigate to the Exceptions node in the Settings area to view the Exceptions list. You can feed the unique token displayed in the error message in the Quick Find search box to quickly find the relevant Exception record.
You can export list of Exception records to Excel in order to send out to be reviewed by external developer.

image_thumb14_thumb[3][4]

image_thumb11_thumb[1][4]

image_thumb2_thumb[1]

 

As this solutions is unmanaged, you can can customize and extend it as you wish, here are some ideas:

  • Create a Workflow Rule that will add exceptions to a designated Queue from which Exceptions can be pulled and handled by developers or system administrators.
  • Create a Workflow Rule that will send email to administrators group when critical Exception record is created.
  • Create a Workflow Rule that will send an acknowledgment email to the end user when Exception record is created or handled (status change).
  • Develop a Custom Workflow Activity or Plugin component that will integrate with your ALM application to automatically send exceptions to be handled by the development team.
  • Add JavaScript code to display a nice error message using setFormNotification when error is displayed in entity forms, instead of displaying an alert.

I would like to hear your ideas about extending and improving this solution.

Logging and Handling Microsoft Dynamics CRM 2013 Exceptions – Part 2

In the previous post, I described the business problem of logging and handling implementation level exceptions. In this post, I would like to suggest a solution.

Before you continue reading, note that logging implementation level exceptions is up to your code developer. Although the solution supplies easy methods to log exceptions (to be demonstrated in part 3), nothing will be logged If your custom code does not use the solution components.

Let’s start by defining some business requirements for the suggested solution:

  • Support Microsoft Dynamics CRM 2013 of any deployment type
  • Expose minimum exception details to the user, maximum to the System Administrator
  • Supply the client (user or software component) with a unique correlation code to id each exception instance
  • Require minimum effort from software developers to log exceptions
  • Allow logging and handling client side & server side exceptions in a similar manner
  • Supply System Administrator with an efficient way to search and view exceptions
  • Allow flexible handling when exception is logged: send email notification, move to queue, assign to user/team, send acknowledgment to user etc.

Next, here are the suggested solution major components along with some architecture consideration:

  1. Exception Entity
    This custom entity describes an exception instance details and will be created whenever an exception is logged. Using a custom Exception entity will allow using Microsoft Dynamics CRM UI to easily search and view exception records. A custom entity will also allow great flexibility for handling exceptions by using all Microsoft Dynamics CRM magic: Processes, Queues, Advanced Search, Dashboards, Charts, export to Excel etc.

    The Exception entity will have the following attributes:

    • Name – describes the exception’s raw error message  
    • Created By – reference to the user who initiated the process from which the exception was raised
    • Created On – Date & time on which the exception was logged
    • Friendly Message – the error message that was displayed to the user
    • Unique Token – a unique id assigned to the exception instance and returned to the client along with the friendly message. This token will be used by the System Administrator to correlate the logged exception instance to the error message displayed to the user
    • Priority – describes the exception handling priority
    • Stack Trace – describes the exception’s stack trace which supplies contextual information regarding the calling source function, file etc.
  2. LogException Action
    A custom Action will be used to create an Exception record according to the specified details, generate and assign a unique token and return a friendly message to the client.

    Why Action?

    • Compared to writing custom code, Actions simplify the task of creating and updating an Exception record as it harness the powerful Process tools
    • Action is the only MSCRM mechanism that can receive input parameters and return output parameters in a straight forward manner. Sort of a web service
    • As Action can be accessed from client side (JavaScript code) and server side (Plugins, Custom Workflow Activities, external applications), it provide a single point of implementation
    • Actions allow defining business logic in a declarative manner, so Actions can be easily created and maintained by non developers, as long as the Action signature (parameters and name) is not changed

  3. Unique Token Generator Custom Workflow Activity
    This component will be used by the LogException Action to generate the unique token value. The unique token will be saved to the Exception record and also displayed to the user as part of the managed error message. This will allow the System Administrator to quickly locate the exact exception instance the user has reported.
    As the Process basic tools do not allow generating a unique token, a custom component is required. I prefer a Custom Workflow Activity component as it can be embedded in Actions natural (and transactional) flow, rather than Plugin which is external to the Action. It can also be reused in Workflow Rules and Dialogs.
  4. Sdk.Soap.js
    The Sdk.Soap.js package was recently released by Microsoft as part of the 6.1 SDK. It supplies a comprehensive and intuitive framework for MSCRM JavaScript developers and I recommend it completely.
    One of the package’s tools is the Action Message Generator which automatically generates a JavaScript API for Actions. This API simplifies the execution of Actions from JavaScript and the solution will use it to execute the aforementioned LogException Action from JavaScript Code.
     
  5. ExceptionManagement.js
    This JavaScript Web Resource is used to wrap the functions required to log exception from JavaScript code. 
    In order to use the exception logging functionality, a developer is required to reference this Web Resource in forms event handlers or other Web Resources. This file includes the Sdk.Soap.js library in order to prevent the need for additional reference.
  6. ExceptionManagement.cs
    This C# class is used to wrap the methods required to log exception from c# code written in Plugins, Custom Workflow Activities, external .net applications etc.
    In order to use the exception logging functionality, the developer is required to add this class to his project or compile it into a DLL and reference it.
     
  7. Exception Management Dashboard
    This dashboard is aimed to help the System Administrator manage exceptions with the following components:
    • Exceptions by priority and creation day chart
    • Exceptions by priority and user chart
    • Exceptions by priority chart
    • Critical & High priority exceptions view

 

Finally,  a nice diagram of the solution’s major components:

Solution Components Diagram

Feel free to comment on the architecture and the planned Implementation, I welcome your feedback.

In the next post I will describe the actual Solution and will demonstrate common usage scenarios.