Execute a Recurring Job in Microsoft Dynamics 365 with Flow

I have written about executing recurring jobs in Dynamics 365 few times in the past. Over time, I suggested different scheduling mechanisms such as Microsoft Dynamics Workflow Timeout step or Azure Scheduler, as the pattern I suggested allows changing the scheduling mechanism without impacting other solution parts.

Flow can be also used as a scheduling mechanism, one that does not require coding like Azure Function,  as it has a built in integration with Microsoft Dynamics 365 Online.
Once invoked on schedule, the executing component query Dynamics 365 for target business records and apply some business logic (Process) to each business record.

In this post I would like to demonstrate a solution for executing a recurring job in Microsoft Dynamics 365 using Flow as the scheduling component.

Sample business requirement

Weekly Leads Evaluation: once a week, disqualify all Leads that are more than 5 days old and are not rated ‘Hot’.
Add a note to each Lead record to indicate that it was disqualified by an automated process.

Walkthrough

1. Download, Import and publish the ABP unmanaged solution.
2. Create the Action component: Disqualify Lead Workflow

   This Workflow Rule will be applied to each valid target business record. It simply attaches a Note and changes the Lead record state to Disqualified. It can be a/synchronous, but you may want to start with a asynchronous execution to monitor the process execution.
   Make sure you define the process as an on-demand process with no automated triggers.

   

3. Define Target Business Records Query

   Using the Advanced Find, define a new query to retrieve all Lead records where Created On date is older than 5 days and are not rated ‘Hot’.
   Click to Edit Columns button and remove all possible columns to maximize the query efficiency.
   Click the Download Fetch XML button and open the resulting file with some text editor. Copy the FetchXML query text to the clipboard

   

4. Create a Batch Process Record

   This record is used to orchestrate by holding a FetchXML query to define the target business records and also the Action process which will be applied on the the target business records.
   If you imported an unmanaged solution, find the Batch Process entity and set it to appear in the Settings area.
   Create a Batch Process record and paste the FetchXML query text into the Target Records textbox.
   Name the record ‘Weekly Leads Evaluation Process’.
   Select the Disqualify Lead Workflow (created in step 2) in the Process Lookup field and save the record.
   Copy the Reference Token value for the next step.

   

5. Define Flow Rule

   In your tenant, navigate to the Flow area and create an empty Flow named Weekly Leads Evaluation.
   Define a Recurrence Trigger to trigger the batch process

   

   Next, define a List records action.
   In the Filter Query text box, paste the Batch Process Reference Token value and precede it with dyn_referencetoken eq (e.g. dyn_referencetoken eq ‘LOGY8Y1W6’).

   

   Define Update record action which will automatically wrap it with Apply to each wrapper. Add a Condition before the Update Action to verify that the Flow operates only if the Batch Process record status is Scheduled. 

   

6. Activate

   Change the Batch Process record status to Scheduled and save.
   Activate the Flow Rule.
   You can monitor the Flow activation by navigating to the Run History area.

   

   For each run, you can see the completion status (success/failure) and failure reason

   

Similarly, you can now add additional scheduled batch processes. 

Implementation Details

• An alternative approach is described here. Though may be more simple, It requires maintaining user credentials in Flow.

Advertisement

Animated Tip: Create a Fully Functional Portal Page in 60 Sec. (Or Less)

The Create Portal Content wizard is a nice new feature in Portal version 8.3 that was published recently. This wizard simplifies the process of creating webpages along with entity forms and entity lists by admins.

If you are not familiar with the process of provisioning new pages in Microsoft Dynamics 365 Portal, you can do some reverse engineering of the Portal elements automatically created by the wizard.

In the following demo, I am creating a new Page for authenticated Portal users which displays Lead records with an option to create new record. It assumes that the user has the required Entity Permission for the Lead entity.

Click here to view a larger version of this animated tip

Animated Tip – Find Inactive Records with Quick Find

Contrary to common belief, the filter criteria for the Quick Find View can be edited

Customization Aware Web Resource – Part 3

In the previous post I demonstrated an implementation of the Customization Aware Web Resource approach using SOAP to retrieve metadata. In this post, I’ll demonstrate a similar implementation, this time using Web API.
You can download an unmanaged solution containing the working sample code presented in this post. Note that this is a different solution version from the one made available in the previous post.

Why should you use Web API instead of SOAP? Here are some of the reasons which importance rise as your implementation scales up:

• Reducing network traffic: compared to SOAP, REST requests are lighter and require less network resources
• Reduce maintenance efforts: REST based code require considerably less code than SOAP based code. Less code require less maintenance
• Robustness: even when using a library like Sdk.Soap.js or SDK.MetaData.js to abstract SOAP plumbing, you depend on an external library you have to drag along, maintain, minify etc. With Web API, you can simply use native XMLHttpRequest object to submit requests. Less moving parts means a more robust solution 
• You would prefer code with an attitude:

Bits & Bytes

Few details to note regarding Web API metadata queries:

1. The target table used is EntityDefinitions
2. Each entity and attribute has a unique metadata id stored in a property named MetadataId
3. Retrieving metadata using attribute logical name will only return general properties (which all attribute has: display name, requirement level, description etc.)
4. Retrieving type specific attributes (OptionSet options, number values range, text format etc.) require a query involving the attribute MetadataId value, sometimes with a proper type cast. It means you have to know each attribute MetadataId in advance if you wish to perform only one query per attribute. Fortunately, entity and attribute MetadataId values are static between Organizations, so once you found the required MetadataId value, you can safely use it as a constant in your queries, even when importing code via Solution into other Organizations.

Here are some sample queries and responses:

The following query require entity and attribute logical name but will return only general characteristics for the specified attribute:

…/EntityDefinitions?$select=MetadataId&$filter=SchemaName eq ‘Lead‘&$expand=Attributes($select=LogicalName,SchemaName,Description,RequiredLevel,DisplayName;$filter=LogicalName eq ‘firstname‘)

The response contains the Lead entity MetadataId, the firstname attribute MetadataId and the requested general attributes:

{
  "@odata.context":"https://thekk.crm.dynamics.com/api/data/v8.0/$metadata#EntityDefinitions(MetadataId,Attributes,Attributes(LogicalName,SchemaName,Description,RequiredLevel,DisplayName))","value":[
    {
      "MetadataId":"dc6574cb-92ce-446c-a5d6-885a75107d52","Attributes@odata.context":"https://thekk.crm.dynamics.com/api/data/v8.0/$metadata#EntityDefinitions(dc6574cb-92ce-446c-a5d6-885a75107d52)/Attributes(LogicalName,SchemaName,Description,RequiredLevel,DisplayName)","Attributes":[
        {
          "@odata.type":"#Microsoft.Dynamics.CRM.StringAttributeMetadata","LogicalName":"firstname","SchemaName":"FirstName","Description":{
            "LocalizedLabels":[
              {
                "Label":"Type the first name of the primary contact for the lead to make sure the prospect is addressed correctly in sales calls, email, and marketing campaigns.","LanguageCode":1033,"IsManaged":true,"MetadataId":"8c4901bf-2241-db11-898a-0007e9e17ebd","HasChanged":null
              }
            ],"UserLocalizedLabel":{
              "Label":"Type the first name of the primary contact for the lead to make sure the prospect is addressed correctly in sales calls, email, and marketing campaigns.","LanguageCode":1033,"IsManaged":true,"MetadataId":"8c4901bf-2241-db11-898a-0007e9e17ebd","HasChanged":null
            }
          },"RequiredLevel":{
            "Value":"Recommended","CanBeChanged":true,"ManagedPropertyLogicalName":"canmodifyrequirementlevelsettings"
          },"DisplayName":{
            "LocalizedLabels":[
              {
                "Label":"First Name","LanguageCode":1033,"IsManaged":true,"MetadataId":"8b4901bf-2241-db11-898a-0007e9e17ebd","HasChanged":null
              }
            ],"UserLocalizedLabel":{
              "Label":"First Name","LanguageCode":1033,"IsManaged":true,"MetadataId":"8b4901bf-2241-db11-898a-0007e9e17ebd","HasChanged":null
            }
          },"MetadataId":"4ed9d959-5425-468c-bf1c-98a8b125ab31"
        }
      ]
    }
  ]
}

 

The following query uses the entity and attribute MetadataId values to retrieve type specific characteristics

…/EntityDefinitions(dc6574cb-92ce-446c-a5d6-885a75107d52)/Attributes(4ed9d959-5425-468c-bf1c-98a8b125ab31)

The response contains the text format which was can not be retrieved by the previous query :

{
  "@odata.context":https://thekk.crm.dynamics.com/api/data/v8.0/$metadata#EntityDefinitions(dc6574cb-92ce-446c-a5d6-885a75107d52)/Attributes/$entity,

    "@odata.type":"#Microsoft.Dynamics.CRM.StringAttributeMetadata","Format":"Text","FormatName":{
    "Value":"Text"
  },"ImeMode":"Active","MaxLength":50,"YomiOf":null,"IsLocalizable":false,"FormulaDefinition":null,"SourceTypeMask":0,"AttributeOf":null,"AttributeType":"String","AttributeTypeName":{
    "Value":"StringType"
  },"ColumnNumber":19,"Description":{
    "LocalizedLabels":[
      {
        "Label":"Type the first name of the primary contact for the lead to make sure the prospect is addressed correctly in sales calls, email, and marketing campaigns.","LanguageCode":1033,"IsManaged":true,"MetadataId":"8c4901bf-2241-db11-898a-0007e9e17ebd","HasChanged":null
      }
    ],"UserLocalizedLabel":{
      "Label":"Type the first name of the primary contact for the lead to make sure the prospect is addressed correctly in sales calls, email, and marketing campaigns.","LanguageCode":1033,"IsManaged":true,"MetadataId":"8c4901bf-2241-db11-898a-0007e9e17ebd","HasChanged":null
    }
  },"DisplayName":{
    "LocalizedLabels":[
      {
        "Label":"First Name","LanguageCode":1033,"IsManaged":true,"MetadataId":"8b4901bf-2241-db11-898a-0007e9e17ebd","HasChanged":null
      }
    ],"UserLocalizedLabel":{
      "Label":"First Name","LanguageCode":1033,"IsManaged":true,"MetadataId":"8b4901bf-2241-db11-898a-0007e9e17ebd","HasChanged":null
    }
  },"DeprecatedVersion":null,"IntroducedVersion":"5.0.0.0","EntityLogicalName":"lead","IsAuditEnabled":{
    "Value":true,"CanBeChanged":true,"ManagedPropertyLogicalName":"canmodifyauditsettings"
  },"IsCustomAttribute":false,"IsPrimaryId":false,"IsPrimaryName":false,"IsValidForCreate":true,"IsValidForRead":true,"IsValidForUpdate":true,"CanBeSecuredForRead":false,"CanBeSecuredForCreate":false,"CanBeSecuredForUpdate":false,"IsSecured":false,"IsRetrievable":false,"IsFilterable":false,"IsSearchable":true,"IsManaged":true,"IsGlobalFilterEnabled":{
    "Value":false,"CanBeChanged":true,"ManagedPropertyLogicalName":"canmodifyglobalfiltersettings"
  },"IsSortableEnabled":{
    "Value":false,"CanBeChanged":true,"ManagedPropertyLogicalName":"canmodifyissortablesettings"
  },"LinkedAttributeId":null,"LogicalName":"firstname","IsCustomizable":{
    "Value":true,"CanBeChanged":false,"ManagedPropertyLogicalName":"iscustomizable"
  },"IsRenameable":{
    "Value":true,"CanBeChanged":false,"ManagedPropertyLogicalName":"isrenameable"
  },"IsValidForAdvancedFind":{
    "Value":true,"CanBeChanged":true,"ManagedPropertyLogicalName":"canmodifysearchsettings"
  },"RequiredLevel":{
    "Value":"Recommended","CanBeChanged":true,"ManagedPropertyLogicalName":"canmodifyrequirementlevelsettings"
  },"CanModifyAdditionalSettings":{
    "Value":true,"CanBeChanged":false,"ManagedPropertyLogicalName":"canmodifyadditionalsettings"
  },"SchemaName":"FirstName","IsLogical":false,"InheritsFrom":null,"SourceType":0,"MetadataId":"4ed9d959-5425-468c-bf1c-98a8b125ab31","HasChanged":null
}

 

The only exception is related to OptionSet control options collection which require an expanded query along with a proper type cast:

…/v8.0/EntityDefinitions(dc6574cb-92ce-446c-a5d6-885a75107d52)/Attributes(49afb6d2-986f-4ccc-b07d-a38de518b8d1)/Microsoft.Dynamics.CRM.PicklistAttributeMetadata?$expand=OptionSet

As shown in the HTML below (contained in dyn_CreateLeadWebAPI.htm), I made some design changes to support both Logical name and metadataId.
Adding just the logicalname attribute to any input control will cause the code to perform two queries to retrieve full metadata details. Adding the optional metadataid attribute will perform one query. Not adding any of the attributes will cause the code to ignore the control.

 

The two interesting functions here are retrieveMetadataByLogicalName and retrieveMetadataByMetadataId which actually query metadata using Web API.
Note how simple it is to construct a query, submit it and parse the response.

 

 

Conclusion

With minor effort, the sample code in the previous or current post can be turned into a JS library which will quietly apply the CAWR approach to your Web Resources just by referencing it.
Setting HTML controls properties to match MSCRM attribute logical name or metadataid will enable the code to automatically retrieve and apply metadata.

If you have any ideas or suggestions regarding the CAWR approach, share them here.

Customization Aware Web Resource – Part 1

Web Resource is one of the most powerful and versatile extension mechanism in Microsoft Dynamics CRM. It supplies an open canvas to draw custom UI, business logic and whole applications within entity forms, dashboards and stand alone pages.
Coupled with the plethora of  free JavaScript libraries out there, you can easily provide your users with advanced solutions that work with Microsoft Dynamics CRM Data. In that sense, Web Resource can implement a custom client which can also be embedded into MSCRM elements.

One concern regarding the investment in Web Resource development is the loss of dynamic customization capabilities, which is MSCRM most prominent feature. Indeed, there is a price to pay when stepping outside Microsoft Dynamics CRM native clients.
The Customization Aware Web Resource approach can somewhat compensate as it suggests that whenever possible, attributes of elements within a Web Resource (mostly HTML & JavaScript) will be coupled with relevant customization features. So customizing entity field attributes like display name, description, requirement level, OptionsSet options etc. will be automatically reflected in your Customization Aware Web Resource.

Technically, this can be achieved by using the metadata querying capabilities of the OrganizationService or the Web API services.
Off course, this approach is relevant to any custom client, not just Web Resources.

One issue to consider when using the Customization Aware Web Resource approach is performance. While the OrganizationService.svc allows querying multiple entities & attributes with one query, the Web API service can only query one attribute at a time, so multiple queries may be required. Metadata queries are considered expensive, so if you plan to rely heavily on this approach, you should cache metadata queries results along with some cache invalidation mechanism.

In the following post, I’ll demonstrate a possible implementation of this approach. Stay tuned

Web API: Executing Custom Action via JavaScript

The Custom Action is an important tool in Microsoft Dynamics CRM architect belt. Here are some of the Custom Action key features:

1. It can be called from both client & server side, enabling the SPoI approach (Implement once, consume anywhere)
2. Supply a flexible, declarative business logic definition mechanism which can be called from other Processes as well as from custom code (unlike Plug-in)
3. Can be extended with Custom Workflow Activities
4. Can receive input parameters and return output parameters in a straight forward manner, similar to an Organization level Web Service

Why would you want to execute a Custom Action (which is a server side mechanism) via JavaScript? Here are some excellent reasons:

• Provide an improved UX, as Custom Action can be executed a-synchronously from entity forms, web resources and Ribbon/Command Bar controls without blocking the user UI
• You can bind server side logic to client side events, e.g. calling a Custom Action from an onChange or onSave event handlers
• Implement a chunky rather than chatty communication with the server, as Custom Action usually wraps multiple operations which otherwise require multiple API commands

You can execute a Custom Action via JavaScript using raw SOAP (not recommended) or better yet, using the Sdk.Soap.js library (still SOAP, but nicely wrapped).
With Microsoft Dynamics CRM 2016 Web API you can now execute a Custom Action in the most simple manner, which is demonstrated in the rest of this post.

You can download an unmanaged Solution containing the sample Action and JavaScript code. This solution is for demonstration purposes only, please do not install on production or other critical environments.

1. Define and activate a Custom Action in MSCRM 2016 Organization

   My sample Action receives few parameters, creates a Lead record and returns the new record URL.
   Note that this Action is defined for a global context. Also note the Action’s unique name and parameters names which are used in the following code.

   

2. Write JavaScript code to consume your Action
   
   The handleWebLead function defines the dyn_HandleWebLead Action name and parameters.
   It than calls the generic activateCustomAction function to execute the Action and return the output result.

   
    
   The Action is executed using XMLHttpRequest which is defined to execute synchronously.

   

Visible By Default – What Is It Good For?

Short answer: improving form performance & user experience, that’s what.

Long answer:

Since version 2011, major entity form elements (Control, Section, Tab) have the Visibility attribute added. This attribute, which is checked by default, determines if the form element will be visible to the user when the entity form loads.

Why would you want to render some elements invisible when the form loads? Here are some common reasons:

1. Conditional logic: The form has some business logic related process which requires displaying some elements according to the user actions, e.g, hide Government Id attribute if the user selected Organization as Customer Type.
2. Hiding built in required controls which can not be removed from the form and are meaningless to the end user.
3. Hiding controls that are used to support the form business logic and has no meaning or use to the end user.

One uncommon reason to use the Visibility attribute is to improve form performance. I would like to elaborate on this, since I noticed many Microsoft Dynamics CRM developers are not aware of the impact which JavaScript code has on form performance which In most cases directly affect UX.

How can the Visibility attribute improve form performance and UX?

Business logic often requires script to hide/display form elements according the user actions. When the form loads, only a subset of controls should be visible to the user.
In many cases, the JavaScript code/form business rules which is triggered by the form onLoad event, literally hides all the form elements (some times, each individual control) and then render the required elements visible again.
Not only this approach waste client resources (inefficiently hiding controls which were just ‘painted’) but it also provides bad UX as it slows the form load time. With low resources machines, the user can also see the controls flick on the form before they are hidden by the JavaScript code.

The right approach in this scenario would be to uncheck the ‘Visible by default’ attribute for the elements which are not required to display on form load  (preferring tabs over sections, sections over controls) and writing JavaScript code to render elements visible when required.

This approach will improve form performance & UX because of the following reasons:

1. Less form controls to render: the form will load faster because it has less controls to render. In control heavy forms/slow machines, this is substantial.
2. Less code lines to execute: since custom hiding related code/Business Rules logic is removed, the onLoad event will complete and turn control to the user faster. You can write less code by hiding/displaying containers such as Tabs and Section rather than relating to each individual control. Again, think control heavy forms.
3. Reduce visual load: no element flickering on the form will provide better UX.

User experience is critical to the application acceptance and over whole rating by users. Developers sometime miss these problems because the from loads fast enough on their dev machine.
For users who open record forms many times a day, even 2 seconds less form loading time can make a major difference.

Asynchronous Batch Process Solution Revisited – part 4

In the previous post, I walked through the ABP (Asynchronous Batch Process) Aggregative Query Scenario. In this post, last in this series, I’ll go through the External Action scenario.

Prerequisites

1. Download the Asynchronous Batch Process Solution, import into Microsoft Dynamics CRM 2015 on-premise/Online organization
2. Go to Settings – > Solutions and Open the ABP solution. Go to the Batch Process entity definition and check the Settings checkbox in the ‘Areas that display this entity’ section
3. Save and publish the solution

As always, I advise against publishing any external solution on your production environment without testing it first.

External Action scenario

This scenario describes a scheduled action, not necessarily related to Microsoft Dynamics CRM records, hence external.
The target records FetchXML query can be used but it is optional.
This action is usually a wrapper for Custom Workflow Activity performing some magic which the native process tools can not.

Some sample business requirements of this type:

– Retrieve users data from AD (Active Directory) on daily basis and update Microsoft Dynamics CRM users data accordingly
– Import/export data to/from Microsoft Dynamics CRM to an external application over night
– Generate and email SSRS report to a mailing list every month

Following are the steps to implement the daily AD users data synchronization. For this scenario I assume you already registered a Custom Workflow Activity with some capability to retrieve data from AD and update System User records.

1. Create a new Action targeting ‘None (global)’ and name it ‘Synchronize AD users’
2. Add a step to activate the AD synchronization Custom Workflow Activity component
3. Save and activate the process

4. Create a new Batch Process record (Settings –> Batch Processes)
5. Name the Batch Process: ‘Daily AD users synchronization’ (or other meaningful name)
6. Set the Activation Frequency to ‘Daily’
7. Set the Process field value to ‘Synchronize AD users’ process
8. Set the Next Activation to any future date and time. I suggest selecting the application’s least busy time
9. Set the Status Reason to ‘Scheduled’
10. Click Save to schedule the Batch Process

You are done! The Batch Process is set and waiting for the next activation date and time. If all goes well, the AD users
synchronization will execute as defined and the Batch Process will reschedule to the same time next day.

If you would like to deactivate the Batch Process scheduling, change the Status Reason value to ‘Suspended’ and save. Since there may already be a Workflow instance scheduled and waiting, go to the Background Processes for the Batch Process record and cancel waiting instances.
This also applies if you change the Next Activation date/time once the Batch Process record is already created and scheduled.

I hope you find the scenarios I have described in the series as useful to you. If you have development skills or resources, you can customize the ABAP solution as you like, the source code is available on CodePlex.
I will keep adding features to this component and upgrade along with Microsoft Dynamics CRM versions until MS supplies a similar built in solution.