Sandbox Execution Timeout

While reviewing existing Microsoft Dynamics 365 Online implementations, I have noticed a repeating assumption by which Custom Workflow Activity, executing within an a-synchronous process, is not subjected to the sandbox 2 minutes execution timeout limitation.
This false assumption led to wrong architectural designs in which long running processes are assigned to be handled by CWA components.

As this article states, Custom Workflow Activity code registered to work in sandbox mode is subjected to the 2 min. execution timeout limit, just like Plug-in code. If you want to test this declaration yourself, just put a Sleep statement in your CWA code and get the following exception for both sync. and a-sync. processes:

The plug-in execution failed because the operation has timed-out at the Sandbox Host

So here are my tips regarding the Sandbox timeout limitation:

  1. Plan for big scales: You may have hundreds of records to process on day 1, but in a year or two, you might have thousands and your CWA/Plug-in component will start hitting the execution timeout limit.
  2. Use the right tool: Both CWA and Plug-in components are not meant for heavy lifting. For potentially long running processes or complex computations, these mechanisms should be used only as event wrappers, triggered by a business event and passing on execution to a more robust and manageable mechanisms such as Azure App Services, Azure Functions or other external mechanisms.

  3. Handle all exceptions: You can easily hit the the timeout limit, especially when integrating external services which you have no control of and may not response in a timely manner.
    In general, If you don’t handle exceptions, your business logic may disgracefully break and your users/clients might experience some ugly ass raw exceptions.
    While handling exceptions, you can single out timeout exception by testing the exception status:
    catch (WebException exception)
    {
    string str = string.Empty;
    if (exception.Response != null)
    {
    using (StreamReader reader = new StreamReader(exception.Response.GetResponseStream()))
    {
    str = reader.ReadToEnd();
    }
    exception.Response.Close();
    }
    if (exception.Status == WebExceptionStatus.Timeout)
    {
    throw new InvalidPluginExecutionException(
    “The timeout elapsed while attempting to process the request.”, exception);
    }
    throw new InvalidPluginExecutionException(String.Format(CultureInfo.InvariantCulture,
    “A Web exception occurred while attempting to process the request. {0}: {1}”, exception.Message, str), exception);
    }
  4. Diagnose: You can diagnose your CWA/Plugin components by using the Trace service in your code. Tracing can be easily switched to display all tracing, just error or turned off by setting the Plug-in and custom workflow activity tracing in the Organization Settings dialog and then viewed in the Plug-in Trace Log area.
    Tracing all incoming parameters values (as this article suggests) can help you better diagnose your CWA operations.

    Plug-in and custom workflow activity tracing in the Organization Settings dialog
     Plug-in Trace Log area

  5. Monitor: Sandbox Run time statistics can help you monitor your Plug-in/CWA performance over time, failures and running time metrics.
    Run the following FetchXML query to get statistics about your component. Replace the name attribute value with your component name.
    Note the averageexecutetimeinmilliseconds and failurecount results – it might indicate that your business logic is failing or close to failure.

    <fetch>
    <entity name=”plugintypestatistic”>
    <attribute name=”averageexecutetimeinmilliseconds” />
    <attribute name=”failurecount” />
    <attribute name=”failurepercent” />
    <attribute name=”crashpercent” />
    <attribute name=”crashcount” />
    <link-entity name=”plugintype” from=”plugintypeid” alias=”PluginType” to=”plugintypeid”>
    <attribute name=”isworkflowactivity”></attribute>
    <attribute name=”typename”></attribute>
    <filter>
    <condition attribute=”name” operator=”eq” value=”MyProject.Workflows.ProcessSomething“></condition>
    </filter>
    </link-entity>
    </entity>
    </fetch>

Advertisements

2 Comments

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s