BizTalk Health Monitoring and Message Box Viewer

BizTalk Message Box Viewer (MBV):

BizTalk provides Message Box Viewer (MBV) tool out of box to generate report for BizTalk health and performance monitoring activity. It’s collecting details about BizTalk Server, Configuration, database and performance details.

MBV analyze in details of BizTalk MsgBox Db but also other BizTalk Dbs and generates an HTML file containing different REPORTS including a “WARNINGS REPORT” which report in yellow and red some warnings.

This tool comes along with BizTalk. When we installed BizTalk, this will available at below location:

C:\Program Files (x86)\Microsoft BizTalk Server 2013\SDK\Utilities\Support Tools\MsgBoxViewer

MBV.jpg

Dashboard and Warnings report can now be sent automatically by mail after a collect

email.jpg

Purpose of MBV:

  • Health Check tool to analyze quickly and easily a BizTalk group
  • Generates different sub-reports in HTML and plain text files
  • Targeted users : BizTalk Admins

Why MBV ?

  • Retrieve quickly maximum info from a BizTalk Production System in one click
  • Avoid sending too many queries and their usage to customers!
  • Identify all possible issues which could be critical or need attention
  • Present them in a user friendly Format
  • Emit suggestions via comments and Doc links

Features of MBV:

  • Provided in BizTalk 2013 and each next CU.
  • Support all BizTalk versions since BizTalk 2004
  • Safe to run in Production
  • No setup, just copy and run
  • Frequently used by both customers and MS engineers to check for the good health and integrity of a BizTalk group, and to retrieve a complete topology of the BizTalk Platform
  • Used by BizTalk Users to deliver Health Checks
  • Checks the integrity of BizTalk Databases and propose immediate remediation tasks to famous “Terminator” tool  (http://www.microsoft.com/en-us/download/details.aspx?id=2846)
  • Provided as both a GUI and a console application. The console version is used more and more as a custom monitoring tool to be scheduled as a task, generating periodically HTML reports but also sending periodically raised warnings in the App journal, and the warnings and dashboard in emails. It can so complement  ideally System Center Operations Manager (SCOM) and the BizTalk Mgmt Pack to provide a complete monitoring of a Production BizTalk platform
  • Query and Topology Reports
  • Support ALL versions of BizTalk including BizTalk 2K4 and 2K9
  • Can run on ANY type of Servers – SQL, XP/Vista client
  • Can analyze ALL BizTalk complex configurations  : Multiple Servers  & Multiple MsgBox databases
  • Easily Extensible (XML, DLL) – MS Internal for the moment

The goal of MBV is to retrieve the maximum amount of information from a BizTalk System in one click in the minimum amount of time, identify all possible issues which could be critical or need attention, and present them in a user friendly format.

Below is screenshot of Summary Report

mbv1.jpg

References:

https://blogs.technet.microsoft.com/jpierauc/2013/10/28/msgboxviewer-13-662-is-released-and-provides-great-new-features/

https://blogs.technet.microsoft.com/jpierauc/2007/12/20/what-is-biztalk-msgboxviewer-aka-mbv/

Advertisements

WCF-Custom (SQL) Receive Location Configuration with BizTalk in Multi-Server Environment

Issue:  In multi-server BizTalk environment, BizTalk WCF-Custom (SQL Binding) Receive Locations poll SQL Server more than one times instead of single polling.

Normally in multi-server BizTalk environment, we are creating host instances for each host on each BizTalk server. But BizTalk has little limitation for active/active multi-server setup.

Below are adapters which only work on single server host instance.

  • FTP
  • POP3
  • WCF-SQL (Receive Adapter)

Normally we get information on internet search about FTP and POP3 adapters for single server and forgot about mention about WCF-SQL Receive adapter. So when we deploy code on multi-server environment then we face multiple polling from WCF-SQL Receive location.

Reason: We have created two Host Instances for Host for Both BizTalk Servers and apply that host on Receive Location which is polling to Database.  So both Host instances are polling data from SQL that’s why BizTalk receive two time data.

If we apply multiple host instances on these adapters for Receive Location then every host instance will run and process data into BizTalk.

For more details you can refer to below link:

https://stackoverflow.com/questions/2165668/biztalk-receive-port-reading-twice-from-db

Resolution:

                     We can apply one of three approaches to resolve this issue. Below 2nd is mostly used in BizTalk development scenario.

  1. Create a Single Server Host Instance as Receive Handler for WCF-Custom (SQL) Receive Location
  2. In SQL, update Stored Procedure with lock for processing data and update the flag not to send any data 2nd time call until unless data is modified.
  3. Create Host Instance cluster which run on both machine. It will run as active/passive mode. If one server will unavailable then start polling with other server.

Below is link for setup for cluster Host Instances

         https://www.biztalkadmin.com/biztalk-active-active-cluster-and-msdtc/

ESB Exception Handling in Orchestration

In Orchestration when there is requirement is that all the exception should be logged into ESB Management Portal or ESB Exception Database, then we need to add some code in orchestration catch block to send exception to ESB Exception Db and Management Portal.

We need to follow the below process to send exception details to ESB

  1. Firstly we need to add bellow dlls in reference of the project. you can get these dlls in GAC

Dlls.jpg

2. Add exception massage in orchestration with  Message Type           Microsoft.Practices.ESB.ExceptionHandling.Schemas.Faults.FaultMessage     

   EsbExceptionMessage.jpg

3.  In catch block create a message Assignment shape and write below code

ExceptionMsg = Microsoft.Practices.ESB.ExceptionHandling.ExceptionMgmt.CreateFaultMessage();

ExceptionMsg.FailureCategory = “SystemException”;

ExceptionMsg.FaultSeverity =  Microsoft.Practices.ESB.ExceptionHandling.FaultSeverity.Error;

ExceptionMsg.FaultCode =”1″;

ExceptionMsg.FaultDescription = systemException.Message.ToString(); 

Microsoft.Practices.ESB.ExceptionHandling.ExceptionMgmt.AddMessage(ExceptionMsg, ReceiveMsg);

ExpressionShapeDetails.jpg

You can change the message description according Business requirement. In last line AddMessage function contains Exception message first and then provide that message which you want to get from ESB Portal.

4. Create a Correlation type and corresponding correlation set with below property assign in correlation type:

  • Practices.ESB.ExceptionHandling.Schemas.Property.Application
  • Practices.ESB.ExceptionHandling.Schemas.Property.FaultCode

CorrelationProperties.jpg

5. Define Correlation Sets

Correlation.jpg

6. Create a send port and assign exception message and initiate correlation set as define above. SendPortConfiguration.jpg

    7. Port Configuration should always be direct binding for this send port. As below                  screenshot.

PortConfuguration.jpg

 

Now build and deploy the application.

Resumable and Non-resumable message

When any message is suspended in processing then it’s log into the suspended instance in BizTalk admin console. Sometimes we can directly resume message from admin console and it’s processed successfully and some time it doesn’t.

In BizTalk, when message is suspended, then it’s fall into one of these two categories.

  1. Resumable
  2. Non-Resumable

Suspended_Message.jpg

Resumable:

The service instance is suspended. Admin may be able to resume the service instance from admin console just right click on suspended instance and resume. Messages which can be manually resumed.

Reason:

  • If the external resource (example webservice, folder location in case of file adapter) is not available then the message will go for suspend resumable mode.
  • Resumable means the service may be able to be recovered.

Resuming a messaging instance will do the following:

  • Resume the messaging instance.
  • Send the message to the send port.
  • The send port delivers the message to the destination; even if the send port is not in a Started state.

    Note that when you suspend a scheduled instance and then resume it, the instance goes into a dehydrated state.

Non-resumable:

In Non-resumable, service instance associated with the message is suspended, and cannot be resumed.

Messages, which typically hold metadata and cannot be resumed. They will either disappear when the corresponding resumable instance is resumed, or in other cases they might need manual termination. You can save the Messages referenced by the instance, and then you can terminate the instance.
Non resumable instances are not necessarily zombies. They might be, but they could also be:

  • Instances for debugging purposes (typically after a “no subscribers found” error)
  • Instances failed at receive time when using an Isolated Host.
  • It will go after number of retry that you have set in the sendport.  The message will go for Suspend non-resumable mode if the message is not proper format (or) if it is rejected by the target system.
  • The response waits timed out.
  • In an Ordered Delivery send port with continue on failure enabled, if there is a failure in the pipeline, mapping or transmission.
  • In an Ordered Delivery receive port, if the adapter is configured to suspend messages on non-resumable on failure.
  • In a two-way receive port, if the response message fails in the pipeline, mapping, or transmission.
  • In a two-way receive port, if the receive message fails in the pipeline, mapping or transmission. Individual adapter behavior may be different. For example, the HTTP adapter does not suspend messages by default but can be configured to do so.
  • And yes…Zombies.

Custom Pipeline

Microsoft provides lots of feature in BizTalk Pipeline with having inbuild pipeline components. E.g. JSON Encoder, Decoder, MIME/SMIME Decoder etc.

But sometimes we need add something in context of message then we need to do customization in pipeline, BizTalk provides the facilities to customise with help of Custom pipeline. C# and VB.Net languages are basically used to write code for this.

Below are few scenarios where you need to apply Custom Pipeline

  • Conversion from another format to XML vice-versa:

As we know BizTalk has the facilities to receive file in XML, Text, csv and json (from BizTalk 2013). But if you have any other format e.g. Excel or PDF, then you need to change the format from Excel to xml because BizTalk always plays in xml format. So here in custom pipeline you can write code in C# or VB.Net form.

  • Add namespace (this facility is existed in BizTalk pipeline, when use ESB pipelines)
  • Promote message context property on custom data
  • Set dynamic send port
  • Message archiving
  • Handling large message

Requirement To create a Custom Pipeline:

Dlls:

  • BizTalk.ExplorerOM.dll (C:\Program Files (x86)\Microsoft BizTalk Server 2013 R2\Developer Tools\Microsoft.BizTalk.ExplorerOM.dll)
  • BizTalk.Pipeline.dll (C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\PublicAssemblies\Microsoft.BizTalk.Pipeline.dll)
  • BizTalk.Streaming.dll (C:\Program Files (x86)\Microsoft BizTalk Server 2013 R2\Microsoft.BizTalk.Streaming.dll)

Pipeline Interfaces:

  • IBaseComponent
  • IComponent
  • IComponentUI
  • IPersistPropertyBag
  • IAssemblerComponent
  • IDisassemblerComponent
  • IProbeMessage

IBaseComponent Interface:

Defines properties that provide basic information about the component. Below are methods used in this component.

  • Description: Describe pipeline component description.
  • Name: Define the component name.
  • Version: Define the component version.

    public interface IBaseComponent

    {

        string Description { get; }

        string Name { get; }

        string Version { get; }

    }

IComponent Interface (COM)

Defines the methods used by all pipeline components except assemblers and disassemblers.

  • Execute: Executes a pipeline component to process the input message and get the resulting message.

   public interface IComponent

    {

        IBaseMessage Execute(IPipelineContext pContext, IBaseMessage pInMsg);

    }

IComponentUI Interface (COM)

Defines methods that enable pipeline components to be used within the Pipeline Designer environment.

  • Icon: Provides the icon that is associated with this component.
  • Validate: Verifies that all of the configuration properties are set correctly.

     public interface IComponentUI

    {

        IntPtr Icon { get; }

        IEnumerator Validate(object projectSystem);

    }

IDisassemblerComponent:

A disassembling component is a pipeline component that receives one message on input and produces zero or more messages on output. Below are two methods used in this component.

  • Disassemble: Performs the disassembling of the incoming document message.
  • GetNext: Gets the next message from the message set that resulted from disassembler execution. Returns NULL if there are no more messages.

    public interface IDisassemblerComponent

    {

        void Disassemble(IPipelineContext pContext, IBaseMessage pInMsg);

        IBaseMessage GetNext(IPipelineContext pContext);

    }

IAssemblerComponent

An assembling component is a pipeline component that receives several messages on input and produces one message on output. Assembling components are used to collect individual documents into the message interchange batch.

  • AddDocument: Adds the document message to the list of messages that will be included in the interchange.
  • Assemble: Builds the interchange from the messages that were added by the previous method. Returns a pointer to the assembled message.

     public interface IAssemblerComponent

    {

        void AddDocument(IPipelineContext pContext, IBaseMessage pInMsg);

        IBaseMessage Assemble(IPipelineContext pContext);

    }

IProb Message:

Defines the methods and properties for components that need probing functionality. Any pipeline component (general, assembling, or disassembling) can implement the IProbeMessage interface if it must support message probing functionality. A probing component is used in the pipeline stages that have FirstMatch execution mode. The IProbeMessage interface exposes a single method, Probe, which enables the component to check the beginning part of the message. The return value determines whether this component is run.

    public interface IProbeMessage

    {

        bool Probe(IPipelineContext pContext, IBaseMessage pInMsg);

    }

IPersistPropertiesBag Component:

Defines the methods to prepare for, load, and save the properties of pipeline components during design time.

This is responsible for getting the design time properties. If you need to have some properties be set during design time or during the deployment stage, you must add the loading and saving functionalities for those properties.

    public interface IPersistPropertyBag

    {

        void GetClassID(out Guid classID);

        void InitNew();

        void Load(IPropertyBag propertyBag, int errorLog);

        void Save(IPropertyBag propertyBag, bool clearDirty, bool saveAllProperties);

    }

Custom pipeline Components:

You can create three types of pipeline components:

  • General: This component can be fit at any stage of Receive and Send pipeline,
  • Assembling: This component can only fit at assemble stage of Send Pipeline
  • Disassembling; This component can only fit at Disassemble stage of Receive Pipeline

Developing a General Pipeline Component

A general pipeline component is a .NET or COM component that implements the following interfaces:

 Developing an Assembling Pipeline Component

An assembling component must implement the following interfaces:

  • IBaseComponent
  • IAssemblerComponent
  • IComponentUI
  • IPersistPropertyBag

Developing a Disassembling Pipeline Component

 A disassembling pipeline component receives one message on input and produces zero or more messages on output. Disassembling components are used to split interchanges of messages into individual documents. Disassembler components must implement the following interfaces:

  • IBaseComponent
  • IDisassemblerComponent
  • IComponentUI
  • IPersistPropertyBag