BizTalk Exception: HRESULT=”0x80040e57″ Description=”The statement has been terminated.” HRESULT=”0x80040e57″ Description=”String or binary data would be truncated.

Exception:

HRESULT=”0x80040e57″ Description=”The statement has been terminated.”
HRESULT=”0x80040e57″ Description=”String or binary data would be truncated.

Explanation:

This exception is occurred when we define FaultCode  more than 20 characters.

When we are using ESB management Portal for exception handling then ESB Exception on port level (Send & Receive Port) is directly stored in ESB management portal as well as ESB Database through checking Enable routing for failed message check box on port.

We can track exception in fault table in EsbExceptionDB.

But in exception handling in orchestration level, we need to define some of ESB context property to our self and send file to message box through direct binding from where ESB Application pick the file and upload on database as well as management portal.

below are details which are defiled in orchestration:

msg_Fault = Microsoft.Practices.ESB.ExceptionHandling.ExceptionMgmt.CreateFaultMessage();
msg_Fault.FailureCategory = “CategotyName”;
msg_Fault.FaultSeverity = Microsoft.Practices.ESB.ExceptionHandling.FaultSeverity.Severe;
msg_Fault.FaultCode = “FaultCode_name”;
msg_Fault.FaultDescription = exceptionObject.Message;

here exceptionObject should be catch exception object name or we can put custom exception description also.

And msg_Fault type should be Microsoft.Practices.ESB.ExceptionHandling.Schemas.Faults.FaultMessage defined under Orchestration Message folder.

Solution:

msg_Fault.FaultCode should be defined within 20 characters it should not be more than that otherwise we get above exception.

Advertisements

BizTalk Production Support

There are two kind of job in BizTalk world.

first is as BizTalk developer where people use to create BizTalk applications with help of VS. In creating of BizTalk application we are mostly use schema, pipeline, orchestration and mapping. After development the project its deploy on BizTalk Admin console.

Second is BizTalk Production support. In Production support we are basically taking care of BizTalk applications in BizTalk Admin console. There are following task should be doing in Production support.

Production Support Activity:

  1. Monitoring: Every day we need to monitor BizTalk Admin Console that every thing is running as expected or not. Currently in the market there are some tool is available for monitoring but from my point of view we should monitor production server once in a day.

Following point which should check in Monitoring:

  •      Check Host  Instances is running
  •      Check Application Status is running. Sometimes we need to stop receive location or Send port as per business requirement, in that case we need to check the running status for Receive location, Send port, Send port group, and orchestration
  •      Check Running instance in BizTalk Group Hub. Running state should be empty or its count should be increase and decrease. If Running state count is only increasing, then it’s problem. It may happen when BizTalk server hang up. its very rare scenario but it can be happen. then we should restart the related and BizTalk default host instances
  •     Check the Suspended Instances. If message available in this instance then we need to figure out about the exception and ask to development team to work on that. And one more important thing. We should terminate the suspended instance. We should not keep suspended instance for long time e.g. 5-7 days because when Suspended Instances count is increase, then it increase load on Message Box database and its show the BizTalk performance and service. So be care in handling suspended instance
  • Do other activity e.g. failed message reprocess as per business, analysis or exception, BizTalk Health monitoring etc.

2. Re-Process Activity

Some times message is failed in BizTalk due to varius reason e.g. Destination side server unavailavle, timeout or network connection problem or Business validation. In these case we need to re process failed files. When we developing BizTalk then we need to take care of re process activity. We need to create Receive port and receive location to re-process the file.

How to get message from Suspended Instance:

BizTalk Admin Console->BizTalk Group Hub->Suspended Service Instances -> double clik on any instance -> Message -> New message pop up open-> Body

3. Exception Management (Incident/Problem management)

4. Deployment:

BizTalk Application deployment on UAT/Production Server is also part of Production Support Activity.

There are several way for deployment activity, some of as below:

Manual Deployment: In this deployment process we need to deploy MSI, Binding,SSO and others manually one by one.

BTDF: This is automated deployment in which configuration info defined in excel file and all information is wrap up in MSI. Deployment team only needs to run the MSI file with small configuration.

BAT File/Poweshell Script: This is also the automated deployment. Here every thing e.g. MSI,Binding,SSO information (source and destination) set in script and run on the server.

BizTalk Health monitoring

  1. Health check up for BizTalk Server as well as BizTalk database. In BizTalk 2013 R2 , BizTalk health monitoring tool is added in BizTalk admin console. this tool provides the status of BizTalk Database, MsgDB and MngtDB. how much data use from database. MsgDB database size should be verry from time to time. it should not be increasing.
  2. need to check spool table and its row count should verry. its row cound should not be increasing order day by day. Spool table contains a record for each message in the system (active or waiting to be garbage collected). Monitoring the number of rows in this table, and the number of messages received per second, while increasing system load provides an easy way to measure the maximum.
  3. BizTalk Server CPU utilization.

Exception Handling in BizTalk

Exception handlin in BizTalk: It is most important point in BizTalk to handing exception. If we don’t apply exception handling in our BizTalk application then when ever message failed in processing then its suspend and store in BizTalk Message box and availabe in suspended instance in BizTalk admin console. It will create heavy cost for BizTalk performance as well as production support activity.

Here I say that If exception is not handle proper way then it will create problem for BizTalk Performance. for this we need to understand some activity related to message box database (BizTalkMsgDb). As we know every message stored at message box database and process. And Message box db deletes the message when it successfully send. So  when message failed and availabe at suspended instance in admin console, then it stored in database and will not be deleted untill unless we terminate instance. BizTalkMsgDb size is fixed mostaly with 4 GB.

There are two way to handle exception

  1. ESB Management Portal
  2.  Create custom pipeline or orchestration to track the exception and send alert.

How to apply Exception handling on BizTalk Application

  1.  Exception Handling at port: We can handle exception at port level through Enable Routing failed meesage check box.
  2.  Exception Handling in orchestration through custom coding in catch block.

For more info about how to handle exception at BizTalk Port level, you can refer to below reference:

https://vkbiztalk.wordpress.com/2017/01/15/exception-handling-at-port-level/

Difference between BizTalk and SSIS

BizTalk and SSIS both are Microsoft product and both are designed for integration. But both are used in different prospective. Below we are discussing main difference between them.

BizTalk:

BizTalk is based on publish-subscribe model and used for EAI (Enterprise Application Integration). It is used for combining EAI and B2B Integration. It is used for process workflow, real time processing, network based progression, time-consuming commercial process. like ATM machine- whenever we swap card, we get information at that time like balance inquiry or cash withdraw.

BizTalk would be used in following scenario:

  1. Real time processing
  2. Process Integration (EAI or B2Bi)
  3. Process Management
  4. long-running and transactional business processes
  5. For low message size. message size should not be exceeding to 1 MB.
  6. BizTalk is used for complex transaction
  7. BizTalk is not only for simple message transaction. It has its own feature to provide as SOA kind of feeling.
  8. BizTalk provides the operations to be perform on one message at time or real-time processing. Because everything in BizTalk is XML so BizTalk is very slow on large set of data.

SSIS:

SSIS is used for batch processing. It works for ETL (Extract, Transform and Load) model with simple business transaction logic. It mostly works for data transaction between different databases and some others e.g. file.

SSIS should be use in following scenario:

  1. for batch processing.
  2. for Simple transaction e.g. transferring data from one database to another database. SSIS should not be used for complex business logic.
  3. SSIS is used for heavy data transactions.
  4. File size could be in MB or GB in transactions doesn’t matter.
  5. SSIS runs through schedule jobs, in real scenario when we transfer money through online, then it will not transfer to at that time. it will take time. so here we can say that Bank has some schedule jobs for transferring money which is running 2 or 3 hours interval. This is the best example to understand that how is SSIS working.
  6. SSIS package are used to migrate large set of data or dataset.
  7. custom tracking system in SSIS which require a lot of coding
  8. When we have large data, required less complexity and requirement of integrated systems are based on Same technology then we have to use SSIS. Usually SSIS used to migrate or integrate the non-transaction data or step up data. The delay of migration and integration possible or example Batch processing. SSIS is built for ETL process, it is not rapid integration tool.

Below is difference between BizTalk and SSIS:

BizTalk SSIS
BizTalk is a licence tool and its require SQL Server for integration. SSIS is component of SQL Server licence.
BizTalk is used for integration and migration of transactional data SSIS is used for integration and migration of Non-Transactional data
BizTalk is rapid development tool as compare to SSIS it is not rapid integration tool
When we required less data integration/migration and require complex decision making we are used BizTalk for example we have to implement complex work flow on single record. When we have large sum of data, required less complexity and requirement of integrated systems are based on Same technology then we have to use SSIS. Usually SSIS used to migrate or integrate the non-transaction data or step up data.
Allowing for interaction with multiple platforms through adapters supporting multiple protocols (SQL, File, SOAP, HTTP, POP3, etc) and applications (SAP, Siebel, JDE, etc) using a variety of protocols (SQL, FTP, SOAP, etc) allowing for transformation/cleaning/aggregation between those data sources
BizTalk does not support batch processing. BizTalk always plays with XML format. For handling other format of Data, BizTalk has pipeline component, which convert data format to xml vice-versa. SSIS support Batch Processing
BizTalk has a lot of rich set of Adapters (SAP, MSMQ, MQ Series, FTP, File, etc.) which are may or may not depend on OLEDB Connection.

SSIS is totally depend on OLE Database (OLEDB) connection

BizTalk has built in Tracking System named BAM Portal, which can track in process data into BizTalk and show to Business users. Custom tracking system in SSIS which require a lot of coding

 

Business rule engine. BRE provide the condition whose value can be changed and complete follow of BizTalk application These functionalities can be achieved on config files in SSIS.

BizTalk Design Pattern

BizTalk provides some sample and best design pattern which are helpfull to create new BizTalk application.

You can get these sample applications in BizTalk development server at below location:

C:\Program Files (x86)\Microsoft BizTalk Server 2010\SDK\Samples

Bellow are brief description about these design pattern.

Aggregator

Aggregator is the pattern of receiving information (message) from multiple sources and consolidating it into a single message. For an example of this pattern, see Aggregate.odx in Aggregator (BizTalk Server Sample).

Calling Pipelines from an Orchestration

You can call send and receive pipelines from your orchestrations. This allows the reuse of pipelines and helps maintain the decoupling of an orchestration from the pipeline stages. For an example of this pattern, see Aggregate.odx in Aggregator (BizTalk Server Sample). Another example is CMP.odx in Composed Message Processor (BizTalk Server Sample).

Composed Message Processor

In this design pattern we receive file in bach or multiple file at a time then debatch this file into multiple files and also before sending to send port its asseble all files into single file.

Composed Message Processor is the pattern of processing individual items from an aggregated or batched interchange message. For an example of this pattern, see CMP.odx in Composed Message Processor (BizTalk Server Sample).

Content-Based Router

THis pattern is basically used for message transaction on the basis of Message context properties. e.g. on send port filter you can add ReceivePort name contest property (BTS.ReceivePortName) to to send all file wich is received from specific Receive Port.

Content-Based Router is the pattern of determining the recipient of a message based on some part of the message content. For an example of this pattern, see CBRSample (BizTalk Server Sample).

Dynamic Router

Dynamic Router is the pattern of determining the destination address as well as the transport protocol based on the result of message processing. You can use a dynamic send port or a Role Link shape to implement this pattern. For an example of this pattern, see ReceiveSend.odx in SendMail. Another example is SupplierProcess.odx in PartyResolution (BizTalk Server Sample).

Error Handling

BizTalk Server allows you to designate automated handling of messaging failures as an alternative to the default behavior of placing failed messages in the Suspended queue. You can route failed messages to a subscribing port for reporting or processing. For an example of this pattern, see ResubmitLogic.odx in Error Handling (BizTalk Server Samples Folder).

Exception Handling and Compensation

You can use an exception handler and a Throw Exception shape or an Expression shape for exception handling. For example, you can place the following code in the Expression shape to throw the exception:,

excp = new System.Exception(); throw(excp);

You can use a compensation block and a Compensate shape to perform the compensation on the transactions that have been committed. For an example of this pattern, see UpdateContact.odx in Compensation (BizTalk Server Sample). Another example is in Custom Exceptions.

Message Broker

Message Broker is the pattern of determining the destination of a message and still maintaining control over the message flow. For more, seeProcessing in the OrderBroker Orchestration.

Message Filter

The Message Filter pattern selects messages meeting particular criteria for processing. You can implement this pattern by adding the filter expression to an activated Receive shape. For more information, see Using Filters With the Receive Message Shape.

Message Translator

The Message Translator pattern converts a message from one form to another form. You can implement this pattern by using a BizTalk map with a Transform shape in an orchestration. For an example of this pattern, see HelloOrchestration.odx in HelloWorld (BizTalk Server Sample).

Parallel Convoy

The Parallel Convoy pattern enables multiple single items to join together to achieve something that an individual item cannot accomplish by itself. The set of related items can arrive in any order, but BizTalk Server must receive all of them before starting the process. For an example of this pattern, see http://go.microsoft.com/fwlink/?LinkId=56035.

Scatter and Gather

The Scatter and Gather pattern enables messages to be sent to multiple recipients and messages to be received back from each recipient. You can implement this pattern by using the Splitter pattern and the Aggregator pattern. You use the Aggregator pattern to assemble the results from using the Splitter pattern and put them under a Parallel Actions shape. For an example of the Splitter pattern, see SDK sample named Implementing Scatter and Gather Pattern at http://go.microsoft.com/fwlink/?LinkId=65185.

Sequential Convoy

The Sequential Convoy pattern enables multiple single items to join together to achieve something that an individual item cannot accomplish by itself. A sequential convoy is a set of related items that have a predefined order. Although the items do not have to be exactly the same, BizTalk Server must receive the items in a sequential order. For an example of this pattern, see http://go.microsoft.com/fwlink/?LinkId=56035.

Splitter

The Splitter pattern takes a single message and splits it into multiple messages.

Suspend with Retry

The Suspend with Retry pattern enables the orchestration to suspend a message when there is an error. The suspension occurs within a loop so that the orchestration suspends, asks for intervention, and then retries the operation a fixed number of times.