BAM in Orchestration

In BizTalk, BAM  applies on pipeline level and in orchestration. Here we are going to implement BAM in Orchestration through BAM API programming.

For more details of BAM APIs, you can refer to

https://vkbiztalk.wordpress.com/2017/07/17/bam-api/

Normally for BAM Programming below three functions are required under EventStream class.

  • BizTalk.Bam.EventObservation.OrchestrationEventStream.BeginActivity
  • BizTalk.Bam.EventObservation.OrchestrationEventStream.UpdateActivity
  • BizTalk.Bam.EventObservation.OrchestrationEventStream.EndActivity

If you want some more advance feature in BAM tracking then you can refer below reference:

https://msdn.microsoft.com/en-us/library/microsoft.biztalk.bam.eventobservation.eventstream.aspx

Requirements:

Add below Dlls in BizTalk project references:

  • Biztalk.BAM.Xlangs.dll
  • BizTalk.Bam.EventObservation.dll

Write below code in orchestration expression shapes:

//Create a new, unique activity identifier to use as the ActivityID in BAM

string activityId = Guid.NewGuid().ToString();

Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.BeginActivity(ActivityName, activityID);

// Updates the activity record.

Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.UpdateActivity ActivityName, activityID, “FIleName”, msg(BTS.FileName), “ProcessStart”, DateTime.UtcNow,);

// End the activity record.

Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.EndActivity(ActivityName, activityID);

Here you can use different expression shape for above methods, Be sure EndActivity method should be in last of orchestration. And StartActivity methad in starting of Orchestration.

Below is explanation of data and function which are used in orchestration expression shapes:

ActivityName: BAM Definition File Name

ActivityID: It can be a GUID or any unique id

UpdateActivity function:

public virtual void UpdateActivity( string activityName, string ctivityInstance,                     params object[] data)

In Update activity, after first two parameters, data items are next parameters which are defined in key value pairs.

These data items can be defined in single UpdateActivity function as well as in separate activity function. E.g. In below BAM table, there are following data items.

Data1, data2, starttime, endtime, errordetails, status, lastmodified.

Here we can write update function as per requirements, it’s not mandatory to define all data items in one time.

UpdateActivity(“BAMSample”, ActivityID, “starttime”, DateTime.Now);

UpdateActivity(“BAMSample”, ActivityID, “Status”, “OperationStart”);

BAMTable.jpg

You can track info on BAM Portal or BAM definition table in BAM Primary Import Database.

When you defining BAM Definition file then BAM creates five table in BAMPrimaryImport table.

https://vkbiztalk.wordpress.com/2017/06/14/implementation-of-bam-activity-definition-file/

You can check data in Bam_<BAM Definition File>_Completed table.

e.g. bam_BAMSample_Completed

How does EventStream work in Orchestration?

The OES API stores tracking data first in the BizTalk MessageBox database. Periodically the data is processed and persisted to the BAM Primary Import database by the Tracking Data Decode Service (TDDS).

The OES API is found in the Microsoft.BizTalk.Bam.EventObservation namespace.

Advertisements

BAM in Custom Pipeline

In BizTalk, BAM can be applied on pipeline level and in orchestration. Here we are going to implement BAM in BizTalk through BAM API programming.

For more details of BAM APIs, you can refer to

https://vkbiztalk.wordpress.com/2017/07/17/bam-api/

Normally for BAM Programming below three functions are required under EventStream class.

  • BeginActivity
  • UpdateActivity
  • EndActivity

But if you want some more advance feature in BAM tracking then you can refer below reference:

https://msdn.microsoft.com/en-us/library/microsoft.biztalk.bam.eventobservation.eventstream.aspx

Normally, we write the code to produce BAM events in the Execute method of the pipeline component.

The pipeline context has a GetEventStream method that returns a MessagingEventStream.

Requirements:

Dlls:

  • BizTalk.Bam.EventObservation.dll
  • BizTalk.ExplorerOM
  • BizTalk.Pipeline
  • BizTalk.Streaming

Namespaces:

  • usingBizTalk.Bam.EventObservation;
  • usingBizTalk.Message.Interop;
  • usingBizTalk.Component.Interop;
  • usingBizTalk.Streaming;
  • usingBizTalk.ExplorerOM;

Example:

public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext context, Microsoft.BizTalk.Message.Interop.IBaseMessage inMsg)

{

      EventStream BAMes;

BAMes = context.GetEventStream();

string ActivityID; // it can be a GUID

String operation; // message context properties or any custom value to track in BAM

BAMes.BeginActivity(ActivityName, ActivityID);

BAMes.UpdateActivity(ActivityName, ActivityID, “Data”, operation);

BAMes.EndActivity(ActivityName, ActivityID);

BAMes.Flush();

}

Here ActivityName: BAM Definition File Name

ActivityID: It can be a GUID or any unique id

UpdateActivity function:

public virtual void UpdateActivity( string activityName, string activityInstance,                   params object[] data )

In Update activity, after first two parameters, data items are next parameters which are defined in key value pairs.

These data items can be defined in single UpdateActivity function as well as in separate activity function. E.g. if a BAM table has following data items.

Data1, data2, starttime, endtime, errordetails, status, lastmodified.

Here we can write update function as per requirements, it’s not mandatory to define all data items in one time.

BAMes.UpdateActivity(“BAMSample”, ActivityID, “starttime”, DateTime.Now);

BAMes.UpdateActivity(“BAMSample”, ActivityID, “Status”, “OperationStart”);

You can track info on BAM Portal or BAM definition table in BAM Primary Import Database. You can check data in Bam_<BAM Definition File>_Completed table.

e.g. bam_BAMSample_Completed

BAMTable.jpg

When you defining BAM Definition file then BAM creates five table in BAMPrimaryImport table. you can refer to following reference to create BAM Definition file.

https://vkbiztalk.wordpress.com/2017/06/14/implementation-of-bam-activity-definition-file/

How does EventStream work in pipeline?

MessagingEventStream (MES) is used inside a BizTalk pipeline component to write Bam as part of the messaging transactions ensuring that your BAM event persistence remains in sync with the BizTalk pipeline transactions.

Messaging Event Streams are asynchronous and store tracking data first in the BizTalk MessageBox database. Periodically the data is processed and persisted to the BAM Primary Import database by the Tracking Data Decode Service (TDDS).

 

BAM API

In BizTalk, BAM (Business Activity Monitoring) can be applied on pipeline and in orchestration. BizTalk also provide Tracking Profile Editor tool to implement BAM in BizTalk,  You can get info about tacking profile editor from below reference:  https://msdn.microsoft.com/en-us/library/aa547038.aspx.

But this tracking tool has some limitation (for more info you can refer to https://blog.sandro-pereira.com/2010/08/20/bam-limitation-of-tracking-profile-editor-tpe ),  that’s why we need to move to apply BAM through programming. Here we are going to implement BAM in BizTalk through programming. Before going to coding we need to understand concept of BAM API, which is used in BAM Programming.

The BAM API defines four main classes:

  • DirectEventStream (Synchronous, no latency): used in a .NET application to do a buffered write of data to BAM. Here data is persisted synchronously to the BAM Primary Import database. Except for DES, all other EventStream classes are asynchronous and have some latency. The data is first persisted to MessageBox and subsequently processed by TDDS which moves data into the BAMPrimaryImport database.
  • BufferedEventStream (Asynchronous, high throughput, some latency): used in a .NET application to do an unbuffered write of data to BAM.
  • OrchestrationEventStream (Asynchronous, participates in BizTalk orchestration transactions): used within an orchestration; Provides transactional consistency with the orchestration.
  • MessagingEventStream (Asynchronous, participates in BizTalk Server pipeline transactions): used within a pipeline; Provides transactional consistency with the messaging engine.

All of these classes are derived from the base class EventStream.

Note:

For BizTalk OrchestrationEventStream (For Orchestrations) and MessagingEventStream (For Pipelines) are applicable and other two event streams are applicable for non-Biztalk e.g. WCF and WF. Link is below for how to apply BAM through BAM API in BizTalk Pipeline and Orchestration.

Apply BAM in Pipeline:

https://vkbiztalk.wordpress.com/2017/07/26/bam-in-custom-pipeline/

Apply BAM in Orchestration:

https://vkbiztalk.wordpress.com/2017/07/27/bam-in-orchestration/

The EventStream APIs for BAM reside in the Microsoft.BizTalk.BAM.EventObservation namespace. To code against the APIs, you must add the following DLLs to your project reference and include the namespace in your code with the following using statement:

using Microsoft.BizTalk.Bam.EventObservation;

  • BizTalk.BAM.EventObservation.dll (Available at C:\Program Files (x86)\Microsoft BizTalk Server 2013 R2\Tracking)
  • Biztalk.BAM.Xlangs.dll (Available at C:\Program Files (x86)\Microsoft BizTalk Server 2013 R2\Tracking): This DLL is required when coding for Orchestration event streams.
  • BizTalk.Pipeline.dll (Available at C:\Program Files (x86)\Microsoft BizTalk Server 2013 R2): This DLL required when using coding pipeline contexts for Messaging event streams.

Implementation of BAM Activity Definition file

When we install BAM in BizTalk Server then its add BAM (Business Activity Monitoring) in Add-Ins in Microsoft Excel File. For implementation of BAM or create Activity Definition file of BAM we need to follow the following process.

Open Excel File -> File -> Options -> Add-Ins

BAM1.jpg

Click on Go… Button as in above screenshot. Now below popup will open, Select the Business Activity Monitoring and click OK.

BAM2.jpg

Now click on Add-Ins Tab in Excel file as below screenshot. And select BAM Activity. as defined below.

BAM3.jpg

Now Activity Definition file will open as below:

BAM4.jpg

Here we need to define the data and milestone, which will need to capture from the BizTalk applications when message process.

Here I have defined some data as below screenshot:

BAM5.jpg

Item type as Business Milestone is related to Date Time.

BAM6.jpg

BAM7.jpg

BAM8.jpg

After defining Activity data, we can also define these things in BAM View as below to look more familiar to Business Users.

BAM9.jpg

BAM10.jpg

BAM11.jpg

BAM12.jpg

Now if require then we can go for below feature otherwise we can escape this.

BAM13.jpg

BAM14.jpg

Click next and finish the Activity Creation.

After finishing the creation of Activity details, we need to export then details in xml format as defined below:

BAM15.jpg

BAM Definition file will look like as below:

BAM16.jpg

Now we need to execute the below command to setup the activity Details in BAMPrimaryImport Database.

bm.exe deploy-all -DefinitionFile:<def file>[ -Server:<server> ][ -Database:<database> ]

BAM17.jpg

The BAM management utility deployment commands allow you to apply, modify, and remove definitions.

  • exe deploy-all: Deploys a BAM definition.
  • exe update-all: Updates a BAM definition.
  • exe remove-all: Removes a BAM definition.

BAM Portal:

Now you can check this on BAM Portal. If you run BAM Portal first time then need to set the Compatibility View Setting on Internet Explorer. If you want to open BAM Portal on Google Crome then you need to add IE Tab in Crome.

For more information about how to open BAM in different browser, you can refer below reference:

https://blog.sandro-pereira.com/2015/07/31/user-experience-running-bam-portal-in-different-browsers/

BAM18.jpg

For more information about BAM, please refer the below reference:

https://msdn.microsoft.com/en-us/library/aa558802.aspx

https://msdn.microsoft.com/en-us/library/aa560294.aspx

 

 

 

 

BAM Installation and Configuration in Multi server Environment

In Most of the Project development, BAM is always chosen as second phase. In Existing BizTalk Project, BAM will add as an additional layer to the solution.

In this blog We are explaning about BAM installation in Multi server Environment. The below BAM installation steps assumes that BizTalk is already installed and configured. To install BAM on BizTalk Servers, we need to again run the BizTalk server setup file on all BizTalk Servers and follow the below steps

Note: The Log on User account to install and configure BAM should have admin rights on SQL server and Biztalk both.

  1. When we run the set-up file, below page will be popup.

BizTalk1.jpg

  1. Click on Install Microsoft BizTalk Server 2013 R2

BizTalk2.jpg

  1. Select Modify as below:

BizTalk3.jpg

  1. Select Portal Component along with all others Component, which had already installed on the machine.

BizTalk4.jpg

Note: Please select all components which are already installed otherwise It will remove those component from the BizTalk Server and then we need to install all the components again.

  1. Select all BAM Component in Additional Software as Below:

BizTalk5.jpg

  1. Click Next and install.

Configuring BAM

As a part of multi-computer environment, while configuring BAM, we need to select the Custom Configuration option in the BizTalk Server Configuration tool on each Biztalk Server in the group.

1. Configuring BizTalk BAM Server

Here we assume that Biztalk Server is already installed and configured, we just need to add BAM Tool and BAM portal Configuration on the BizTalk Server. BAM also requires the following:

  • SQL Server Analysis Services (SSAS)
  • SQL Server Integration Services (SSIS)
  • SQL Server Database Mail or SQL Server Notification Services (SSNS)

To configure BAM

  1. Click Start, point to All Programs, point to Microsoft BizTalk Server, and then click BizTalk Server Configuration.

BizTalk6.jpg

  1. In the left pane, click BAM Tools, and then enter or select the following values:
Name Value
Enable Business Activity Monitoring tools (selected)
Enable Analysis Services for BAM aggregations (selected)
BAM Primary Import Database: Server Name the SQL Server host name
BAM Primary Import Database: Database Name the default name is BAMPrimaryImport
BAM Archive Database: Server Name the SQL Server host name
BAM Archive Database: Database Name the default name is BAMArchive
BAM Analysis Database: Server Name the SQL Server host name
BAM Analysis Database: Database Name the default name is BAMAnalysis
BAM Star Schema Database: Server Name the SQL Server host name
BAM Star Schema Database: Database Name the default name is BAMStarSchema

Screenshot as below:

BizTalk7.jpg

Note: The service account used for the OLAP service should have db_datareader permissions on the BAM Star Schema database.

  1. In the left pane, click BAM Alerts, and then enter or select the following values (Optional).
    BAM alerts require SQL Server Database Mail , and BAM tools to be enabled.
Name Value
Enable SQL Notification Services for BAM alerts (selected)
Windows service BTService
BAM Alerts SMTP Server (the SMTP server that will be used to send the BAM alerts.)
BAM Alerts File Location (the network share that will be used to store the BAM alerts.)
Note : We must manually create this share before BAM alerts can store the files.
SQL Server for Alerts Databases the name of the SQL Server
Prefix for Alerts Database Names (a prefix that will be used for the alerts databases)

Note: Do not install BAM Alerts unless you have a SMTP server available and planning to use BAM Alerts feature. SKIP installation of BAM Alerts by Unchecking the box in case it is already checked.

  1. In the left pane, click BAM Portal, and then enter or select the following values. In case you have not created domain groups for BAM , create the below three domain groups to control access to BAM:
  1. (domain)\BAM Portal Users
  2. (domain)\BAMAppPoolUser
  3. (domain)\BAMMgmtWSUser
Name Value
Enable BAM Portal (selected)
BAM Management Web Service user (domain)\BAMMgmtWSUser
BAM Application Pool Account (domain)\BAMAppPoolUser
BAM Portal Users (domain)\BAM Portal Users
BAM Portal Web Site Default Web Site

Note: BAM Management Web Service user and BAM Application Pool Account can be service account but service account will have admin access on BizTalk server.

Screen Shot as below which uses a Bam Portal users group as everyone which is not recommended in production environments and create a separate “BAM Portal Users” in domain.

BizTalk8.jpg

  1. Click Apply Configuration.
  2. On the Summary page, to apply the configuration, click Next.
  3. Verify that the Configuration Result is Success, and then click Finish.
  4. Close Microsoft BizTalk Server 2013 R2 Configuration.

2. BAM Portal Configuration

After BAM Configuration with BizTalk Server Configuration. We need to check BAM Portal and config setting (if Require) on IIS.

BizTalk9.jpg

Select BAM in Left side and open Authentication in Right side of IIS window as mentioned below screenshot:

BizTalk10.jpg

Now Select Window Authentication and Click on Provider.. as mentioned in Below Screenshot.

BizTalk11.jpg

Note: Only Enable ASP.NET Impersonation and Windows Authentication.

Select NTLM and Click Move Up Button. NTLM should be the first in the list of Providers

BizTalk12.jpg

When we browse BAM Portal, it will open as below:

BizTalk13.jpg

Note: BAM Portal works on Internet Explorer. For Chrome , a separate Extension IE tab is required.

Reference

Resource Link
Installation of BAM http://go.microsoft.com/fwlink/p/?LinkID=208597
Installation of BizTalk https://msdn.microsoft.com/en-us/library/jj248681.aspx
Installation & Configuration Issue https://blog.sandro-pereira.com/tag/bam/