Today I had for specific reasons to install the older BizTalk 2006 R2 on a Windows 2008 R2 server. Now although this leads to an unsupported scenario; it is still valid for development environments (as in the case I encountered). Another issue that complicated the problem is that this server already had Visual Studio 2010 with .NET framework 4.0 installed on it.

I had already also installed SQL server 2008 on this server. So I started to follow the BizTalk 2009 installation guide. It went actually very smooth (to my surprise). The issues or notes I faced were as follows.

1-    First the prerequisites file it downloaded was kind of strange for me. It downloaded a file called “BTSRedistVistaEn64.cab”. So this is a file that should be directed to Vista and not 2008 R2. But it was installed with no problems.

2-    The MSDTC have to be configured as per the BizTalk 2009 guide to get it BizTalk to be configured properly. This is done as below image:

3-    Since I also had SQL express installed on the same box I had to disable it to make sure it does not interfere with the configuration.

4-    I installed VS 2005 with no noticeable issues.

5-    I then installed BizTalk and had no issues there.

6-    The main problem I faced was related to ENTSSO configuration. First I could not get it to start configuring it as it reported that the SSODB does not exist on the SQL server! So I am thinking; but I want to create it. Why does it need this DB although it will be created? After several hours of research I found the root cause. The problem was related to that the ENTSSO was using a wrong set of assemblies since its SQL communication COM+ component was registered using the REGASM utility of .NET4.0. The resolution was to re-register this assembly using the .NET 2.0 frameowork utility. To do so.

a.     Open a command prompt.

b.     Go to C:\Windows\Microsoft.NET\Framework64\v2.0.50727

c.      regasm “C:\Program Files\Common Files\Enterprise Single Sign-On\win32\ssosql.dll”

d.     regasm “C:\Program Files\Common Files\Enterprise Single Sign-On\ssosql.dll”

7-    I also needed to disable the shared memory protocol on the SQL server.

8-    For the BAM I was not yet able to get it to work or configured as it reported that the SSAS is not a supported version. I will do further investigation on how to configure BAM on this configuration.

There are already several standard and well documented methods for creating a NLB cluster for BAM Portal like here. The problem with all these methods is that they tend to be very complex and long. Usually you would end up missing some steps and it will not work.

What I was able to find is a very simple and short way to configure BAM portal on several servers using Windows NLB.

1. Start by configuring BAM Portal on the first server “Server1” using the BizTalk Configuration Tool (normally).
2. Open the SQL management studio.
3. Open the Database “BamPrimaryImport”.
4. You will find a table called ”bam_Metadata_Properties”.
5. Remove the row for the value “BAMVRoot”
6. Now login to the second server “Server2”.
7. Open the BizTalk Configuration Tool and WOW you can use it to configure the Portal on this node also.
8. Repeat steps 2-7 for as many nodes you have in your NLB.
9. Now you are not done yet please continue. Change the web.config on all nodes so that the management web service and query web service URLs are pointing correctly to the NLB address and not the nodes addresses.
10. Now perform the steps below to update the BAM configuration for the location of the cluster.

So you are implementing a custom BizTalk tracking solution using the BAM APIs. In this solution you need to track when the message arrives at the receive port exactly and even before the receive pipeline starts to execute. so simple right you just end up doing something like this.

BUT BAM (not the BizTalk BAM but something blowing up) a NullReferenceException is thrown. Strange!!! and also the strangest part is that you will find that on some environments it works and on others it throws this exception.

Well the  issue here is related to the tracking setting for the receive port. If you enable tracking of message bodies on the receive port somehow this property is removed from the message context.

I faced a strange issue were I was trying to deploy a new BAM activity and view using the bm.exe tool. Whenever I executed the deploy-all command it simply reports that it deployed the activity and fails while deploying the view with an error like this:

OLE DB error: OLE DB or ODBC error: The SELECT permission was denied on the object ‘bam_<Some view name>_RTATable’, database ‘BAMPrimaryImport’, schema ‘dbo’.; 42000.

I then go to the BAMPrimaryImport database and find that nothing is deployed!

Actually this is not entirely true as when I opened a SQL profiler trace I found that it is really deploying the activity and then re-trackting everything once the view deployment fails. So what I found is that the utility would execute a SELECT statement on the given table in the BAMPrimaryImport database using the identity of the SQL server service account!!! And as this account really has no access to the database it fails! I do not know why it tries to use the service account but I was able to resolve this by giving the SQL service account db_datareader permissions to this database and it turned out in this case also I need to give this account the BTS_Admin_Users permissions on the BAMStarSchema database.

This post will detail how to configure the ESB management portal on a multi-machine environment with minimal access configuration in mind. The environment consists of mainly three server roles:

1.       An active directory server role.

2.       A SQL server role.

3.       An application server role hosting both the BizTalk server along with the management portal itself.

Assume that we have the domain name as: esbtest.local

1         Introduction

The ESB Management Portal, included as part of the Microsoft BizTalk ESB Toolkit, is a sample site that demonstrates the use of metrics and the possibilities that exist for extending the BizTalk ESB Toolkit. The portal provides a starting point from which you can build your own customized portal.

The ESB Management Portal is also a highly capable and useful tool that can help to maximize the use and efficiency of the ESB exception management system. In addition, the portal provides a user interface for an underlying Universal Description, Discovery, and Integration (UDDI) registry server and graphical configuration capabilities for features such as auditing, viewing history information, configuring alerts and notifications, and allowing users to subscribe to alerts that indicate faults occurring in ESB applications.

2         Preparation Steps

These steps need to be performed prior to installing the ESB management portal as per the next section. These include creating the domain passwords and granting them the proper rights.

2.1       Domain Service Accounts

During the setup process you will need to have created a service account that will be used for the following.

1-      Web applications (Portal, Exceptions service, UDDI service, and BAM service).

2-      The Alerts windows service.

3-      The UDDI publisher windows service.

This account will not be used to access any SQL server databases as all SQL access will be done impersonating the portal user account. In this case we will not need to have created any access rights on SQL server side for this account.

This account will need to be used as an application pool so it should be added to the local IIS WPG group. It will also be accessing the ESB Toolkit event viewer so it should have write access to the “Applications” event log.

Assume we created the account as “saESBPortalAppPool”.

We will create also a new group that will include all the ESB portal administrators. Call this group “gEsbPortalAdmins” for example.

2.2       The Identity Delegation Steps

Since this user will be used to impersonate the portal user identity and it will try to access SQL server resources using this user identity; a double hop will happen and to allow this user identity delegation rights should be given to the application pool user used. The procedure steps to perform this are listed below.

2.2.1        To create a service principal name for IIS that impersonates a user

1.     Log on to the domain controller.

2.     Open a command prompt.

3.     Run the following set of commands:

setspn -A HTTP/<ApplicationServer1Name> ESBTEST\<ESBPortalAppPool>

setspn -A HTTP/<ApplicationServer1Name>.esbtest.local ESBTEST\<ESBPortalAppPool>

setspn -A HTTP/<ApplicationServer2Name> ESBTEST\<ESBPortalAppPool>

setspn -A HTTP/<ApplicationServer2Name>.esbtest.local ESBTEST\<ESBPortalAppPool>

By running these commands, the user can use any of the names when calling the Web or Windows Communication Foundation (WCF) service.
OR
if you are using the portal using a load balancer then you need to run the commands:

setspn -A HTTP/<LoadBalancerName> ESBTest\<ESBPortalAppPool>

setspn -A HTTP/<LoadBalancerName >.esbtest.local ESBTEST\<ESBPortalAppPool>

For our test environment we have only one server with no load balancer so we issue the commands:

setspn -A HTTP/ESB-Test-Bts ESBTEST\saESBPortalAppPool

setspn -A HTTP/ ESB-Test-Bts.esbtest.local ESBTEST\saESBPortalAppPool

2.2.2        To create a service principal name for SQL that is consumed by IIS that impersonates the user

1.     Log on to the domain controller.

2.     Open a command prompt.

3.     Run the following set of commands:

·         If SQL is running under a domain identity, run the following commands:

setspn -A MSSQLsvc/<Database Cluster Name>:<Named Instance Port Number> ESBTest\<SQL server service account>

setspn -A MSSQLsvc/<Database Cluster Name>.esbtest.local:<Named Instance Port Number> ESBTest\<SQL server service account>

The following provides more information about the preceding commands:

a.     Named Instance Port Number is the port number that the named instance hosting the application databases is using.

b.     MSSQLsvc is the SQL service for which you are registering the Service Principal Name (SPN).

By running these commands, the user can use any of the names when connecting to an instance of Microsoft SQL Server.

An example of these commands in our test environment are as follows. Assuming that we have a named instance working on the port 55738.

setspn -A MSSQLsvc/ESB-Test-SQL:55738 ESBTest\saSQLSvcESB

setspn -A MSSQLsvc/ESB-Test-SQL.esbtest.local:55738 ESBTest\saSQLSvcESB

2.2.3        To configure the account for trusted for delegation

1.     Log on to the domain controller.

2.     Start the Microsoft Management Console (MMC) Active Directory Users and Computers snap-in.

3.     In the left pane of the MMC snap-in click Users.

4.     In the right pane, double-click the account identity under which your application pool runs (ESBTEST\<ESBPortalAppPool>) to display the Properties dialog box.

5.     On the Delegation tab of the Properties dialog box for the WCF server computer, Do not trust the user for delegation is selected by default. Select Trust this user for delegation to any service (Kerberos only). This is shown in the snapshot below.

2.3       Install The Microsoft Chart Controls for .NET framework 3.5

Install the MSCHARTS package on the server.

3         Portal Deployment Steps

3.1       The Portal Management Database

You need to create the ESB portal management database ESBAdmin on the SQL server cluster.

3.2       The Portal Web Applications

You need to install the ESB management portal web application and the supporting WCF services on both application servers that will be hosting the portal. Follow the following steps to complete the installation and configuration of the web applications.

3.2.1        Create the Application Pool

1-      Open the IIS management console by running the command INETMGR

2-      Expand the server node.

3-      Right click on “Application Pools” and click add application pool.

4-      Enter the name to be “ESBPortalAppPool”.

5-      Select .NET framework 4.

6-      Make sure “Classic” mode is used.

7-      Click ok.

8-      Click on the “Application Pools” node from the left.

9-      Right click on the application pool “ESBPortalAppPool” and click advanced settings.

10-   Make sure you set enable 32 bit applications to false.

11-   Under identity click on the ellipses button.

12-   Enter the application pool domain account already created previously in the preparation steps.

3.2.2        Install the Web Application Package

1-      Open the folder where you have built the ESB management portal installation package.

2-      Run the Setup.exe

3-      Select the ESBPortalAppPool as the web application pool.

4-      Finish the installation.

3.2.3        Change the Authentication Mode

1-      Open the IIS management console by running the command INETMGR

2-      Expand the server node.

3-      Expand the node “Default Web Site”.

4-      Click on the web application “ESB.Portal”.

5-      Double click in the right panel on the “Authentication” icon under IIS category.

6-      Make sure that it is configured as the below screen:

7-      Right click on “Windows Authentication” and click “Advanced”.

8-      Make sure you uncheck the check box for the Kernel mode authentication.

9-      Click ok.

10-   Right click on “Windows Authentication” and click “Providers”.

11-   Remove all providers added by default and only add the “Negotiate:Kerberos” provider.

3.2.4        Change the ESB Portal Web.Config

1-      Open the Web.config file from the folder “C:\inetpub\wwwroot\ESB.Portal” in notepad

2-      Search for the DB connection string “AdminDatabaseServer” near the top of the file.

3-      Change the connection string to the proper value to connect to the SQL server database already created ESBAdmin.

4-      Make sure that we have an authorization line for the “gEsbPortalAdmins” group.

5-      Search in this file for any occurrence of “Ntlm” and replace that with “Negotiate”.

6-      Save the file and close it.

7-      Open the Web.config file from the folder “C:\inetpub\wwwroot\ESB.Portal\Admin” in notepad

8-      Make sure that we have an authorization line for the “gEsbPortalAdmins” group.

9-      Save the file and close it.

3.2.5        Change the Exceptions Service Web.Config

1-      Open the Web.config file from the folder “C:\inetpub\wwwroot\ ESB.Exceptions.Service” in notepad

2-      Search for the DB connection string “EsbExceptionDbConnectionString” near the top of the file.

3-      Change the connection string to the proper value to connect to the SQL server database of the ESB exceptions default name is “EsbExceptionsDb”.

4-      Search in this file for any occurrence of “Ntlm” and replace that with “Negotiate”.

5-      Save the file and close it.

3.2.6        Change the UDDI service Web.Config

1-      Open the Web.config file from the folder “C:\inetpub\wwwroot\ESB.UDDI.Service” in notepad

2-      Search for the UDDI server URL string “http://esbwssit.esbtest.local/uddi” and change all occurrences with the proper location of the UDDI server was deployed on.

3-      Save the file and close it.

3.2.7        Change the BAM service Web.Config

1-      Open the Web.config file from the folder “C:\inetpub\wwwroot\ESB.BAM.Service” in notepad.

2-      Search for the DB connection string “BAMEventSource” near the top of the file.

3-      Change the connection string to the proper value to connect to the SQL server database of the ESB exceptions default name is “BAMPrimaryImport”.

4-      Save the file and close it.

3.2.8        Change the Services Application Pool

Make sure that the services installed “ESB.BAM.Service”, “ESB.Exceptions.Service”, and “ESB.UDDI.Service” are all using the application pool created earlier for the ESB portal.

3.2.9        Change the Services Authentication Mode

This will be done for the web applications “ESB.BAM.Service”, “ESB.Exceptions.Service”, and “ESB.UDDI.Service”.

1-      Open the IIS management console by running the command INETMGR

2-      Expand the server node.

3-      Expand the node “Default Web Site”.

4-      Click on the web application to change. (this should be done for the three listed applications above)

5-      Double click in the right panel on the “Authentication” icon under IIS category.

6-      Make sure that it is configured as the below screen:

7-      Right click on “Windows Authentication” and click “Providers”.

8-      Remove all providers added by default and only add the “Negotiate” provider.

3.2.10    Verify the Configuration Sections Locking

You need to go through the below steps to verify that it is configured as shown.

1-      Open the IIS management console by running the command INETMGR

2-      Expand the server node.

3-      Click on the node “Default Web Site”.

4-      Click on “Configuration Editor”.

5-      Open the Section combo box and select the handlers node under “system.webserver”

6-      Make sure in the actions pane it is saying Lock and not unlock as per the screen below.

7-      If it is saying unlock then press it to unlock this section.

8-      Open the Section combo box and select the modules node under “system.webserver”

9-      Make sure in the actions pane it is saying Lock and not unlock as per the screen below.

10-   If it is saying unlock then press it to unlock this section.

3.3       Security Requirements

3.3.1        Give the Operators Group “gEsbPortalAdmins” Write Access to the Application Event Log

This is needed as the portal will impersonate the identity of the user and if an error happens then it will need this write access to allow it to write to the server event log. To perform this follow these steps on all servers hosting the portal.

1-      Download the PsTools from the link: http://technet.microsoft.com/en-us/sysinternals/bb896649

2-      Extract the download to any folder.

3-      Open a new command prompt window.

4-      Change to the PsTools folder.

5-      Execute the command “PsGetSid.exe gEsbPortalAdmins”.

6-      Mark the SID returned by this command as it will be needed later.

7-      Execute the command “wevtutil gl Application”.

8-      Under the section “channelAccess:” copy the entire string starting with “O:BAG:SYD:”.

9-      Execute the command “wevtutil sl Application /ca:O:<Copied Channel Access string got in step 8>(A;;0x3;;;<Group SID got in step 6>)”

3.3.2        Give the Operators Group “gEsbPortalAdmins” Full Control on the BizTalk WMI Model

This is needed to allow the operators group to get the list of BizTalk applications from the portal.

1-      Open “Computer Management”.

2-      Expand “Services and Applications”

3-      Left click on “WMI Control”.

4-      Then right click on “WMI Control”.

5-      Click “Properties”.

6-      Click “Security” tab

7-      Expand the root one level.

8-      Search for “Microsoft BizTalk Server” then click “Security”

9-      Click “Advanced”

10-   Make sure that the check box below is checked and click “Add”

11-   Search for the operators’ group “gEsbPortalAdmins” and give it “Allow” access to everything as below.

12-   Click ok and close the computer management.

3.3.3        Add the Operators Group “gEsbPortalAdmins” to the BizTalk Operators Group

This is needed to give the operators group the required database access to the BizTalk server databases. This can be done in two ways.

1-      Either we give the operators group the same access level as the BizTalk operators group on all BizTalk databases using SQL server management studio. This should be done on all BizTalk databases.

a.       Add a new login to the group “gEsbPortalAdmins” to the SQL server.

b.      Under the tab “User Mapping” make sure you select the databases BizTalkDTADb, BizTalkMgmtDb, and BizTalkMsgBoxDb

c.       Under each database make sure that the user has the database role “BTS_OPERATORS” selected.

2-      OR Add the group as a sub group to the BizTalk operators group and convert the BizTalk operators group to be a universal group.

3.3.4        Give the Operators Group “gEsbPortalAdmins” ESBPortal and ESBPortalAdmin Roles on the ESBExceptionsDb

This is needed to allow proper operation on the portal functionality. This should be done using the SQL management console.

1-      Open the SQL management console.

2-      Open the database ESBExceptionsDb and expand the security node.

3-      Add a login for the group “gEsbPortalAdmins”. If not already added.

4-      Give that group the database roles ESBPortal and ESBPortalAdmin roles

4         Recycle the ESB Management Portal application pool

1-      Open the IIS management console by running the command INETMGR

2-      Click on the Application pool on the left.

3-      Click on the management portal application pool.

4-      Click on the right panel recycle.

Introduction

Although the requirement I had was to add a custom page and customize the BizTalk BAM portal; this post actually applies to any web application that is using the compiled web site template.

The BAM portal is the business analysis and monitoring portal that comes with BizTalk server. This is actually a portal that is using the compiled web site template.

Problem

The need was to add several custom pages to this portal so that they would be completely integrated with the portal. Meaning that they should be using the same master page and hence include all the main menu and links within this master page.

Now if this is a web application that would be an easy task (right?) since you would have already the mark up of the master page file but since this is a compiled website the site.master is actually a placeholder file and contains nothing L

Solution

Step 1:     So what I did is that I created a new website with any name. In this new website you would get a complete solution with many out of the box files as below:

Step 2:     Now what we need to do is to delete all things we don’t need. So basically you need to delete everything except:

1-      Site.master

2-      Web.config (Although we don’t need it it will not compile without this)

Step 3:     Now we need to mimic the structure of the BAM portal. So we create a new folder called “MasterPages” and move the Site.Master to this folder.

Step 4:     Create a folder called Pages and create any needed custom ASPX pages to this folder. These pages should refer to the master page already moved to the same folder as the original BAM website as described above.

Step 5:     If you want to refer to available classes and helper methods within the original web site then add references to the needed assemblies. For me that was “Microsoft.BizTalk.Bam.Portal.dll”

Step 6:     Right click on the website and click properties. Change the properties to be as in the below image

Note that you must make the site not updatable and using fixed naming.

Step 7:     Make sure that you are targeting the same .NET framework as the original website. For my case that’s .NET 4.0

Step 8:     Right click on the web site and click deploy and select a valid deployment folder (not an IIS website)

Step 9:     Now open the deployment PrecompiledWeb folder and then copy the ASPX custom pages from the pages folder to the pages folder under the original website. Also copy the pages compiled files and DLLs from the bin folder to the original website bin folder.

That’s basically it and now your pages would has the same master page as the BAM portal.

Introduction

The BizTalk ESB toolkit is an implementation of an enterprise service bus messaging standard. It allows for separation between message content, processes implementation, and process configuration.

This is using what is called ESB itineraries. An itinerary would define the lifecycle of the message as it goes through the service bus. The itinerary is composed mainly of itinerary services. There are already itinerary services provided with the ESB out of the box, such as the transformation and routing services.

Usually in many situations you would need to provide more business related logic to be executed using BizTalk orchestrations. This is defined as a custom orchestration service.

Step by step guide

To create a custom orchestration extender you should follow the following steps:

Step 1:     Create a new BizTalk project in visual studio. For example let’s call this “ESB.Samples.Itinerary.Extenders”.

Step 2:     Add references to the ESB assemblies needed which are

All these should be added from the server GAC.

Step 3:     Add a new orchestration to this assembly and call it (for example) ProcessGWMsg.

Step 4:     Add a direct bound receive port to get the message from the message box database

Step 5:     Add a new message with the expected message type you will be processing in this orchestration.

Step 6:     Add a receive shape to this orchestration and choose the message variable you will receive the message in and link that to the direct receive port already created and mark the Activate property of this shape to True.

Step 7:     Open the editor for the filter expression property of this orchestration.

Step 8:     Add a filter expression like the below

The service name should be unique between all your custom orchestrations to make sure there is no multiple-subscription (unless this is needed of course).

Step 9:     Give the assembly a strong name, build it, and deploy the assembly to the GAC using the command
gacutil /I ESB.Samples.Itinerary.Extenders.dll

Step 10:  Using the command
gacutil /l ESB.Samples.Itinerary.Extenders

Step 11:  Mark the strong assembly full name.

Step 12:  Open the ESB.Config file in the path “C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit 2.1”

Step 13:  Search for the section itineraryServices and add a new line at the end of this section as below:

Where the id actually is a new generated GUID and the ProcessMsg is the name of the orchestration you will be creating later.

Step 14:  Close visual studio and start it over.

Step 15:  Create a new itinerary called for example “MsgProcessingIti”. Now add to this itinerary all the on-ramp, off-ramp, and itinerary services you want to process your messages. But what we will be interested in is how to call the custom orchestration.

Step 16:  Add a new itinerary service

Step 17:  Change the extender to be an orchestration extender

Step 18:  Open the service name property and you will find the ProcessGWMsg orchestration. Select that.

Step 19:  Now add all the needed resolvers for this itinerary service as needed.

Step 20:  Validate, save, and publish the itinerary to the itinerary database.

Step 21:  Now revert back to the orchestration and define new variables to handle itinerary operations as below

Step 22:  Define a new correlation type called “ItineraryAdvance” and define it to include the properties

BTS.OutboundTransportLocation,BTS.OutboundTransportType

Microsoft.Practices.ESB.Itinerary.Schemas.IsRequestResponse

Microsoft.Practices.ESB.Itinerary.Schemas.ServiceName

Microsoft.Practices.ESB.Itinerary.Schemas.ServiceState

Microsoft.Practices.ESB.Itinerary.Schemas.ServiceType

Step 23:  Define a new correlation set of the previously created type in the orchestration with the name “ItineraryAdvanceSet”
Now the real purpose of this correlation set is to promote the itinerary properties to allow the message to be routed to the next itinerary service step.

Step 24:  Add a new expression shape to initialize the itinerary variables as below.

Step 25:  You can access the resolvers in the current executing itinerary service by first making sure we have resolvers by adding a decide shape as below.

Where the “ResolverValid” condition is as below.

Step 26:  And to get the resolver dictionary

Step 27:  And to configure a dynamic send port using parameters coming out of the resolver

Step 28:  Finally when you done processing and want to advance the itinerary you create a new message that will pass to the next step and while creating this message add a message assignment shape with the code.

Step 29:  And when you are sending this message you send it using a direct port to the message box and initialize the correlation set already added to the orchestration previously.

Step 30:  Now edit the orchestration to process the messages as you need.

Introduction

You find yourself writing custom pipeline components in nearly all BizTalk related projects. So this is a common activity. And usually it will mean eventually you will need to extract data from the message coming on the wire using XPath statements.

Problem

The issues you usually face while trying to extract data from BizTalk massages is that the message stream would not seekable. And another problem is how to do this data extraction with the minimal memory footprint (no XmlDocument L) and also as fast as possible.

Solution

I created mainly two functions that I use commonly for these tasks. The first one is to get the stream from the message and create a seekable one to be used later on.

The second method is the one that would perform the data extraction as follows.

As you can see I am using XPathDocument with a XmlReader to perform this as fast as possible.

What is the ESB Exceptions notifications service?

The BizTalk ESB toolkit is an implementation of an enterprise service bus messaging standard. It allows for separation between message content, processes implementation, and process configuration.

As part of the ESB samples you find the ESB portal sample with some helper services. One of these services is the ESB exceptions notifications service. This is a windows service that checks every specified timespan for new exceptions happening on your platform and if there is anyone who created a subscription to this type of exception or not. If it finds valid subscriptions to the new exception it simply send emails to the subscribing user notifying them of the exceptions.

Problem

This service is actually very useful and would make your ESB support team more responsive and Performant.

The problem I found is that if someone actually added new subscriptions with a non-valid email address this would throw an exception while sending the email. And because this service is processing emails in batches it will consider the entire batch as not being sent although it might even have already sent messages within this batch already. This would make the service just keep sending the same messages over and over again until the request to send this email is removed from the database.

Solution

The solution is to change the behavior of the exceptions service by handling this condition. In my case all I needed in this case is to mark this email as being sent and just completely discard the error.

1-      Open the Alerts visual studio project from the folder “C:\Projects\ESBSource\Source\Samples\Management Portal\ESB.AlertService”

2-      Open the file “Nofier.cs”

3-      Go to the function called Notify

4-      Change the send lines to be like the below

Please note that I left the marking of the email as being sent as I do not want it to reprocesses the same email again.

Usually when writing custom BizTalk pipeline components you find yourself wanting to extract specific values from the message passed using Xpath statements.

You can do this either by XPathDocument or XDocument, but this solution would require loading the entire XML into memory and if the XML file is huge that can be not possible. Also it makes the pipeline component slower. The solution is to use a streamed class such as XMLReader. But that would be too much work to do, right?

The solution comes in the form of a hidden GEM in the BizTalk installed components, called the XPathReader. This is a stream based class that would search for a node or element using the given set of XPath strings.

This class is defined in the assembly Microsoft.BizTalk.XPathReader.dll deployed to the GAC. You need to add a reference to this assembly first and then use the class as below.

Where the MsgStream is a seekable steam obtained from the message.

Recently I was working on a very complex integration project using BizTalk Server 2010 stack. This project involved integrating with legacy IBM mainframe system. The integration involved working with BizTalk Adapter for Host Applications to call Mainframe hosted applications as part of specific processes. The application mainly involved passing parameters to this COBOL application and getting the results back. Some of the parameters and results returned contained Arabic latters and characters. Arabic is a considered a complex language which requires complex language processor which is called a Transcriber. If you are familiar with the Arabic language it needs to actions to be rendered correctly:

1- The shape of the Arabic letters changes according to their context; meaning that the way we write the latter differs depending upon which letter is before and after the letter we are writing. This phase we call the shaping phase.

2- The second phase is the reversal; Arabic language is written from right to left and hence the characters need to be reversed for it to be rendered correctly. This phase we call the reversal phase.

Usually when you are dealing with Arabic programs and languages you need to execute both these phases for correct display of the sentence. Now HIS 2010 contains a special engine that deals with complex languages such as Arabic this is the TransBIDI.dll and by default this is working as normal and performs both phases as expected.

The problem we faced while integrating with the COBOL program is that it required only the execution of the shaping phase but wanted the letters not to be reversed! Although this (for me) is a strange request and initially I started to think about some sort of customizations and complex scenarios to my surprise I have found that HIS server (starting of HIS 2004) recognized this need and provided a solution to it. Although the KB article I have found talks about the DB2 adapter I found that it applies to all HIS host adapters including the Host Application adapter. Here is an excerpt of the KB article on how to apply this hot fix.

The complete KB article can be found here: http://support.microsoft.com/default.aspx?scid=KB;EN-US;890353

I hope this saves someone else’s day as it saved mine

Introduction

This blog to consider some important information when upgrading to BizTalk 2010

Upgrade roadmap

For BizTalk 2006 R2 and 2009 can be upgraded directly to BizTalk 2010:

• Upgrade from BizTalk 2006 R2 to BizTalk 2010
• Upgrade from BizTalk 2009 to BizTalk 2010

But unfortunately upgrading directly from BizTalk 2006 to 2010 is not supported, so what you need to do in this case is to perform two stages upgrade:

• Upgrade from BizTalk 2006 to 2006 R2
• Then upgrade from BizTalk 2006 R2 to 2010.

Infrastructure Consideration

• When upgrading to BizTalk 2009
• Windows server 2003 and windows 2008 are supported.
• SQL 2005 and SQL 2008 are supported

• When upgrading to BizTalk 2010

  • Widows server 2008 and windows 2008 R2 are only supported
  • SQL 2008 and SQL 2008 R2 are only supported

BizTalk 2010 upgrade considerations:

• You cannot roll back to BizTalk Server 2009/2006 R2.
• Trading Partners engine is completely different in BizTalk 2010 , as a result you need to run (party migration tool) to migrate parties from the old TPM to the new TPM model.
• Human Workflow Services (HWS) is deprecated in BizTalk 2010.
• When upgrading to SQL Server 2008 make sure that you install SQL Server 2005 notification services if you are using BAM alerts. This is required for enabling BAM alerts.
• You must run a migrate command if you upgraded the SQL server (bm.exe migrate-sql)
• SSO DB and Master Secret should be backed up before starting the upgrade process.
• The BizTalk Server 2010 upgrade process requires the existence of the default host.
• User-defined custom bindings that are built with earlier versions of the .NET Framework 4 will not be available after you upgrade to BizTalk Server 2010. To use the custom bindings, you have to manually add the custom bindings in the .NET Framework 4 machine.config file.
• Before you upgrade to BizTalk Server 2010, you must back up all custom managed configuration files in BizTalk Server 2009/2006 R2. BizTalk Server 2010 supports migration of changes only in the btsntsvc.exe.config and bm.exe.config files

References

i recommend to refer to this valuable documentation http://www.microsoft.com/en-us/download/details.aspx?id=11503 

one of the great capabilities provided by the ESB is the ability to create a very scalable yet dynamic receive port , this port can be as generic to accept any XML message without the need to pass the itinerary , in other words we can  create a web method that can accept any XML message as input parameter for the web method which is subscribed to your ESB engine.

This can be done using BizTalk Web Services Publishing wizard BTSWebSvcWiz.exe , this post will walk you through the steps to create a Generic ESB Receive port:

1. Browse to the publishing wizard,  C:\Program Files (x86)\Microsoft BizTalk Server 2010\BTSWebSvcWiz.exe

      2.   In this step we will create a web service using  “publish XML schema as a web Service” option

3.  now we need to define the Request schema type (input schema) for the web method to accept any schema type , to do this configure the receive port to use (Microfsoft.Xlang.XmlBase) by selecting “Select schema type”

now in order to be able to pickup any schema type you need to browse and pick up the BaseTypes.dll from : C:\Program Files (x76)\Microsoft BizTalk Server 2010 and pick the Microsoft.XLANGs.BaseTypes.dll

4.  Publish the port under IIS and place it under the desired web application 

5.  Now go to BizTalk and configure the port to subscribe to a particular Itinerary as displayed :

Note : it can be any resolver like (BRE) for instance.  BRE:\\policy=PolicyName

This is a small trick that I have been doing for sometime now and wanted to share. The scenario is that you have prepared the BizTalk BAM definition and you ready to deploy it to the production environment, you run the bm.exe command and you see the situation; the BAM definition file is an Excel sheet and since you do not have Excel installed on the production server the installation has failed.

The options you have are either to your back to your development environment and export the XML from the Excel sheet or to install Excel on the production environment (not really an option).

The trick is actually to open the Excel sheet in notepad as a normal text file and you would see something like this:

WOW wait a minute this XML seems familiar, right?

Well you are correct this is the XML definition for the BAM view and activity. So simply copy this text and paste it in a new XML document and there you have it the XML definition of your BAM activity

Happy BizTalking

The Problem

When configuring BizTalk Server 2013 RTM on a multi-computer environment you might face multiple issues that are new and did not exist in previous versions of BizTalk server. One of these issues that I faced was when the BizTalk Server and the database server are two separate servers. I was trying to configure only BAM tools and without the configuration of the BAM alerts. Of course you already know that the BAM alerts in BizTalk 2013 is using a new model of Database mail instead of SQL server 2005 notification services (finally) but it seems this caused some problems. In this scenario the configuration failed and when you open the log you see this error.

The Solution

This is a known and reported issue in BizTalk server 2013 and there are two workarounds for this issue:

1-     Install SQL Server 2005 Notification services on the BizTalk server machine. (again L) I personally do not like this option as you are installing something you are not using.

2-     Configure the SQL server 2012 database mail and configure both the BAM tools and BAM alerts at the same time. This is a good option if you are going to use BAM alerts but if you are not going to use it then you are configuring a feature that you will not be using.

The official and recommended solution for this is to install the already available Hot Fix for this issue. This hot fix would be automatically installed on the BizTalk server if it has internet access, but like my case where the servers does not  have internet access I had to manually download and install it. Please find the download for this hot fix here.

http://support.microsoft.com/kb/2838133

The Problem

When configuring BizTalk Server 2013 RTM on a multi-computer environment you might face multiple issues that are new and did not exist in previous versions of BizTalk server. One of these issues that I faced was when you try to configure the BAM portal after you have configured the BAM Alerts service. Of course you already know that the BAM alerts in BizTalk 2013 is using a new model of Database mail instead of SQL server 2005 notification services (finally) but it seems this caused some problems. In this scenario the configuration failed and when you open the log you see this error.

The Solution

When I started looking at this error I started thinking that the BizTalk configuration application is corrupted so I restarted it, with no luck. I then restarted the entire machine with no luck. I then started to look into the log file more carefully and found that there is another error being reported before this one complaining about a problem with one database role called “NSSubscriberAdmin” in the BAM Alert Application database.

I opened this database and I could not find this role defined in it. I started to search the internet for someone who faced this problem with BizTalk 2013 and I found one here. He is suggesting to just add the role “NSSubscriberAdmin” to the BAM Alert Application database. So I did this and it worked.

BUT when I started thinking of what I did, I thought this is wrong since the configuration application did something using this role and this role is not used anywhere in the application so this must hunt me back sometime later. So I opened the SQL management studio and looked at this role and what the configuration application did to it. I found that it actually added the BAM Management Web Service user to this role! So this will be used by this service; what I found also is that there is another role in this database called “BTS_SubscriberAdmin” role (rings any bells ;) ) so I am thinking this is the true role that the configuration application should have used. So what I did is that I added the “BAM Management Web Service User” to this role “BTS_SubscriberAdmin”. I hope that this is enough to make the solution work with no problems. J

Happy BizTalking ;)

I faced a requirement today that I need to enable some BizTalk best practices and create a custom check-in policy for these practices while the developers are checking in code. While searching the internet I found the BizTalk Cop (which is a great tool) that does exactly that, but unfortunately this tool supports only Visual Studio 2010 and BizTalk 2010 and does not support BizTalk 2013 and visual studio 2012. So the challenge was how to enable this to work with Visual Studio 2012. Another challenge that I had was to create a default custom check-in policy and a rules set that is enabled by default on TFS that would only allow the developers to check-in their code if they enable this rule set.

The solution that I performed involved the following changes (updates)

1. I copied the DLL “BizTalkCop.Rules.dll” to the folder “C:\Program Files (x86)\Microsoft Visual Studio 11.0\Team Tools\Static Analysis Tools\FxCop\Rules”
2. I created a file called “BizTalkRules.ruleset” (attached at the end of this article) containing the set of rules that are enabled by default for the projects.
3. I copied the file “BizTalkRules.ruleset” to the folder “C:\Program Files (x86)\Microsoft Visual Studio 11.0\Team Tools\Static Analysis Tools\Rule Sets”
4. Changed the file “devenv.exe.config” in the folder “C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE” so that it redirects all FxCop dependencies from the 10.0 and 9.0 versions to the 11.0 version in the assembly binding section as follows:

<dependentAssembly>
  < assemblyIdentity name="FxCopCommon" publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
  < bindingRedirect oldVersion="9.0.0.0" newVersion="11.0.0.0"/>
< /dependentAssembly>
< dependentAssembly>
  < assemblyIdentity name="FxCopSdk" publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
  < bindingRedirect oldVersion="9.0.0.0" newVersion="11.0.0.0"/>
< /dependentAssembly>
< dependentAssembly>
  < assemblyIdentity name="Microsoft.Cci" publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
  < bindingRedirect oldVersion="9.0.0.0" newVersion="11.0.0.0"/>
< /dependentAssembly>
< dependentAssembly>
  < assemblyIdentity name="Microsoft.VisualStudio.CodeAnalysis" publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
  < bindingRedirect oldVersion="9.0.0.0" newVersion="11.0.0.0"/>
< /dependentAssembly>
< dependentAssembly>
  < assemblyIdentity name="Microsoft.VisualStudio.CodeAnalysis.Common" publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
  < bindingRedirect oldVersion="9.0.0.0" newVersion="11.0.0.0"/>
< /dependentAssembly>
< dependentAssembly>
  < assemblyIdentity name="Microsoft.VisualStudio.CodeAnalysis.Interop" publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
  < bindingRedirect oldVersion="9.0.0.0" newVersion="11.0.0.0"/>
< /dependentAssembly>

1. Changed the file “FxCopCmd.exe.config” in the folder “C:\Program Files (x86)\Microsoft Visual Studio 11.0\Team Tools\Static Analysis Tools\FxCop” so that it redirects all FxCop dependencies from the 10.0 and 9.0 versions to the 11.0 version in the assembly binding section as follows:

<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
  < dependentAssembly>
    < assemblyIdentity name="FxCopCommon" publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
    < bindingRedirect oldVersion="9.0.0.0" newVersion="11.0.0.0"/>
  </dependentAssembly>
  < dependentAssembly>
    < assemblyIdentity name="FxCopSdk" publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
    < bindingRedirect oldVersion="9.0.0.0" newVersion="11.0.0.0"/>
  </dependentAssembly>
  < dependentAssembly>
    < assemblyIdentity name="Microsoft.Cci" publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
    < bindingRedirect oldVersion="9.0.0.0" newVersion="11.0.0.0"/>
  </dependentAssembly>
  < dependentAssembly>
    < assemblyIdentity name="Microsoft.VisualStudio.CodeAnalysis" publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
    < bindingRedirect oldVersion="9.0.0.0" newVersion="11.0.0.0"/>
  </dependentAssembly>
  < dependentAssembly>
    < assemblyIdentity name="Microsoft.VisualStudio.CodeAnalysis.Common" publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
    < bindingRedirect oldVersion="9.0.0.0" newVersion="11.0.0.0"/>
  </dependentAssembly>
  < dependentAssembly>
    < assemblyIdentity name="Microsoft.VisualStudio.CodeAnalysis.Interop" publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
    < bindingRedirect oldVersion="9.0.0.0" newVersion="11.0.0.0"/>
  </dependentAssembly>
< /assemblyBinding>

1. Restart VS 2012
2. I enabled a default check-in policy using the source control settings as per the below screen shoots
   
   
   
   
3. Make sure you perform the same steps on the TFS Build server and all developer machines.

Happy coding :)

Introduction

BizTalk mapping tool is a very powerful tool once you get the hang of it. Sometimes you would need to produce some structure in the output schema that did not exist in the source schema, the Table Looping functoid is built for this specific task.

In some other cases you would need to generate a parent child kind of structure. In this blog post I will present the solution to use out of the box functoids to be able to generate this structure.

Problem

Suppose you have the following as the input schema.

And the following output schema.

The requirement is to generate two Message elements in the ProcessRoot output schema. It is also required to generate some fields in the fields array inside each message. These fields are specific for each message and depend on the message ID field. So to be more specific let’s say that we have the following XML input:

And we want to generate the following XML output from it:

As you can see we have two challenges:

1-     You need to create not just an array of two messages for one node in the input message, but also some sub-fields within each message.

2-     The fields generated are dynamic and dependent on the value of one of the fields of parent array.

Solution

Because I want to generate some structure that does not exist in the original message then we have to use the Table Looping and Table Extractor functoids. Now the challenges listed above is the key here on how to develop this. So I had many ways I tried to implement this but mainly they all involved creating two Table looping functoids one for the parent structure and one for the child array. The challenge I had was how to link them and make the second table looping functoid dependent on the first one. Here is the map I ended up implementing:

There are some key points here to consider.

First the first table looping functoid generates the parent array of messages and hence it is linked to the output node of the messages as per this figure:

Also the table of this functoid has the following structure to be generated:

The second notice here is that we have a link from the first column of the parent table looping functoid and the first column also of the child table looping functoid linked to the output fields array to make the generation of the fields conditional if it is equal to the message ID of its parent. This means that second table looping functoid has to include the message ID as part of the table also it will not be generated in the output.

Now ideally this would mean that the table of the child table looping functoid should be as follows:

But when I tried this it never worked (and until now I do not know why) when I debugged the generated XSLT I noticed that the first two rows generated from this functoid are always wrong. When performed further testing I noticed that the child functoid always has bad rows generated out of it that are exactly equal to the number of rows generated from the first table looping functoid. So what I ended up doing is that I added two extra dummy rows that I made sure will never generate anything in the output schema by making sure that the value that would be compared is always not equal. So what I ended up doing is configuring the second table looping functoid as follows.

This achieved exactly what I wanted and produced the required output XML.

The sample input schema, output schema, and the above described map can be found here.

Happy BizTalking J

Introduction

In this blog post I will present a very simple solution to integrate with Apache ActiveMQ platform. Actually I got this question from a friend I could not believe how easy it is to integrate with this platform using the new BizTalk 2013 WCF-WebHTTP adapter. The adapter provides many features out of the box that really makes the integration engineer life much easier and better.

In this scenario I have installed Apache ActiveMQ on a private VM, created a test queue and made sure that I am able actually to send messages to that test queue. The target then was to create a way to receive messages from this queue to BizTalk server.

Discussion

The following procedure is what I did to perform this POC.

1-     First I started by installing ActiveMQ and to my surprise that was super easy. So I downloaded the binaries from here and I also downloaded the JDK from here. I installed the JDK then extracted the ActiveMQ to any folder of my choice.

2-     I then had to set the JAVA_HOME environment variable (I do not know why the JDK setup did not do this) but I did that manually to point to the JDK folder as C:\Program Files\Java\jdk1.7.0_45

3-     I then started the ActiveMQ platform by browsing to the folder where I extracted it and then typing the command:

4-     Then I opened the new administration portal using the URL (http://localhost:8161/hawtio) then I logged in using the username admin and password admin.

5-     Clicked on “ActiveMQ”

6-     Then I created a new Queue called “Testing” by clicking “Create” then “Create Queue”

7-     Click on the queue on the left then click “Send” then make sure you change the format to XML type in a valid XML message and finally send the message.

8-     If you now go to the browse of the queue you will find that the message is ready for pickup as per below:

9-     Now the ActiveMQ supports the REST API to be able to pick up messages (or post them) the address is as follows (http://localhost:8161/api/message/Testing?type=queue&clientId=BTSconsumer). So note this address as we will use it later in BizTalk.

10-  Now let’s go to the BizTalk administration console. I created a new send port to be able to pick up messages from the Queue using the WCF-WebHTTP adapter. So I created a new send port and configured it as follows:

11-  Then click configure and change the address and operation as follows:

12-  One key point here (and a new feature that I really like about this new adapter) is that if you are using the HTTP GET operation then you cannot sent anything in the BODY of the HTTP request. But BizTalk out of the box have to send a message so the adapter solved this by adding a key feature called suppress outbound messages body for verbs GET, so you need to configure this as below:

13-  Now I need a way to issue a GET verb using this send port. So usually what you would configure is a scheduled task adapter to schedule checking of messages from the queue. In my case I wanted something simple and without installing anything else on the VM. So I configure a messaging approach to issue a GET verb after I send an empty file through a file receive location and then created another file send port to pick up the final message received form the Queue and write it to a folder.

14-  Started everything and BINGO worked like a charm J I have sent the message to the queue and then I drop the empty file to the input folder to issue a GET verb and finally the sent message is written to the output folder as follows:

Conclusion

The conclusion here is that REST is very powerful and makes you do amazing things. I really think everything should support REST and the other very important conclusion is that I need to thank the BizTalk product group to support this out of the box in BizTalk Server 2013 J

Introduction

Solution

·        ConfigurationInfo

·        ReceiveLocationProxy

Happy BizTalking.