Seperating Items from Order without Orchestration

Recently, a co worker approached me with a problem. He had the following xml struture:

1
ABC

01123
35
300.50


45221
27
1430.50

45222
127
11430.50



He wanted the following structure as output:

1 ABC

01123 35 300.50

1 ABC

45221 27 1430.50




There is a solution where you have to promote property and use XPath in orchestration to do this. I saw this on Stepehen's website. I found another way to achieve same. This new way is even faster.


1) Generate the schema from the original file call it "IN"



-
-
-
-
-
-
-





-
-
-










2) Generate another schema call it OUT


-
-
-
-
-
-
-













3) Use Map1 to populate fields from IN to OUT.

4) The resutlant file will be:

-
-
01123
35
300.50
1
ABC

-
45221
27
1430.50
1
ABC

-
45222
127
11430.50
1
ABC



5) Now you can drop this file to a folder. This file format is easy to split now. data

6) Split it using XML disassembler. Envelope schema will be type

7) Create another map that takes input structure as the "split" structure. i,e

01123
35
300.50
1
ABC


8) The output of that map will be of type: (Original type in step 1)


-
-
-
-
-
-
-





-
-
-










9) Now this map will give yu the desired output after completing the mapping.

10) You can put this "second" map in either a "Port" or "Orchestration"

Biztalk and Integration Patterns

I found this very nice white paper on Integration Patterns. It gives a very macro level view of Solution Architecture and then breaks it down to specific Biztalk level. I recommend this paper to everyone who is involved in system architecture and implementation.

This paper can be found at:

www.eaipatterns.com/docs/integrationpatterns_biztalk.pdf

Efficient Mapping Technique

If you have sample input and output files with same header level node names and structures(some of the field names can be different) . Then instead of doing individual drag and drop (can't use mass copy as these files are not 100% similair). You can use "Link by Node Name". This is how you do that:

1) Click on the MAP grid.

2) Go to properties box

3) Select "Auto Link By" : NODE NAME

Faster way of Deploying assemblies to the servers.

One of techniques of deplying assemblies directly to server without building it on your local machine is:

1) Right Click the Project in Solution Explorer.

2) Go to Properties----Configuration Properties-----Deployment.

3) Select the server name, database and make "Redploy"= true.

4) Click Deploy

The Redeploy="True" enables to deploy over the older assemblies on the server. This way we do not have to undeploy and then deploy.


Warning: Make sure to revert back the settings from your server to your local machine after completing the above steps. Else you might end up overwriting the server solution with your local machine solution (good or bad).

Synching the Method executions from Orchestrations

One of the problem I faced in one of my project was, in the event of multiple files entering a same method. And if this method is reading information from database. This can result in that method being executed by multiple xml documents and hence exceeding the database connection pool. In order to avoid this, DOT NET offers "SyncLock" object.

Wrap all your methods in this object. This will ensure that second document does not enter this method till the previous one is completely processed.

Example:


SyncLock object name
Call the function here
End SyncLock

De-Batching and Re-Batching the Files (Scatter-Gather)

In my current project we had requirement to split the incoming XML files into individual transactions. These individual transactions would then go through some rule validations in Orchestration. Depending on the validation rules set up in the external assemblies, these transactions will be either rejected or accepted. The business need for splitting the file came from the fact that the transaction has to be accepted or rejected as a whole.

Splitting part was easy. We used envelope and document schemas for that. But the difficult part was to batch the split transactions per incoming file they originally belonged too. I will give possible solutions for "Re-Batching" here and also limitations for each.

1) Use a loop within the Orchestration. Add each transactions on basis of node. Co-relation set will have to be set up . This is important as the correlation ID will help us to add only the transactions that belong to a particular file. This approach is useful if we have something like a "Orders" file that consists of various "Items". Order ID will be the correlation ID in this case. This will tie up all the "Items" for a particular "Order ID".

This approach fails in my situation as I have no correlation ID. My sample file consists of a root element immediately followed by child nodes. These child nodes contain all the information. The sample file looks something like:

(a)(b)contains all the data(/b)(/a)

Ideal structure to use the looping would be:

(a)correlation ID here(b)contains all the data(/b)(/a)

2) Loop it and add on basis of the count.

Problem with this approach is that in scenario of 10 input files being dropped at the same time and each having 10 each transactions. The count will not ensure that the transactions we are adding are actually coming from a single file (as we need in our case). It is not guaranteed that the Transactions in each file will be processed in a sequence.

3) Third approach works for me. If I set up a unique ID at each transaction level. This ID is same for all transactions in one particular file. Then I can equate this ID as "File Name" in the Send Port and then use "Append". Biztalk will append all the transactions with the same unique file name. This is what I need!

Work around "Node" name changes in Schemas

Once the mapping is done and the assembly is deployed. Any change in the root node name of the schema means re-mapping the whole thing again. There is a quick work around to this.

Create a function using VB.net/C#. Pass the xml document (with the old root name) to this function. Suppose the incoming Doc is "DocIn". This function will take DocIn as a parameter. Replace the "oldname" with "newname" using replace command on "Outerxml". Pass the resulting document to the final destination. This is how the function will look like.

Public Function Convert (Byval DocIn as XMLDocument)

Dim DocOut as new XMLDocument

DocOut.Load(DocIn.Outerxml.replace("OldName","RootName"))

Return DocOut

End Function

Stored Procedures for Oracle through Dot Net

One of the requirements in the current Project is to call queries from Oracle and perform validations. We are calling about 100 of the queries using "script funtoids" in the mapper. I wanted to optimize the database connections. Instead of sending 100 queries and hitting the database 100 times. I wanted to have single stored procedure to replace the 100 queries. Having worked with stored procedures for SQL database before, I did not find it an issue. My plan was to create an array that is input to the Stored Procedure and get an array as an output to the stored procedure. The idea was to send all possible variables in an array to the stored procedure. Process them and get possible values as an array. Use this array to populate the "Right Hand" side of the map.

On course of my effort, I realized that there are limitations as such of working with Oracle through arrays. I gave up after trying!

Pipeline tools

For those that don't know, there are some really handy pipeline tools in the SDK under the folder:

\Program Files\Microsoft BizTalk Server 2004\SDK\Utilities\PipelineTools

These tools allow you to test your pipeline components and, even more handily, flat file schemas without having to set up the BizTalk Server environment. They are pretty much essential when developing flat file schemas (particularly when you have a header, trailer and body schema).
Here is a list of the pipeline tools available:

DSDump.exe - this tool outputs a textual description of the schema.

Pipeline.exe - this tool executes a pipeline and produces one or more output files. Very useful for testing custom pipelines.

XMLDasm.exe - this tool executes the XML Disassembler component by simulating a receive pipeline.

XMLAsm.exe - this tool executes the XML Assembler component by simulating a send pipeline.

FFDasm.exe - this tool executes the Flat File Disassembler component by simulating a receive pipeline. Essential for flat file development!

FFAsm.exe - this tool executes the Flat File Assembler component by simulating a send pipeline. Again, essential for flat file development!

Note: The pipeline.exe tool does not access the BizTalk Server databases, so executing a pipeline that contains the BizTalk Framework Assembler/Disassembler may not work as it accesses the BizTalk Management database during runtime to match up receipts to documents.

Remote Connection to HAT

I read some articles on Remotely connecting to HAT. Not very straightword. My requirements were to just get a view of transaction process without using Remote Desktop to the server. Due to limited server connections it was always a problem to connect to it. This is how you connect to server:


1) Open the HAT.

2) Go to Tools----Preferences-----Select "Archived Data"....Enter the SQL server name and select the Database.

You are all set!

Converting Eurpean/American Dates to ISO-format

We are getting European/American dates in the inbound files. We need to convert them to ISO format dates before sending it over to database. This is how to do it:

1) Drag a script functoid over to the map grid.

2) One input to that "functoid" will be the date field from the input schema and the second input will be a constact value of either "1" ( for European format) and "2" (American format).

2) Create a Public Function in external assembly with two paramaters.

Public Function dateformat( byval strDate as string, Byval flag as integer)
Dim newdate As Date
Dim strReturnDate As String

Select Case flag
Case 1
newdate = Convert.ToDateTime(strDate , New CultureInfo("en-gb"))
strReturnDate = Format(newdate , "yyyy-MM-dd")

Case 2
newdate = Convert.ToDateTime(strDate , New CultureInfo("en-us"))
strReturnDate = Format(newdate , "yyyy-MM-dd")
End Select

Return strReturnDate

End Function