EDI Reporting and curious case of missing 997

We all agree that out of box EDI reporting in Biztalk leaves lot of gaps. For example, it is not easy to find all messages which were sent to our partners that didn't send 997 back to us. One can go on Biztalk console and click each message to find if Ack was received. But this is a very tedious approach and time consuming too. Instead, creating a query using BAM views can quickly give us desired results. This way a report can be set up using SSRS that can be viewed across the organization.

SELECT TOP (100) PERCENT InterchangeControlNo, ReceiverID, ReceiverQ, ApplicationReceiver, DATEADD(HOUR, - 5, TimeCreated) AS ProcessTime, TransactionSetId
FROM dbo.bam_TransactionSetActivity_AllInstances AS T
WHERE (SenderID LIKE ' [your org ID] %') AND (TransactionSetId <> '997') AND (NOT EXISTS
(SELECT RecordID, ActivityID, InterchangeActivityID, GroupControlNo, InterchangeControlNo, ReceiverID, SenderID, ReceiverQ, SenderQ, InterchangeDateTime,
Direction, AckProcessingStatus, AckStatusCode, DeliveredTSCount, AcceptedTSCount, AckIcn, AckIcnDate, AckIcnTime, ErrorCode1, ErrorCode2,
ErrorCode3, ErrorCode4, ErrorCode5, TimeCreated, RowFlags, LastModified
FROM dbo.bam_FunctionalAckActivity_AllInstances AS F
WHERE (GroupControlNo = T.GroupControlNo) AND (SenderID = T.ReceiverID)))
ORDER BY TimeCreated DESC

Remote Desktop issue with VM role

While trying to connect to Biztalk VM image in Azure, if the remote desktop does not connect, try one or all of the following things:

1. Reduce the size of VM image to small (smaller size and lesses processors)
2. Ensure endpoint exists or create one.
3. Delete the VM image and create a new one with a different geographical location. Try different locations within USA. (this did the trick for me)

SSIS Package Error

Biztalk is not always the right tool for Bulk data inserts. SQL Integration Services provide powerful tool set for inserting large bytes of data from source ( flat files / excel files/DB) into DataTables. In my scenario, I created SSIS package using SQL 2008R2 Business Intelligence Development Studio. It was a simple Data flow, read the excel file from a network location and insert to a remote DataTable. Image below:


On executing the package, OLE Destination 1 shape would threw the following error:
“[OLE DB Destination 1 [421]] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 10.0" Hresult: 0x80004005 Description: "The statement has been terminated.".
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 10.0" Hresult: 0x80004005 Description: "Violation of PRIMARY KEY constraint 'PK_tbl_so'. Cannot insert duplicate key in object 'dbo.tbl_so'.".”

I tried various online forums but the solutions didn’t work for me. Then I tried data encryption and it worked. Check the image below.







Update

After I deployed the package to remote server, I tried executing it and received validation errror around login/username. I am using SQL authentication for the DB that will receive the data from excel sheet. Internally, SSIS encrypts the credentials information. If you open the package in a notepad, you can view the encrypted information. But, when the same package is run on a different machine the user context changes and decryption process throws the error. Setting the protection level as below can get you around that issue.

Microsoft BUILD

Starting this year, Microsoft rebranded its Professional Developer Conference (PDC) as BUILD. I have no idea what message they are trying to convey with this name change. For some strange reasons, this change was not broadcasted strongly by the numerous Microsoft blogs and sites. The registration page was so hidden that even search engines could not find it using standard words like “PDC Microsoft conference” etc. Well, for people who still don’t know, BUILD is being held in Anaheim, California in middle of September this year. At the time of writing this article, no agenda have been published by Microsoft for the conference. This is rather strange for an elite conference like this. My guess is that there will be breaking news around direction of Biztalk Cloud technologies. Update to Windows mobile platform (“Mango” code name) will get good coverage. Microsoft is really pushing hard to break into the Smartphone market. Even though Mango is looking like a robust platform, it is hard to penetrate the barrier created by Android and iPhone. Recent buy out of MMI by Google, bodes well for Microsoft. It leaves windows mobile as the only true Open Source mobile platform in the market. It is rather ironic that a company traditionally linked with developing proprietary technologies is taking this mantle from a company like Google that is traditionally linked with developing and encouraging Open Source systems. This demonstrates the fast changing technology landscape and underlines the difficulties faced by CIOs in planning strategic IT initiatives.

Microsoft is consolidating its offering around Desktop, Web, Cloud and Mobile with the latest release of Visual Studio 2011. These are very exciting times for Microsoft Developers, it seems that finally Microsoft is streamlining its technologies and removing the redundancies. Biztalk-Appfabric consolidation is happening along with Silverlight-HTML5 consolidation. Sharepoint story is unchanged and it is one of the few non-redundant technologies in the Microsoft stack.

So, where do we stand amidst all this? Consolidation of technologies will lead to more clarity for customers. One reason, Biztalk didn’t maintain the steep growth curve was because of the confusion created by conflicting technologies on the Microsoft Connected System stack. Customers like clarity, especially while evaluating different technologies to implement. They tend to select products with more clarity around their future roadmap. Microsoft shops in general are prime candidate for Biztalk, but they saw technologies like AppFabric and WCF close substitute for Biztalk. For most of them, sole reason for buying Biztalk was because they were an EDI shop. On the other hand, Biztalk was a tough sell in non-microsoft shops as it works with Microsoft framework. A java shop will most likely go for a GIS or webMethods implementation.

Real value of Biztalk lies in developing it as an integration hub for both EDI and Non-EDI integrations and then extending it to link with Sharepoint. Next step will be to combine on-premise and cloud integration on same platform. This is where Azure Appfabric is leading us to. Keep an eye on PDC (BUILD) space for more on this.

Death of Biztalk -- Are you kidding me?

Honestly, I am tired of statements like “BizTalk is Dead” or “BizTalk is Obsolete”. With every release of new technology around Connected System stack, the noise around speculated BizTalk demise gets louder. Right from the time Oslo was released (which never saw light of the day) to recently released Azure App Fabric –June CTP, blogosphere gets noisy with shouts of BizTalk is Dead.

In my opinion, if one thinks of BizTalk as a brand then it could be said that BizTalk may die as a brand in near future. Even that is a speculation for sake of argument. But if one considers BizTalk as Integration enabler and a set of tools, then it can’t be dead. There may be re-organization around the tools but they will not die down. I agree that redundancies in form of Windows App Fabric and Windows Azure App Fabric do take something away from BizTalk. They also create confusion in the mind of new and existing clients. Especially, when the marketing pitch is not in tune with technical pitch. Few years ago, I would just blindly recommend BizTalk in scenarios involving hosted workflows and service compositions. But now it is not so easy. Microsoft has labeled BizTalk as Integration Server, Windows App Fabric is termed as Application Server and Windows Azure App Fabric is a cloud integration platform. It can get confusing between these 3 technologies. If I need to mathematically establish a relationship between these 3, it will look something like:

BizTalk >= Windows Server Fabric + Windows Azure App Fabric.

It can be argued that in near future Windows Azure App Fabric may expand to include more of the existing BizTalk tool set and that may make the relationship more like

BizTalk = Windows Server Fabric + Windows Azure App Fabric.

This is where people jump in and say that BizTalk is dead as it can be replaced by combination of the two technologies! In my opinion:

1. Azure App Fabric is not a mature product yet and it is too early to say anything. Remember Oslo?

2. It is quite possible that BizTalk tools will be re-organized around cloud platform using Azure App fabric as core. Depending on how one sees it, it could be branded as a new version of BizTalk with Azure App Fabric as the cloud enabler. This will not mean end of BizTalk.

3. Other possibility is that BizTalk and Azure App Fabric will continue their parallel paths for foreseeable future. This will not mean end of BizTalk.

It could be said that Azure App Fabric is a new technology that BizTalk developers will have to learn. Just like BizTalk 2004 release, made BizTalk developers learn .net. BizTalk 2006R2 made developers look into WCF.

Windows Server App Fabric is more of an Application Server than an Integration server. However, BizTalk Appfabric connect allows .net developers leverage LOB adapters and Mapper to create workflow solutions. This compliments BizTalk just like Azure App fabric does.

As a BizTalk developer, I have mixed feelings. On one side I feel unhappy that the BizTalk tool set is being broken and leveraged around different Microsoft stacks. Now everyone else will be able to enjoy the excellent tool sets of BizTalk that were once sole proprietorship of BizTalk developers. It kind of takes power away from us. But at the same time, I feel happy that as a BizTalk developer we are placed uniquely to leave our footprints across the different technology stacks that use BizTalk tools .

Following are list of tools that are still exclusive to BizTalk. These are key touch points while making a choice between BizTalk and other technologies.

• Accelerators (EDI, SWIFT, Rosettanet, RFID, EDIFACT, HL7)
• Pipelines
• Flat File processing
• BAM
• BRE (it is exposed as a REST API now but still under BizTalk licensing)
• BizTalk Mapper (BizTalk AppFabric connect enables you to use the mapper with Server App Fabric, still under BizTalk licensing)
• Non-WCF adapters like FileAct/Interact
• Host Integration Server

Processing Empty Files in Biztalk

In my current integration project, there is a need to process empty files (0 byte) in Biztalk. Default behavior of FILE adapter is to delete an empty file and raise an event. This empty file is then deleted even before it reaches pipeline on way to Biztalk MsgBox. So there is no way to do a routing based on filename either in a pure message scenario or using an orchestration.

One possible solution is to extend the File Adapter that is part of SDK (Image 1.1). Add the 3 projects in a Visual Studio Solution as shown in Image 1.2

Image1.1

Image 1.2

Key steps:

1. Build the solution, GAC Microsoft.Samples.BizTalk.Adapter.Common.dll.

2. Registry Key.

Use the StaticAdapterManagement.reg registry key file given in the sample and update the locations of the dlls mentioned. Microsoft recommends adding additional string for 64 bit machines. I didn’t do and it worked.

3. Go to Biztalk admin console and try to add the new EMPTY FILE adapter. It is possible that you may get an error similar too :
The system cannot find the file specified.

/C:\Program Files (x86)\Microsoft BizTalk Server 2009\SDK\Samples\CustomAdaptersDevelopment\File Adapter\Runtime\Microsoft.BizTalk.SDKSamples.Adapters.DotNetFile.Runtime.dll

This error threw me off as my registry key file did not contain this path at all and it left me wondering why is Biztalk even looking for this path. I did a search for this file in the entire registry and I found couple of references. I had no idea what were they used for but I knew that Biztalk was somehow reading them from registry key. I renamed the key for these entries without deleting . I was able to add the adapter to Biztalk successfully.

4. Once adapter was added, I assigned it to appropriate Biztalk host so I can use it in send/receive port. While setting the receive handler for this newly added adapter, I got the following error:
The Messaging Engine encountered an error when intializing the receive adapter "EmptyFile", HRESULT:"Property /Config/pollingInterval not found on adapter configuration XML.".

I decided to debug the code to find the point of error. I found the error in method : ExtractPollingInterval in file ConfigProperties.cs under project Microsoft.Samples.BizTalk.Adapter.Common.csproj

Update the following code


long pollingInterval = ExtractInt(document, "/Config/pollingInterval");
string pollingUnitOfMeasureStr = Extract(document,"/Config/pollingUnitOfMeasure", "Seconds");

To

long pollingInterval = Convert.ToInt32(IfExistsExtract(document, "/Config/pollingInterval","5"));
string pollingUnitOfMeasureStr = IfExistsExtract(document, "/Config/pollingUnitOfMeasure", "Seconds");

Or update the root name of ReceiveHandler schema to Config. I did the code change and not the schema change


I was in a position to configure this new custom FILE adapter and receive an empty file. In order to get a handle on the empty file, I need to go to method PickUpFilesAndSubmit which is part of DotNetFileReceiverEndpoint.cs under DotNetFile.csproj The line of code that got me the handle to the incoming file is if (item.Length == 0).
Once I got the handle, I was able to add my logic to deal with the incoming empty file. In my case, I decide to add a string “EmptyFile” if the incoming file was empty. I would then detect this file in the send adapter using the same custom adapter and assign 0 bytes to it. In between, I will have my orchestration reading the file name of this empty file (added string “EmptyFile”) and route it to appropriate directory folder.

This solution requires tampering with contents of an emptry file. It may be to acceptable in some cases. In my case, it was acceptable.

Send Side Changes:


To intercept the EmptyFile (file with “EmptyFile” string) on the send side, I used the method: ProcessMessage in file DotNetFileTransmitterEndpoint.cs under project DotNetFile.csproj.
I inserted the following snippet after cloning the stream. This writes an empty file.
StreamReader sr = new StreamReader(Stream);
string fileContents = sr.ReadLine();
if (fileContents.Contains("EmptyFile"))
{
fileStream.Write(new byte[0],0,0);
return null;
}

Dynamic Send Port:

In order to get this adapter to work with Dynamic Send port following changes will be needed:

1. Use DotNetFILE:// (same as the entry in registry) instead of FILE:// in orchestration for setting dynamic address

2. In the adapter use the following code to extract destination string from the context properties:

filePath = message.Context.Read("OutboundTransportLocation", "http://schemas.microsoft.com/BizTalk/2003/system-properties").ToString();

Use this destination string to parse the final destination of the file and add code to write the file to that location.

Configuring BAM portal on Biztalk 2010

So, I installed Biztalk 2010 on Win2008 server with SQL server 2008 R2 on a 32 bit machine. Biztalk configuration worked smoothly as one would expect it to until the wizard tried to enable BAM portal. Biztalk configuration wizard threw an error saying "Default Web Site can't be validated". I did some research and found that almost everyone that had this error got it fixed by installing "IIS6 compatibility" role on the Server. But in my case, this role was already installed. But just to be sure, I uninstalled it--> restarted the server and installed it back. Only to find that it did not make any difference!

I tried hacking my way by physically creating BAM application on the IIS but did not work either. Since BAM is a pre-compiled version, I could not debug it as I would do in an asp.net web application. Then, I thought of creating a new website called "BAM" at the same level as the "Default Website" but on a different port (not 80). In my case, I used port 81. I ran the Biztalk Configuration wizard, pointed it to "BAM" site instead of "Default WebSite" and it worked!!

Later, I did some research and found out that this is a "bug" or a "known behavior" with Biztalk 2010 installation. It seems creating a new site for BAM is generally accepted solution.

Check out :http://social.msdn.microsoft.com/Forums/en/biztalkgeneral/thread/ef650460-c219-4b34-bc7a-41bb7e75ec81

I would like to hear from others who faced/or not faced same issue.

Biztalk and CEP (Complex Event Processing)

For people interested in integrating CEP with Biztalk, check out the white paper by Jesus Rodriguez at :

http://www.microsoft.com/downloads/en/details.aspx?FamilyID=6c1da607-4785-4668-a274-6898c38ea603

StreamInsight is part of SQL 2008R2. Do some reading on StreamInsight before checking out this paper.

This paper is very conceptual and does not provide any code snippets. If you are looking for some code samples, then use the samples that come with StreamInsight download and extend them.

Appfabric Connect Feature on Biztalk 2010

When Dublin (now Windows Server Appfaric) was close to being released, there was lot of talk around the speculative demise of Biztalk. Some of the key differentiators identified at that time were Mapper tool and ability to resume message flow using persistence data store. At that time, I wrote that it will be matter of time that these differentiators between Biztalk and Dublin will fade away (check the comment section of http://geekswithblogs.net/cyoung/archive/2008/10/15/125848.aspx). Current Avatar of Dublin aka WS Appfabric , provides the ability to persist data and supports resumption of messages. “AppFabric Connect” feature, that is part of Biztalk 2010 installation, exposes Biztalk mapper and LOB directly. So to be used with workflows, that can then be hosted on Windows Server Appfabric which in turn is part of IIS. Light weight applications does not need heavy weight Biztalk database anymore. Low latency scenarios can now easily be constructed using IIS hosting using Dublin.

Since this feature is part of Biztalk 2010 installation, Biztalk license is needed to use this feature. In future this can and will change. It will then be possible to create custom WF that can be persisted to a DB and exposed as WCF endpoints, while mapping .net types using Biztalk with same set of mapping functoids that are part of Biztalk world. And all of this will be free!

Little trick....that will help you long way

During the process of creating cloud project using VStudio, don't forget to mark external "reference" assemblies as "Copy Local =True". If this is not done, your application will run fine on your local cloud emulator but throw an "asp.net application error" when executed on the cloud. Every dll that is referenced, other than the default dlls, that are part of cloud project, should have property "copy local = true". When Visual Studio creates deployment package for cloud, it looks for this property. It adds all the dlls with "copy local = true" in the package and when the solution is deployed on the cloud, these additional dlls are deployed on the cloud too. In the absence of these dlls, your cloud application will throw “asp.net application error”.

Biztalk and the Cloud

I was in NY city on Dec 1 for the Azure Firestarter event at the Microsoft Office. It was a nice overview of Azure. For past year or so, I have been thinking of getting my hands dirty with this technology. Between work and family, never really got a chance to do so. But this time, after the event I promised myself to actually put what I learnt into practice. Moreover so, because the free registration to Azure portal would expire in few weeks time.

So, I came up with a simple scenario.

1. Expose a CreditScore service using Biztalk within a firewall.
2. Publish this Biztalk Orchestration as WCF to Cloud Endpoint using WCF publishing
wizard
3. Use a simple windows client application to send a message and receive a reply from this Biztalk service exposed within firewall.

Step 1:

Create a two way Biztalk Orchestration that accepts an xml message(First Name, Last Name, SSN) and returns a message (Credit Score and Comments).



(Image 1)

Step 2.

Publish this Orchestration as the cloud endpoint.



Pre-Requisite:

1. Biztalk 2010
2. Windows AppFabric SDK 1.0
3. Windows Azure AppFabric Account to create the URL of cloud Endpoint (Free, if you have MSDN subscription)
4. Biztalk 2010 adapter pack

For details checkout : support.microsoft.com/kb/2449478

Step 2. Create a windows client to test this Biztalk service using endpoint exposed in the cloud.



Pre-Requisite for the client machine:

1. Visual Studio(2008 sp1 or 2010)
2. Azure AppFabric SDK 1.0 (to get NetTcpRelayBinding and TransportClientEndpointBehavior).

a) Open a "Windows Form Application" Project

b) Add Reference to "Microsoft.ServiceBus.dll" from location

"C:\Program Files\Windows Azure AppFabric SDK\V1.0\Assemblies\NET3.5\Microsoft.ServiceBus.dll"

c)Add Service Reference point to the URL exposed in the cloud : Sample URL:

shashiraina-demo.servicebus.windows.net/Test.Biztalk.Cloud.Orch/. This namespace in URL is created using AppFabric Account. (Image below)

(Image 2)


d) Use the proxy to generate client side objects. I used code instead of config file because, I was not able to get "TransportClientEndpointBehavior" and "NetTcpRelayBinding" to show up in Config file. This may be because I was just using a Windows Application as test client instead of "cloud" supported client. With code, I could easily use the ServiceBus namespace and get it working.

Sample client code is below:

GetCreditScore.GetCreditScoreRequest req = new GetCreditScoreRequest();
GetCreditScore.GetCreditScoreResponse res = new GetCreditScoreResponse();
NetTcpRelayBinding tcpRelayBinding = new NetTcpRelayBinding();
TransportClientEndpointBehavior clientEndBeh = new TransportClientEndpointBehavior();
clientEndBeh.CredentialType = TransportClientCredentialType.SharedSecret;
clientEndBeh.Credentials.SharedSecret.IssuerName = "owner";
clientEndBeh.Credentials.SharedSecret.IssuerSecret = "Extract this from Appfabric Portal";// (it will look something like the crossed out section in the Image 2" )
EndpointAddress endpoint = new EndpointAddress("sb://shashiraina-demo.servicebus.windows.net/Test.Biztalk.Cloud.Orch/WcfService_Test_Biztalk_Cloud_Orch.svc");

WcfService_Test_Biztalk_Cloud_OrchClient client = new WcfService_Test_Biztalk_Cloud_OrchClient(tcpRelayBinding, endpoint);
client.Endpoint.Behaviors.Add(clientEndBeh);

req.Score = new Score(); ;
req.Score.FirstName = "Shashi";
req.Score.LastName = "Raina";
req.Score.SSN = "123-234-2323";

res.Root = new Root();
res.Root = client.GetCreditScore(req.Score);

Response from the service is below: (Go Ahead and tried it out from your machine)!



Image 3


So, without punching a hole in your DMZ, you could expose your Biztalk services to outside world. I have been involved in complex configurations in ISA reverse proxy to expose Biztalk services to outside world. But using "Appfabric Connect for Services" , all these complex configurations can be eliminated. Moreover, the ACS (Access Control Services) that is part of Azure Appfabric allows creating your own plug-in security framework.

Integrating Biztalk with Mobile CE device

One of my recent projects involved developing Biztalk services to be consumed by Windows CE mobile devices. I got a chance to work with compact Microsoft framework. In an ideal world, one would expose Biztalk as WCF service and use advanced feature set to integrate the services with the mobile client using compact framework. There is a whole set of compact power tools with own svc generating utility that matches svcutil.exe of .net framework world. Compact framework svcutil may work well with pure WCF kind of services (not tested though) but it will not work with Biztalk services that are exposed as WCF using WCF publishing wizard. It seems there is a bug in the svc util tool for compact framework and the only other option is to consume exposed WCF as a webservice from the client side. This can be done by simply using Add Web Reference option in the mobile client project. Doing this will generate requisite client side proxy objects to talk with the exposed Biztalk service. Service side is to be exposed as WCF with basic-http binding to be consumed as a web service from mobile client. There is one caveat to this approach though. While using a SSL at transport level, it was observed that the client could not pass the request to the service hosted inside the corporate domain from a client situated on WAN (internet). Everything would work fine for calls made within the LAN though. The key difference between LAN and WAN scenarios was the use of reverse proxy ISA. Client requests would have to be routed through ISA for WAN scenarios whereas in LAN scenario ISA will be automatically by passed. This issue could be circumvented by exposing Biztalk as ASMX service (instead of WCF) and using SOAP adapter. Apparently, the security information that is carried over from client through ISA to Biztalk is carried in the envelope part of the SOAP message over the wire. It is possible that the way this security information is formatted in a service exposed as WCF v/s the one exposed as ASMX service is different. That could possibly explain why the client call will work in WAN scenario for ASMX type service and not work for WCF type service.

Batching Non-EDI received documents

EDI Batching in Biztalk is done using Batching orchestration that is shipped with Biztalk. This batching orchestration is directly binded with MessageBox and picks up message with EDI.ToBeBatched=True. This flag is set when the message is received using edi receive pipeline. There is a component called "BatchMarker" in the receive pipeline that promotes and sets "EDI.ToBeBatched=True".

In scenario when the incoming message is a Non-EDI type (i.e a Positional Flat file), we have two options to promote the context properties that are required for Batching orchestration to work correctly. First option is to use Orchestration and second is to use a custom pipeline. In this example, I will demonstrate how to use an orchestration:

1. Create an orchestration that takes the Flat File message as input message
2. Create a Outbound Msg (Msgout) representing EDI schema
3. Add a map that does FF-->EDI conversion.
4. Create a correlation set containing 3 EDI context properites. Set these 3 propeties inside an expression shape within the context of a constuct block.

EDI834_Out(EDI.ToBeBatched)=true;
EDI834_Out(EDI.BatchId)=1; //get this ID from "Parties"
EDI834_Out(EDI.EncodingType)=0;// 0 for X12; 1 FOR EDIFACT

5. Initialize the correlation set in send port. This is the key step to promote the 3 context properties else you may end up "writing" the properties and not promoting them.
6. Bind the Orchestration Send Port directly to MessageBox
7. Create a Send Port that Filters "EDI.ToBeBatched=False". You may further differentiate by adding filter for BatchID.
8.Deploy the orchestation and start the send port.

Modifying Biztalk EDI Schemas

There are times when standard Biztalk EDI schemas need to be expanded to include additional validations. It is a 2 step process.

1. Modify the relevant EDI schema and preferably modify the namespace to differentiate it from standard schema. Deploy it.

2. Go To-->Biztalk Admin Console-->Parties-->Select Partner--> Right Click-->EDI Properties-->Click on "Validation and ACK"-->Check "Extended Validation" (Image below)

Checking "Extended Validation" tells Biztalk Engine to validate against the custom schema validations. This is in addition to standard EDI schema validations that come out of box.

Extending VPC hard drive

It has been a while since I last blogged. During this period, lot of things have happened in my life. We were blessed with twins and I took a new role with a new company. A role that lets me focus on integration more deeply and also puts me more closer to my clients. Recently, I completed a POC integrating Mobile Windows Devices with Biztalk. It was an exciting experience. No, this blog is not about Mobile integration. This is about something that we face a lot while using VPC images. Well, sooner or later we all will run out of C drive space on our VPC. Yes, it is straight forward to add a new hard drive and name it anything (say D). But that still does not do anything to existing C drive space. There are some programs that by default install on C drive and for these programs C drive should have enough space. There is a way t0 install programs that are looking for C drive to a D drive for example. This post is not talking about that. This post will help to extend existing C drive without using paid softwares like Partition Magic. I actually used a free software to achieve this. Below are the steps:

1. Add a new hard drive (.vhd file) to VPC of desired size but more than the existing hard drive (.vhd drive) that needs to be extended. Call this New partition D:
2. Use DriveImageXML a free software to create an image of existing hard drive (C:)
3. Restore the created image on to (D:)
4. shut down the VPC and remove the original drive C and keep only drive D:
5. Fire it up, you may get issues in booting up the new drive. Use http://www.runtime.org/driveimage_faq.htm. Go to section:
I restored my boot drive, why does it not boot?
and follow Fix Manually section.
6. This should boot up the machine with new hard drive D:. But you may end up having a blank desktop and no programs will work. This is because they all were installed in drive C and the new drive being used is D.
7. To fix this use: http://support.microsoft.com/kb/223188

Using Biztalk in transaction mode with a WCF client

While trying to use a WCF client to send a message to Biztalk as a transaction, I got the following error:


"Cannot access a disposed object. Object name: 'TransactionScope' "


On investigation, I found that this can happen when:

1. Recieve port/location does not have a corresponding subscription (orchestration / send port) created.


2. Send Port is created but not enlisted. This is same as no subscription found in database.

When enable transactions check box is clicked while configuring receive location as shown below:

Behind the scenes, a transaction scope object is created that will attach itself to incoming message sent from a client and form a one continious transaction involving MSDTC. This enables WCF client to control the whole transaction involving Biztalk. Client can use "commit" and "rollback" commands to commit and rollback message in Biztalk messagebox database. Very powerful!

Reason:

At core, Biztalk works on pub-sub mechanism. In absence of any subscription, receive location can't really participate in transaction and create a transaction object. Hence the error is raised "cannot access a dispose object" (transaction object)

Solution is simple: Create a subscription for the receive port/location in form of a send port or an orchestration.

Using SAP Biztalk 2006 R2 WCF adapter

Work involved in configuring SAP adapter can be divided into 2 steps. First step involves work done by Biztalk team and second step involves work done by SAP team.

Step 1: BIZTALK TEAM

Biztalk side needs to define a Program ID. SAP adapter then registers this Program ID to the SAP gateway defined in the URL. This Program ID then needs to be registered from SAP side. SAP admin should follow “Performing Tasks Using the SAP GUI for Specific SAP Adapter Scenarios” section of SAPAdapter.chm file.

Program ID will be unique per listener else SAP will send the document to random listeners. Each Receive location will be tied to a unique Program ID for an environment. Example: If same receive location is being used in 3 environments (DEV, QA and Prod) then 3 Program IDS are required to be set up for that single receive location. RFC checks the connection between SAP and biztalk using program id registered channel. SAP then sends IDOC to biztalk. Biztalk accepts this IDoc and processes it.
Configure recv location as shown:

Configure following URL at recv location

sap://CLIENT=[client ID] ;LANG=EN;@a/[servername]/[system ID]
?ListenerGwHost=[hostname]&ListenerGwServ=[gateway service name] &ListenerProgramId=[Program ID}

SAP team should supply with the above variables.
Do not forget to ask for user name and password to configure screen below:


Step 2: SAP TEAM

SAP team to supply:

1. client ID
2. servername
3. system ID
4. hostname
5. gateway service name
6. User ID
7. Password

Configuring BAM and WSS adapter on 64 bit machine

BAM and WSS adapter have conflicting requirements for 64 bit machine. BAM needs 32 bit IIS and WSS 3.0 needs 64 bit IIS. To make matters further complex, it is not possible to host 32 bit IIS and 64 bit IIS on same 64 bit machine. So how can Biztalk communicate with a MOSS server running on 64 bit machine? Following configuration makes this possible:

1) Enable 32 bit IIS on the 64 bit machine by running following command:

cscript %SystemDrive%\inetpub\AdminScripts\adsutil.vbs set w3svc/AppPools/Enable32bitAppOnWin64 1

(1 means “true” and 0 means “false”)

2) Now, IIS is 32 bit and follow standard BAM configuration steps.
3) Install WSS 2.0 SP2 on the Biztalk machine to enable configuring WSS adapter. Also, configure WSS adapter on MOSS machine that hosts WSS 3.0
4) To install WSS 2.0, register ASP.Net 2.0 (32 bit version) from “Windows\Framework” folder by running aspnet_regiis.exe –i
5) Download and install WSS 2.0 SP2
6) Install WSS 2.0 in server farm mode and then create and extend the website. This website will host the WSS adapter.
7) When installing WSS 2.0 on clustered machine, use option “Join existing …”
8) Run Biztalk Configuration wizard and configure WSS adapter on Biztalk machine.
9) Run Biztalk Configuration Wizard on MOSS machine and configure adapter there.
10) Make the AppPool Account that is running WSS 2.0 website a member of SharePoint Group and SharePoint Site Collection on MOSS machine hosting WSS 3.0.

When WSS is configured, it creates BTSharePointAdapterWS. WSS adapter uses this web service to communicate with MOSS form libraries, lists and document libraries. This service need not be running on Biztalk machine but it should be running on MOSS machine for proper communication back and forth.

Cross Updating Windows Workflow Tasks in Parallel Activity

Parallel activities in Windows Workflow are independent as the name suggests. How could we then update Task A from Task B when Task A and Task B are running in parallel? After spending 1 hour, I figured it out.

1) Use "Update Task" shape for updating Task A and drop it in the "parallel" arm of Task B. The position of this task could be above the "Complete Task" or as required. But it should be under the "create task" shape for Task B.

2) Here is the imporant part. Set the correlation token as the correlation token used for Task A even though this task is under Task B related tasks. This correlation token will direct the update task to the correct task (Task A in our case)

3) Bind the TaskID to existing Task ID for Task A.

3) Use syntax "TaskProperties.ExtendedProperties["NameOfInfopathField_inTaskA_thatis_to_be_updated"]="desiredvalue"

Important:

Parallel workflows in WF are actually processed in a sequence. Left branch is processed first and then control moves towards the right . While cross updating tasks, correlation ID is the key that ties the different tasks together. For tasks on the extreme right arm, all the correlation IDs in the workflow are available. But this selection reduces are we move towards left. In order to update a task on the right side of a given task, the correlation ID will not be available from the work flow design view. But going to deign code file(*.designer.cs / *.designer.vb) gives the flexibility to link the correlation ID with the task that is to be updated.

i.e.

this.UpdateFromTaskOnLeftSide =CorrelationToken (this will be token for the task on the right side that is being updated from left side).

Accessing SQL data from Infopath Form Control

A control in infopath form can be configured to receive data from XML, WebService, SharePoint List and SQL database (only). In this article I will focus on using SQL database as secondary datasource. I used SQL database to hold values for login in one column and second column gave the user groups for that LogIn. I used these Groups to control the views that a user can access. For Example: You could add rule to restrict an IT user to view data for IT only and not for Finance or Marketing.



This is what you do:


1) Create a table and name it "RulesGroup". Add two columns : UserLogIn and UserGroup




2) In the infopath form create two text box control: UserName and UserGroup

3) Create a secondary datasource and configure it to receive data from SQL database. Use the wizard to point to the correct table.

4) In the infopath form: Use infopath function username() to populate UserName text box.
Map the UserGroup text box with following Xpath:

xdXDocument:GetDOM("RolesGroup")/dfs:myFields/dfs:dataFields/d:RolesGroup[@UserLogIn = "shashi"]/@UserGroup

PS: XML generated by the wizard marks the 2 user defined columns as "attributes". So there is need to user @ in Xpath expression.

In our case if UserName is "shashi" the UserGroup will be "Operations".

New users and Groups can be added in DB as required and accordingly Infopath Views can be programmed based on Groups.

Remember: While using form services to host infopath form, additional requirement will be creating a new Univeral Data Connection(UDC) file in Data Connection Library under the site collection. This UDC file maps the connection details between infopath form and SQL service. SQL server can't really authenticate the user name being passed directly from the infopath form. The user name comes as "" (blank). One way of getting around is to use proxy user name and password in case the user name is blank. But this will compromise security in Big Way. Use of UDC file helps in this regard. Also, it allows you set up custom User Groups based on User Names. These user groups can be then used to limit access to certain views as explained before in this article.