Mike Taulty's Blog
Bits and Bytes from Microsoft UK
Correlation in BizTalk 2004

Blogs

Mike Taulty's Blog

Elsewhere

I'm learning a little bit more about BizTalk Server (R) 2004 at the moment and something that had me stumped until today was how to get correlation working so I thought I'd blog something on that up here having just managed to get what I wanted working (wasn't hard at all in the end). Correlation is the simple idea of sending a message out to someone and then expecting to receive a response back (more than likely at a later point) and using some common piece of data in the returned message to tie it up (i.e. correlate) with the sending message. Within BizTalk 2004, there's some really powerful functionality built around the Orchestration engine that allows for pretty much automatic correlation. As a very simple example, assume I have a message which looks like; <order> <supplier>SUPP001</supplier> <productCode>PROD001</productCode> </order> and I'm going to send this from my Orchestration schedule to some trading partner who's going to reply with a message something like; <invoice> <supp>SUPP001</supp> <prod>PROD001</prod> </invoice> Clearly, it's contrived and we'd normally correlate on something like a GUID but hopefully it's clear that the invoice message that I'm getting back is meant to relate to the one that I sent out. So, how do we make this work? Well, firstly, we've got to define schemas for these 2 document types and, within those schemas, we need to promote properties (through property schemas) that we're going to use in messaging. So, in our case we need to promote (to promoted properties) the 'supplier', 'productCode', 'supp' and 'prod' fields in our order and invoice schema respectively. Having done that we configure the a port in the orchestration to send the order message out and we configure a port to receive the invoice back (presumably this is just one step in some larger business process). We then need to define a 'Correlation Set' that these 2 ports can share. This is an instance of a 'Correlation Type' which is data that we need to match up between the outgoing document and the incoming document in order to consider that incoming document a 'response' to the outgoing one. In order to make data a part of our 'Correlation Type' we need to make sure the data is available to us as promoted properties so we need to go to our properties ('supplier' and 'productCode' in both the invoice and the order schemas) and make sure they're promoted. Having done that we can then define a 'Correlation Type' which consists of the 'supplier' and 'productCode' promoted properties. We then make a Correlation Set of this type for our Orchestration (by configuring it in the Orchestration View window) and we associate this particular Correlation Set with our ports. For the sending port we configure 'Initialises correlation set' and for the receiving port we configure 'Follows correlation set'. If you're playing with BizTalk 2004 a little you can probably imagine that what might be happening here is that the receiving port is gaining a more granular subscription to the message box. Having done that, we should find that our receiving port within our Orchestration nicely picks up the 'correct' messages that are the responses to the outgoing messages that we sent by only picking up messages that match the data fields that we specified. But...one more thing. In the example I gave, I had fields named 'supplier' and 'productCode' in my order document but fields named 'supp' and 'prod' in my invoice document. If I follow the steps outlined above and try and build an Orchestration like this then my current build of BizTalk 2004 gives me an error saying something like 'metadata property supplier for correlation X does not exist in message Y'. Essentially, it looks like the designer is expecting that if I'm doing correlation then the 'fields' that I'm using in both documents will be named the same way. How to get around this? Well, what I'm doing right now is opening up my invoice.xsd document in a plain XML editor (in VS.NET 2003) and just tweaking the schema annotations that have been added for me to change the names that are being used. So, in the example above VS.NET made a schema annotation like this for me for my invoice schema; <xs:annotation> <xs:appinfo> <b:properties> <b:property name='ns0:Supp' xpath='/*[local-name()='Invoice'and namespace-uri()='http://correlate.Out']/*[local-name()='supp'and namespace-uri()='']' /> <b:property name='ns0:Prod' xpath='/*[local-name()='Invoice'and namespace-uri()='http://correlate.Out']/*[local-name()='prod'and namespace-uri()='']' /> </b:properties> </xs:appinfo> </xs:annotation> and we can see that it's used a name of 'supp' and 'prod' becase that's how my elements were named. However, if I change these names in this annotation to be the same as the ones in the order document, i.e. <xs:annotation> <xs:appinfo> <b:properties> <b:property name='ns0:supplier' xpath='/*[local-name()='Invoice'and namespace-uri()='http://correlate.Out']/*[local-name()='supp'and namespace-uri()='']' /> <b:property name='ns0:productCode' xpath='/*[local-name()='Invoice'and namespace-uri()='http://correlate.Out']/*[local-name()='prod'and namespace-uri()='']' /> </b:properties> </xs:appinfo> </xs:annotation> then my Orchestration compiles up and works just fine (note: only the name of the property has been changed in the example above - the xpaths are using the original, real names of the elements involved). If your properties are named the same in your outgoing and incoming documents then you should not have to do this but this works for me right now in the scenario where those names don't match so I thought I'd share.

Posted Sat, Nov 15 2003 12:00 AM by mtaulty

Comments

mtaulty wrote re: Correlation in BizTalk 2004
on Tue, Jul 27 2004 10:51 PM
The article is very informative :)
mtaulty wrote re: Correlation in BizTalk 2004
on Tue, Aug 24 2004 2:04 PM
Very helpful information...stuff like this should make its way into the Biztalk documentation
mtaulty wrote re: Correlation in BizTalk 2004
on Mon, Sep 6 2004 3:37 PM
nice info.
mtaulty wrote re: Correlation in BizTalk 2004
on Thu, Feb 10 2005 2:48 AM
Very handy. This helped clarify correlation for me.
Thanks.
mtaulty wrote re: Correlation in BizTalk 2004
on Tue, Feb 15 2005 3:52 AM
Real Good stuff,

I was getting confused abt Correlation, Correlation types, Correlation Sets.

A huge relief now.. :D
mtaulty wrote re: Correlation in BizTalk 2004
on Mon, Feb 21 2005 10:08 PM
Hi,

I am new to biztalk 2004. I have been following the steps in Module 2 of the tutorial and after assigning correlation sets, I get this error.

"you must specify at least one already-initialized correlation set for a non-activation receive that is on a non-selfcorrelating port "

Have I done anything wrong or is there somethine missing in the tutorial?
please help

rekha
mtaulty wrote re: Correlation in BizTalk 2004
on Tue, Feb 22 2005 4:51 AM
Hi Rekha,

I'm not sure exactly what's in the tutorial but what this is saying is that you have some number of receive ports on a schedule and one of them must be specified as "activating" the schedule (i.e. starting the process).

So, you need to go to the first receive port on the diagram, get its properties and switch the property to say that it "activates the schedule" - can't quite remember what the property is called but I hope that helps.
mtaulty wrote re: Correlation in BizTalk 2004
on Thu, Apr 7 2005 10:32 PM
Hi,
Newbiz question, I try the example in book BizTalk Server 2004 Unleashed chapter 11. I have changed my document type in Business Rule Composer but the process still nothing when I make a rule condition correlate with database. I have made the fact retriever name is dbFactRetriever, but when I'll try to add it in fact retriever property, nothing option for dbFactRetriever assembly.
Here is the result of Tracking output :

RULE ENGINE TRACE for RULESET: RFP 08/04/2005 6:53:57

FACT ACTIVITY 08/04/2005 6:53:57
Rule Engine Instance Identifier: 78cb2950-1fff-42bc-8bf4-6f8df4f1515d
Ruleset Name: RFP
Operation: Assert
Object Type: TypedXmlDocument:RFPEstimateXML.RulesRFP
Object Instance Identifier: 74

FACT ACTIVITY 08/04/2005 6:53:57
Rule Engine Instance Identifier: 78cb2950-1fff-42bc-8bf4-6f8df4f1515d
Ruleset Name: RFP
Operation: Assert
Object Type: TypedXmlDocument:RFPEstimateXML.RulesRFP:/*[local-name()='RFP' and namespace-uri()='http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP']
Object Instance Identifier: 75

CONDITION EVALUATION TEST (MATCH) 08/04/2005 6:53:57
Rule Engine Instance Identifier: 78cb2950-1fff-42bc-8bf4-6f8df4f1515d
Ruleset Name: RFP
Test Expression: TypedXmlDocument:RFPEstimateXML.RulesRFP:/*[local-name()='RFP' and namespace-uri()='http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP'].Hours > 160
Left Operand Value: 180
Right Operand Value: 160
Test Result: True

CONDITION EVALUATION TEST (MATCH) 08/04/2005 6:53:57
Rule Engine Instance Identifier: 78cb2950-1fff-42bc-8bf4-6f8df4f1515d
Ruleset Name: RFP
Test Expression: TypedXmlDocument:RFPEstimateXML.RulesRFP:/*[local-name()='RFP' and namespace-uri()='http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP'].Hours <= 160
Left Operand Value: 180
Right Operand Value: 160
Test Result: False

FACT ACTIVITY 08/04/2005 6:53:57
Rule Engine Instance Identifier: 78cb2950-1fff-42bc-8bf4-6f8df4f1515d
Ruleset Name: RFP
Operation: Retract
Object Type: TypedXmlDocument:RFPEstimateXML.RulesRFP
Object Instance Identifier: 74

FACT ACTIVITY 08/04/2005 6:53:57
Rule Engine Instance Identifier: 78cb2950-1fff-42bc-8bf4-6f8df4f1515d
Ruleset Name: RFP
Operation: Retract
Object Type: TypedXmlDocument:RFPEstimateXML.RulesRFP:/*[local-name()='RFP' and namespace-uri()='http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP">http://RFPEstimateXML.RulesRFP']
Object Instance Identifier: 75

Thanks for all the help in advance

Reng
mtaulty wrote re: Correlation in BizTalk 2004
on Tue, Jun 7 2005 8:00 AM
Good effort in the document. With some formatting and paragraphing, it would make easier reading.
mtaulty wrote re: Correlation in BizTalk 2004
on Tue, Jun 7 2005 3:07 PM

Hi,

The formatting of these posts is poor - I auto-converted them from earlier posts using different blogging software - sorry!

Mike.