This is a follow-up post in a series called Todays Lesson Learned started by fellow Lync MVP Tommy Clarke with this post: Todays lesson learned–Exchange 2010 upgrade with Active Sync and MobileIron It is a blog series noting down “learn from my mistakes” and “gotchas that can stop an entire project” type of problems with learning by doing and learning the hard way.

Lately I have been working with Lync Server 2010 and using System Center Operations Manager 2007 R2 (OpsMgr) for the best possible monitoring and troubleshooting environment for a customer. Everything got set up correct and the integration seemed to look ok. Except that Synthetic Transactions did not work.

Background

Synthetic Transactions can be run from PowerShell in any Lync deployment. They are “test-cs” cmdlets and enables you to test features serverside with actual users. If you run (get-command “test-cs*”).count from Lync Server Management Shell you will see that there are 36 test cmdlets ranging from testing the ability to log on (Test-CsRegistration) to testing Audio Video conferencing (Test-CsAvConference).

OpsMgr continuously run Synthetic Transactions (ST) against the Lync environment to make sure everything is working. In a best practice environment you should deploy a dedicated server to run the ST’s from. This is because they may generate heavy load on you production servers. OpsMgr runs a PowerShell script that access the node remotely and runs the test cmdlets

For more information on how to deploy Lync Server 2010 Management Pack for System Center Operations Manager 2007 R2, download and read the documentation here: http://www.microsoft.com/download/en/details.aspx?id=12375

Symptoms

After OpsMgr was deployed we noticed the following symptoms

• Some of the ST’s did not work failing with a “This operation has timed out” message
• The ones that did not work was
  • AV conferencing
  • Instant Messaging Conferencing
  • Instant Messaging P2P
  • Presence
  • P2P AV
• The rest of them seemed to work
• Logging on the ST Node we ran the test cmdlets manually and all of them worked
  • Telling us that the users configured to run the cmdlets worked with the CsHealthMonitoringConfiguration
  • We did not have the problem depicted in this forum post: http://social.technet.microsoft.com/Forums/en-US/ocsmonitoring/thread/81d1b976-ca07-41e8-9ede-6c600f5c2e33/
• So the problem was that it worked locally, but not remotely from OpsMgr and no other indications or errors where thrown

Resolution

I figured it had to be a configuration problem somewhere and went through all of the configuration on the Node from start to end using this blog post: http://www.vnext.be/2011/03/06/scom-opsmgr-lync-2010-management-pack-deploying-synthetic-transactions-sts/ It hit me when I saw the cmdlet for configuring CsTrustedApplication that we defined a port this application would access.

New-CsTrustedApplication -ApplicationId -TrustedApplicationPoolFqdn -Port <PortNumber> -Verbose

Checking the local Windows Firewall on the server I found that an exception for that port had not been created, and it dawned on me that they need to be created manually. Adding the exception for the port on the server and recalculating the ST health on OpsMgr solved to problem this time, saving the day and made me able to roll out Lync to that customer.

Conclusion

Always remember when creating Trusted Applications you need to manually create a firewall rule allowing inbound traffic to that specific port.