msunified.net

After deploying the Monitoring Server role in your OCS organization you start getting some feedback and reports to analyze. In this blog I will try to explain in short how to analyze the data from the monitoring reports.

On how to deploy the Monitoring role, see my earlier post on the subject http://msunified.net/2009/05/20/ocs-quality-of-experience-qoe-%e2%80%93-quick-facts/

The MOS value

In the monitoring reports you see a MOS score for each instance that is being measured. To determine the MOS score an absolute categorization rating (ACR) is used. The ACR scale rates from 1-5 where 5 represents an excellent experience. In a manual process the users are asked to rate the quality their experience. When all the users have rated their experience, the average of these values are calculated as a mean option score (MOS). Although MOS scores are not a perfect representation of the listening experience, they do make it possible to compare and contrast listening experiences.

If group A reports an MOS of 4.1 and group B reports an MOS of 2.2, it is safe to say that, on average, listeners in group A had a much better experience than listeners in group B. 

The Monitoring server does not ask users to rate their listening experiences on scale of 1-5; instead the Monitoring server uses a series of algorithms to predict how users would rate the quality of each listening experience. Based on those algorithms the Monitoring server reports several MOS scores.

The MOS scores

• Listening MOS -A prediction of the wideband quality of an audio stream being played to a user. The MOS score takes into account audio fidelity and distortion as well as speech and noise levels
• Sending MOS -A prediction of the wideband quality of an audio stream sent from a user. The MOS score takes into account audio fidelity and distortion as well as speech and noise levels
• Network MOS -Another prediction of wideband quality of an audio stream played to a user. In this case, however, only network factors are considered such as the audio codec used, packet loss, packet errors, and jitter (the variation in delay time of packets arriving at a destination) 
  • NOTE: Latency should not exceed 150 ms. In my experience a latency up to 300 gives a satisfying experience, as long as jitter is under control
• Conversational MOS- A prediction of the narrowband conversational quality of the audio stream played to the user. This value is indicative of how a large group of people would rate the quality of the connection for holding a conversation

NOTE: Narrowband refers to audio codec that use an 8-kHz sample rate. Wideband refers to audio codecs that use a 16-kHz sample rate. Telephone-quality communication is normally categorized as narrowband.

For the complete documentation on how to deploy and use the Monitoring Server download the Microsoft® Office Communications Server 2007 Quality of Experience Monitoring Server Guide here: http://www.microsoft.com/downloads/details.aspx?FamilyId=9ED29D74-3391-4902-BF2C-6757410F3335&displaylang=en

Technical Refernces:
Technet: http://technet.microsoft.com/en-us/library/bb894481(office.12).aspx