[VoiceOps] Lack of Quality Industry-wide

[Ivan Kovacevic]

We have had to cook up our own monitoring and right now it consists of:



a)      OPTIONS/ping/latency/jitter monitoring of IP addresses for each
vendor

b)      Near real time monitoring with email alerts and CDR examples
whenever each vendor falls out of bounds on:

a.      Percentage of connected calls (with a breakdown of TOP 5 SIP
failure responses (503, 404, 403, etc).

b.      Percentage of PDD calls (8 seconds or more)



This is a pretty blunt tool, but it has helped identify cases where a
vendor goes completely down or has systemic problems that are noticeable to
customers.



We have been reaching out to carriers proactively, problem CDRs are sent
along with the alert email, which makes it easy to forward. However I am
not sure, doing so actually improves anything. The penalty box approach may
work better.



Best Regards,



Ivan Kovacevic

Vice President, Client Services

Star Telecom | www.startelecom.ca | SIP Based Services for Contact Centers
| LinkedIn
<http://www.linkedin.com/company/star-telecom-www.startelecom.ca-?trk=top_nav_home>

*From:* VoiceOps [mailto:voiceops-bounces at voiceops.org] *On Behalf Of *Anthony
Orlando via VoiceOps
*Sent:* Tuesday, October 06, 2015 12:19 PM
*To:* Peter Beckman <beckman at angryox.com>; VoiceOps <voiceops at voiceops.org>
*Subject:* Re: [VoiceOps] Lack of Quality Industry-wide



You are correct Peter.  It's a  constant struggle.  For years I used a
product from Empirix called Intelisight.  It gave us the ability to monitor
carriers (and other managed objects) with defined sets of KPI's.  Once I
saw a carrier have issues (PDD, quality, excessive 503's) I used to call
them and tell them they had a problem.  Hours or days later they would
resolve.  I got away from that practice and just routed around them.
Funny how they notice that.  Once the issue was resolved I put them in a
penalty box and slowly added traffic.  Once they realized I was measuring
their performance and there would be financial repercussions it's amazing
how the quality of their under lying carriers improved.  My carrier ticket
count dropped by 25-30%.


------------------------------

*From:* Peter Beckman <beckman at angryox.com>
*To:* VoiceOps <voiceops at voiceops.org>
*Sent:* Tuesday, October 6, 2015 10:27 AM
*Subject:* [VoiceOps] Lack of Quality Industry-wide


In the last 3 months I've been consistently frustrated by my carriers.

    "3-4 minutes is acceptable delay for delivery of SMS messages."

    "Our termination change now throws 504s instead of 503s 20-50% of the
      time and adds 2-3 seconds of delay to your call attempts. We didn't
      notice, and though you did, it's been 3 days and we haven't fixed it.
      Sorry!"

    "There was an outage? Works for me now!"

    "Someone upstream is intentionally dropping SMS messages. But we can't
      say who it is, and we can't get them to fix it because they aren't our
      carrier."

Does the industry just suck at knowing when their stuff is broken, or only
react when enough customers complain? Do carriers simply not instrument,
monitor or graph metrics of their operations and proactively monitor and
fix issues?

My Thresholds:
    * SMS delivery end-to-end: Under 10 seconds
    * 503 Route Advance: Under 1 second
    * Response/Notice to termination/API/origination/server outage: 20
minutes
    * Fix a major issue (or provide a fix timeline): 3 days

Too many times in the last few months _I_ have been the canary in the
carrier's coal mine bringing attention to places where their operations are
broken or delayed. And even then, unless I escalate to management (like CEO
level), things move at the speed of a sloth. Most of the time it seems I
monitor my carrier's infrastructure more closely than they do.

I hope and dream of the unicorn carrier -- such great operational awareness
and execution that it doesn't matter how great their customer service is,
I'll never have to talk to them.

Beckman
---------------------------------------------------------------------------
Peter Beckman                                                  Internet Guy
beckman at angryox.com                                http://www.angryox.com/
---------------------------------------------------------------------------
_______________________________________________
VoiceOps mailing list
VoiceOps at voiceops.org
https://puck.nether.net/mailman/listinfo/voiceops
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/voiceops/attachments/20151006/2e5b3ae0/attachment.html>