[VoiceOps] SIP test equipment

[John Todd]

On Jul 9, 2010, at 12:04 PM, Kristian Kielhofner wrote:

> On Fri, Jul 9, 2010 at 2:18 PM, John Todd <jtodd at loligo.com> wrote:
>> I'm surprised you're not creating your own.  Given the cost for the
>> higher-end packages, it seems useful to consider in-house  
>> construction,
>> though I suspect you have considered that already and perhaps  
>> discounted it
>> for political reasons.  I probably would not suggest this of a shop  
>> that
>> doesn't have the expertise, but clearly you DO have the expertise  
>> even if
>> just to create a specification that would be sufficient to give to an
>> outsourced development shop.
>>
>> Do free or reasonably-priced libraries exist to determine MOS/PESQ  
>> given a
>> suitable stream of RTP or other audio format?  That would seem to  
>> me to be
>> the most difficult part of building this system, as other  
>> components to
>> create suitable call volumes certainly seem to pre-exist.
>>
>> JT
>
> JT,
>
>  Build vs buy...  In my various ventures over the last several years
> I/we have already built a dizzying array of tools and test tools.
> Testing is not our core competency and I don't think a single person
> in the company is especially interested in expending resources across
> all departments (legal, development, accounting, etc) to coordinate
> the items necessary for a development project (outsourced or not) that
> really doesn't add significant "value" to the company.
>
>  In these cases it's nice to just make a purchase and have an
> industry recognized test solution available.  I could spend the same
> (or 3x more) money on developing my own equal or better test solution
> and that still won't have the "value" an Ixia or Empirix test report
> can provide (to some people).
>
>  I personally would love to see us spend the time and resources on
> developing a highly capable open source solution.  While it might be
> of the most benefit to the rest of the world it's not in the best
> interest of the company.  I'm also concerned that at this scale
> (several thousand RTP streams with proprietary/patented codecs, full
> stats, MOS, PESQ, etc) we may very well have to depend on some
> proprietary/hardware solution anyway...


Kristian -
   I completely understand the benefit of "build vs. buy" when looking  
at the easier path - it's almost always "buy".  But "easy" doesn't  
always mean "best", though sometimes it does.  Almost always, though,  
buying an off the shelf solution is faster.  I don't _always_ advocate  
building things yourself, but it seems that you of all people would  
have a set of tools already well-understood to go the short distance  
to creation of a workable system that would provide results (but  
again, I don't know what your needs are - is this for giving to less- 
skilled NOC staff, or just for providing pretty graphs for a clueless  
C-level exec, or what?  You don't need to answer, that's just a  
hypothetical question.)

   I'd take some issue with your 3x estimate, but I don't know what  
you want to get out of the system.  Simple reports are easy on a small  
set of focused tests, and could probably be done with not a huge  
investment in development time.  Complex web interfaces and error  
management is what drives up the cost, but you may not need that.  I  
know the Empirix/Spirent/other vendor solutions run into the six- 
figure range for decent-sized platforms that will do the volumes  
you're talking about, which would cover quite a few hours of time and  
effort towards an in-house solution.

   Each company has their own priorities and choices; I'm not trying  
to say you should develop it yourself, I'm just surprised you're  
not.  :-)

   As far as specialized hardware: If you want to run in real-time,  
I'm sure that it would require special gear.  However, if you're OK  
with post-processing, then I suspect off-the-shelf equipment would  
work fine, up to even >10k channels given the success we've had with  
even something as computationally "heavy" as Asterisk at those  
volumes.  With a lightweight RTP collector, I suspect your performance  
could get much better but then disk I/O becomes an issue...)

JT