[j-nsp] Multi Core on JUNOS?

[Jeff Haas]

Adam,

> On Oct 9, 2015, at 9:45 AM, Adam Chappell <adam.chappell at gmail.com> wrote:
>> 
> I can imagine that making rpd MT is probably hard to the point of almost
> not being worth the benefit (with current REs), unless one can adequately
> break down the problem into divisable chunks of work that are insensitive
> to execution order.  BGP peer route analysis, comparison against import
> policy and current RIB might fall into that category, but not without a lot
> of locking and potential for races.

It is non-trivial work.  But it is also somewhat easier to incrementally introduce threading than it is to split large hunks of code and workflow into multiple processes.

We're not going into deep technical detail on mailing lists such as these at the moment.  The work is in progress.  

> 
> I think there's more bang for buck in the 64-bit address space change what
> with Internet FIB table size, and I'm quite interested in the developments
> to make rpd 64-bit clean which I'm sure are also not insignificant. I
> notice Mr Tinka already expressed a conservative view on jumping into a
> 64-bit rpd and I can totally understand and appreciate that view. Juniper
> haven't made this a default on the 14.1R5 cut of code that we're currently
> testing, so I imagine they're still looking for bleeding-edge feedback to
> whittle out the potential nasties.

I've not quite understood our strategies for choosing whether we do 32 or 64-bit by default.  The balance is covered somewhat by the split between having a RE with sufficient base memory vs. release, but the group responsible for that choice is being conservative.  In my opinion, 64-bit has been perceived internally clean for quite some time, and we do a significant portion of our internal testing in that environment.

The biggest reason to be cautious about the default mode is that on systems with insufficient memory, or systems with insufficient need, the cost of 64-bit may be a bit much.  Since the bulk of internal data structures are pointers, you tend to see at least a 2x increase in base memory use when running in 64-bit mode.  

> 
> I'm quite intrigued by the tidbit in the Juniper docs, though, that
> suggests that switching from a 32-bit to a 64-bit rpd is not service
> affecting though which means that the wait-and-see approach is viable?  Or
> am I totally misunderstanding this?
> 
> https://www.juniper.net/documentation/en_US/junos14.1/topics/reference/configuration-statement/routing-edit-system-processes.html

Ok, I find this comment to be confusing if not misleading:
Tip: You need not restart the routing protocol process (rpd) to use the 64-bit mode.

I suspect the most generous reading of this is that rpd is restarted for you.  I'll open a doc PR for clarification.

> It doesn't say that one needs NSR or any sort of "help" from the backup RE
> which might be the first assumption. So I wonder how they manage to get one
> process to exit and the other one to start up with different runtimes,
> differently sized pointers etc., and somehow share the same in-core RIB and
> protocol state etc etc.? If this really does work, there's probably someone
> sitting somewhere in Sunnydale immensely smug and under-stated right now,
> and if so I'm sure he/she'd eat the MT problem for breakfast!

I've often thought working over the hell-mouth would be interesting, but engineer retention in Silicon Valley is already tricky enough.  Good thing the office is in Sunnyvale. :-)

-- Jeff