[j-nsp] MAC table size experience w/VPLS
ILTCHEV Alain
Alain.ILTCHEV at ncnumericable.com
Wed Jun 27 12:24:20 EDT 2007
Hello all,
I have a M320 in production and around 7800 entries in its MAC address table. I am not sure if this is the right command to type in but here is its current memory state. However, I couldn't tell you in which type of memory the router stores this MAC address table.
ailtchev at XXXRJ_re0> show system virtual-memory
Memory statistics by bucket size
Size In Use Free Requests HighWater Couldfree
16 201213 2051 213665014 1280 140
32 19637 75 71773398 640 99635
64 464450 101758 96205695 320 4561
128 11748 2684 10871433 160 296
256 9406 2786 8757101 80 1765
512 169 7 17273697 40 0
1K 39 5 422559 20 0
2K 81 209 4026 10 1375
4K 190 1 261667 5 0
8K 39 3 2842 5 0
16K 29 0 113 5 0
32K 5 0 1963 5 0
64K 16 0 16 5 0
128K 3 0 3 5 0
256K 5 0 5 5 0
512K 2 0 2 5 0
Memory usage type by bucket size
Size Type(s)
16 uc_devlist, nexusdev, iftable, temp, devbuf, atexit, COS, BPF,
DEVFS mount, DEVFS node, vnodes, mount, pcb, soname, proc-args, kld,
MD disk, rman, ATA generic, bus, sysctloid, sysctl, pfestat, ifstate,
pfe_ipc, mkey, rtable, ifmaddr, ipfw, rnode, rtnexthop
32 atkbddev, dirrem, mkdir, diradd, freefile, freefrag, indirdep,
bmsafemap, newblk, temp, devbuf, tseg_qent, COS, vnodes,
cluster_save buffer, pcb, soname, proc-args, sigio, kld, Gzip trees,
taskqueue, SWAP, eventhandler, bus, sysctloid, sysctl, uidinfo,
subproc, pgrp, ippool, pfestat, itable32, ifstate, pfe_ipc, mkey,
rtable, ifmaddr, ipfw, ifstat, ifdevice, tagnh, rnode, rtnexthop
64 isadev, ip6ndp, MFS node, allocindir, allocdirect, pagedep, temp,
devbuf, lockf, COS, NULLFS hash, DEVFS name, vnodes,
cluster_save buffer, vfscache, pcb, soname, proc-args, file,
AR driver, AD driver, Gzip trees, rman, eventhandler, bus, sysctloid,
sysctl, subproc, pfestat, pic, ifstate, pfe_ipc, mkey, ifaddr,
rtable, ipfw, rnode, ingrentry, inifmulti, rtnexthop
128 ZONE, freeblks, inodedep, temp, devbuf, zombie, COS, DEVFS node,
vnodes, mount, vfscache, pcb, soname, proc-args, ttys, dev_t,
timecounter, kld, Gzip trees, ISOFS node, bus, uidinfo, cred,
session, pic, itable16, ifstate, pfe_ipc, mkey, rtable, ipfw, ifstat,
tagnh, metrics, rtnexthop, iffamily
256 iflogical, iftable, MFS node, FFS node, newblk, temp, devbuf,
NFS daemon, vnodes, proc-args, kqueue, file desc, Gzip trees, bus,
subproc, itable16, ifstate, pfe_ipc, sysctl, rtable, ipfw, ifstat,
rnode, rtnexthop, iffamily
512 iflogical, UFS mount, temp, devbuf, mount, BIO buffer, ptys, ttys,
AR driver, Gzip trees, ISOFS mount, msg, ioctlops, ATA generic, bus,
proc, pfestat, lr, ifstate, pfe_ipc, rtable, ipfw, ifstat, rtnexthop
1K iftable, temp, devbuf, NQNFS Lease, COS, kqueue, kld, AD driver,
Gzip trees, sem, MD disk, bus, ifstate, pfe_ipc, mkey, ipfw
2K uc_devlist, iflogical, UFS mount, temp, devbuf, BIO buffer, pcb,
AR driver, Gzip trees, ioctlops, bus, ipfw, rcache
4K memdesc, iftable, UFS mount, temp, devbuf, kld, Gzip trees, sem, msg,
bus, ipfw, ifstat
8K iflogical, UFS mount, temp, devbuf, syncache, Gzip trees
16K indirdep, temp, devbuf, shm, msg
32K devbuf, kld, Gzip trees
64K pagedep, devbuf
128K VM pgdata, MSDOSFS mount, kld
256K mbuf, UFS ihash, inodedep, NFS hash, ISOFS mount
512K vfscache, SWAP
Memory statistics by type Type Kern
Type InUse MemUse HighUse Limit Requests Limit Limit Size(s)
isadev 13 1K 1K291840K 13 0 0 64
atkbddev 2 1K 1K291840K 2 0 0 32
uc_devlist 24 3K 3K291840K 24 0 0 16,2K
nexusdev 5 1K 1K291840K 5 0 0 16
memdesc 1 4K 4K291840K 1 0 0 4K
mbuf 1 176K 176K291840K 1 0 0 256K
iflogical 51 101K 119K291840K 61 0 0 256,512,2K,8K
iftable 16 9K 9K291840K 16 0 0 16,256,1K,4K
ip6ndp 0 0K 1K291840K 4 0 0 64
ZONE 15 2K 2K291840K 15 0 0 128
VM pgdata 1 128K 128K291840K 1 0 0 128K
UFS mount 15 37K 37K291840K 15 0 0 512,2K,4K,8K
UFS ihash 1 256K 256K291840K 1 0 0 256K
MFS node 249 62K 66K291840K 775 0 0 64,256
FFS node 590 148K 148K291840K 7049 0 0 256
dirrem 0 0K 2K291840K 7475 0 0 32
mkdir 0 0K 1K291840K 8 0 0 32
diradd 0 0K 2K291840K 7477 0 0 32
freefile 0 0K 1K291840K 6255 0 0 32
freeblks 0 0K 1K291840K 167 0 0 128
freefrag 0 0K 1K291840K 1123 0 0 32
allocindir 0 0K 18K291840K 6494 0 0 64
indirdep 0 0K 17K291840K 240 0 0 32,16K
allocdirect 0 0K 1K291840K 2943 0 0 64
bmsafemap 0 0K 1K291840K 6451 0 0 32
newblk 1 1K 1K291840K 9438 0 0 32,256
inodedep 1 256K 264K291840K 20184 0 0 128,256K
pagedep 1 64K 65K291840K 6241 0 0 64,64K
temp 1226 91K 106K291840K 1153815 0 0 16,32,64,128,256,512,1K,2K,4K,8K,16K
devbuf 421 2157K 2157K291840K 870 0 0 16,32,64,128,256,512,1K,2K,4K,8K,16K,32K,64K
lockf 40 3K 3K291840K 1528418 0 0 64
atexit 15 1K 1K291840K 15 0 0 16
zombie 0 0K 1K291840K 291118 0 0 128
NFS hash 1 256K 256K291840K 1 0 0 256K
NQNFS Lease 1 1K 1K291840K 1 0 0 1K
NFS daemon 1 1K 1K291840K 1 0 0 256
syncache 1 8K 8K291840K 1 0 0 8K
tseg_qent 0 0K 1K291840K 70 0 0 32
COS 76 23K 25K291840K 84 0 0 16,32,64,128,1K
BPF 83 2K 2K291840K 83 0 0 16
MSDOSFS mount 1 128K 128K291840K 1 0 0 128K
NULLFS hash 1 1K 1K291840K 1 0 0 64
DEVFS mount 2 1K 1K291840K 2 0 0 16
DEVFS name 515 33K 33K291840K 515 0 0 64
DEVFS node 501 61K 61K291840K 508 0 0 16,128
vnodes 28 7K 7K291840K 429 0 0 16,32,64,128,256
mount 15 8K 8K291840K 17 0 0 16,128,512
cluster_save buffer 0 0K 1K291840K 244 0 0 32,64
vfscache 1901 631K 631K291840K 10522 0 0 64,128,512K
BIO buffer 36 72K 518K291840K 1270 0 0 512,2K
pcb 165 16K 17K291840K 156743 0 0 16,32,64,128,2K
soname 83 10K 11K291840K118997991 0 0 16,32,64,128
proc-args 55 2K 3K291840K 675024 0 0 16,32,64,128,256
ptys 32 16K 16K291840K 32 0 0 512
ttys 187 25K 58K291840K 7557 0 0 128,512
kqueue 4 2K 4K291840K 80675 0 0 256,1K
sigio 1 1K 1K291840K 62 0 0 32
file 416 26K 30K291840K 8314502 0 0 64
file desc 74 19K 23K291840K 291330 0 0 256
shm 1 12K 12K291840K 1 0 0 16K
dev_t 297 38K 38K291840K 297 0 0 128
timecounter 10 2K 2K291840K 10 0 0 128
kld 11 121K 126K291840K 34 0 0 16,32,128,1K,4K,32K,128K
AR driver 1 1K 3K291840K 5 0 0 64,512,2K
AD driver 2 2K 3K291840K 85035 0 0 64,1K
Gzip trees 0 0K 46K291840K 140461 0 0 32,64,128,256,512,1K,2K,4K,8K,32K
ISOFS node 1096 137K 137K291840K 1096 0 0 128
ISOFS mount 8 260K 260K291840K 8 0 0 512,256K
sem 3 6K 6K291840K 3 0 0 1K,4K
MD disk 2 2K 2K291840K 2 0 0 16,1K
msg 4 25K 25K291840K 4 0 0 512,4K,16K
rman 72 5K 5K291840K 440 0 0 16,64
ioctlops 0 0K 2K291840K 951 0 0 512,2K
taskqueue 2 1K 1K291840K 2 0 0 32
SWAP 2 549K 549K291840K 2 0 0 32,512K
ATA generic 6 3K 3K291840K 6 0 0 16,512
eventhandler 19 1K 1K291840K 19 0 0 32,64
bus 434 36K 36K291840K 1145 0 0 16,32,64,128,256,512,1K,2K,4K
sysctloid 29 2K 2K291840K 29 0 0 16,32,64
sysctl 0 0K 1K291840K 16474093 0 0 16,32,64
uidinfo 4 1K 1K291840K 210316 0 0 32,128
cred 15 2K 4K291840K 1875757 0 0 128
subproc 150 10K 13K291840K 795074 0 0 32,64,256
proc 2 1K 1K291840K 2 0 0 512
session 11 2K 4K291840K 212680 0 0 128
pgrp 11 1K 1K291840K 212708 0 0 32
ippool 1 1K 1K291840K 1 0 0 32
pfestat 0 0K 1K291840K 2391302 0 0 16,32,64,512
pic 5 1K 1K291840K 5 0 0 64,128
lr 1 1K 1K291840K 1 0 0 512
itable32 119 4K 4K291840K 119 0 0 32
itable16 370 76K 78K291840K 385 0 0 128,256
ifstate238665 15466K 26395K291840K 52956045 0 0 16,32,64,128,256,512,1K
pfe_ipc 0 0K 1K291840K174442553 0 0 16,32,64,128,256,512,1K
mkey 282 5K 6K291840K 500369 0 0 16,32,64,128,1K
ifaddr 40 3K 3K291840K 52 0 0 64
sysctl 0 0K 1K291840K 35 0 0 256
rtable230027 14388K 14498K291840K 18543744 0 0 16,32,64,128,256,512
ifmaddr 137 4K 4K291840K 137 0 0 16,32
ipfw 34 24K 28K291840K 76 0 0 16,32,64,128,256,512,1K,2K,4K
ifstat 171 249K 249K291840K 179 0 0 32,128,256,512,4K
ifdevice 2 1K 1K291840K 2 0 0 32
tagnh 298 26K 37K291840K 9041 0 0 32,128
rcache 4 8K 8K291840K 4 0 0 2K
rnode219096 3725K 3752K291840K 10637467 0 0 16,32,64,256
ingrentry 6 1K 1K291840K 6 0 0 64
inifmulti 2 1K 1K291840K 2 0 0 64
metrics 4 1K 1K291840K 3088 0 0 128
rtnexthop 8624 2056K 2754K291840K 8150269 0 0 16,32,64,128,256,512
iffamily 83 13K 14K291840K 91 0 0 128,256
Memory Totals: In Use Free Requests
42084K 7881K 419239534
ITEM SIZE LIMIT USED FREE REQUESTS
PIPE: 192, 0, 6, 79, 1164380
SWAPMETA: 160, 233016, 0, 0, 0
unpcb: 160, 0, 119, 56, 231032
ripcb: 192, 30000, 7, 35, 63
syncache: 128, 15359, 0, 64, 37309
tcpcb: 576, 30000, 23, 19, 37492
udpcb: 192, 30000, 14, 28, 192528
socket: 256, 30000, 256, 64, 499621
KNOTE: 96, 0, 43, 83, 476
NFSNODE: 352, 0, 0, 0, 0
NFSMOUNT: 544, 0, 0, 0, 0
VNODE: 224, 0, 9676, 61, 9676
NAMEI: 1024, 0, 0, 16, 17192926
VMSPACE: 192, 0, 55, 73, 291171
PROC: 448, 0, 70, 65, 291188
DP fakepg: 64, 0, 0, 0, 0
PV ENTRY: 28, 1473902, 110000, 414151, 226163407
MAP ENTRY: 48, 0, 1377, 1131, 69552056
KMAP ENTRY: 48, 44459, 228, 156, 1182089
MAP: 108, 0, 7, 3, 7
VM OBJECT: 92, 0, 2440, 728, 10081293
337649840 cpu context switches
1591419183 device interrupts
71448921 software interrupts
68975924 traps
3251654479 system calls
15 kernel threads created
285734 fork() calls
5439 vfork() calls
0 rfork() calls
0 swap pager pageins
0 swap pager pages paged in
0 swap pager pageouts
0 swap pager pages paged out
391 vnode pager pageins
497 vnode pager pages paged in
2646 vnode pager pageouts
8576 vnode pager pages paged out
0 page daemon wakeups
0 pages examined by the page daemon
175 pages reactivated
17663246 copy-on-write faults
0 copy-on-write optimized faults
10527332 zero fill pages zeroed
9773105 zero fill pages prezeroed
88 intransit blocking page faults
83929572 total VM faults taken
0 pages affected by kernel thread creation
17521648 pages affected by fork()
463287 pages affected by vfork()
0 pages affected by rfork()
31179249 pages freed
0 pages freed by daemon
20241280 pages freed by exiting processes
26540 pages active
70979 pages inactive
25 pages in VM cache
27640 pages wired down
388614 pages free
4096 bytes per page
72609635 total name lookups
cache hits (94% pos + 5% neg) system 0% per-directory
deletions 0%, falsehits 0%, toolong 0%
interrupt total rate
mux irq10 338037847 61
fxp0 irq11 2 0
ata0 irq14 1197165 0
sio0 irq4 343 0
clk irq0 549185056 99
rtc irq8 702998770 128
Total 1591419183 289
Best regards,
Alain Iltchev
-----Message d'origine-----
De : juniper-nsp-bounces at puck.nether.net
[mailto:juniper-nsp-bounces at puck.nether.net]De la part de David Ball
Envoyé : mercredi 27 juin 2007 17:20
À : juniper-nsp at puck.nether.net
Objet : [j-nsp] MAC table size experience w/VPLS
I have 5 x T640s running JunOS 8.2 and am adding some 10/100
aggregation switches. These switches would be in a ring (say, 3 of
them per ring) and 2 of those switches would have gig uplinks to
different T640s. The switches would run a version of spanning tree
(likely RSTP, unless they can do PVST). Dunno if the format will look
right for everyone, but it should resemble:
T640----------T640
| |
switch----------switch
\ /
switch
A 10/100 customer would connect to a switch, traffic would be QinQ'd
(stacked VLANd) up to the Juniper, where the top tag would be pop'd,
and they would be dumped into a VPLS routing-instance.
While T640s have 2GB of memory, I find myself worrying about long
term memory problems due to MAC table sizes as more customers are
added over time. I imagine this is very case specific and obviously
depends on what else the 640s are doing, but does anyone have any
real-world examples of how they mitigated MAC table size issues with a
growing customer base? I realize there are ways of setting mac table
sizes and aging times, but is this generally how it's done? Anyone
noticed their MAC tables taking up a lot of memory, and if so, how
much and for how many customers (roughly) ?
We're early in our deployment and I'm trying to make sure it's done
right the first time. Thanks for any thoughts/experiences you might
share.
David
_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
More information about the juniper-nsp
mailing list