[j-nsp] Best practice to manage log information

[Beny D Setyawan]

Hi Alain,

I'm sorry to for the late response since I do working deep on this problem.
Finally I already get answer for this particular problem. This problem
happened to all my M10i & M7i router that running JunOS 8.5 R3.4 (this is
the problem) and FYI, I need to use JunOS 8.5R3.4 because of the new PIC 4
port STM-1 SFP interfaces. The history as I said before that my M-series
router hang and need to restart the cfeb and sometime I need to hard reboot
the chassis. During last 2 weeks my 9 M10i suddenly running with backup
routing-engine and my 10 M7i freeze/hang and all running with JunOS 8.5R3.4.
Even the old router that never problem when running JunOS 8.0R34 after I did
upgrade to JunOS 8.5R3.4 also had the same problem. The key is on my M7i
router, during the problem no alarm indicator in the chassis for all
interfaces and chassis itself. Also the router didn't create core dump at
all. JTAC already create PR release for this problem. From the PR said that
this problem happened to JunOS 8.5R3.4 or above.

Since there is no JunOS release to solved this problem, I have to downgrade
all my M-series (total 33 routers) that running JunOS 8.5R3.4 to JunOS
8.4R4.2. It's going to be along hard work weekend ....

Regards,
Beny D Setyawan


-----Original Message-----
From: alain.briant at bt.com [mailto:alain.briant at bt.com] 
Sent: Tuesday, July 01, 2008 6:35 PM
To: benyds at gmail.com; juniper-nsp at puck.nether.net
Subject: RE: [j-nsp] Best practice to manage log information

Hi Beny

Sory for this late answer

I am not sure I catch you well but if you're wondering what is the root
cause of your hard disk activity just see the age of the different files on
it.
If you say that you have some other traces leading to some "Mpls_statistic"
files changed every 5 minutes, I am quite sure that's an heavy load for the
hard disk.
Just think if you realy need this trace (statistics) to be activated !

Regards
Alain


-----Message d'origine-----
De : Beny D Setyawan [mailto:benyds at gmail.com] 
Envoyé : samedi 21 juin 2008 18:34
À : Briant,A,Alain,JPECS R; juniper-nsp at puck.nether.net
Objet : RE: [j-nsp] Best practice to manage log information

Hi Alain,

My router basically use bellow configuration for the system syslog.

syslog {
    user * {
        any emergency;
    }
    host 10.xxx.xxx.xxx { 	### to syslog server ###
        any any;
        authorization any;
        interactive-commands any;
    }
    file messages {
        authorization info;
        daemon any;
        kernel any;
        user any;
        pfe any;
        interactive-commands critical;
    }
    file log_config_user {
        authorization any;
        interactive-commands any;
        archive size 5m files 5 world-readable;
    }
    file new_log {
        any notice;
        authorization info;
        daemon any;
        kernel any;
        archive size 10m files 5;
    }
    source-address 10.xxx.xxx.x;
}

What I'm trying to do is removing log_config_user and new_log and send it to
syslog, also change the configuration of file messages. But in mpls
protocols, we used auto-bandwidth mechanism with file mpls_statistic on it.
Mpls_statistic file changes every 5 minutes and saved it into the harddisk
also making the harddisk doing write-erase. I'm not sure that mpls_statistic
need to be change also based on trends of the traffic itself on the mpls
network. What still out of my mind is the root caused that making harddisk
busy, is it the syslog or mpls_statistic and I'm sure that harddisk working
every time just like others harddsik.


Thanks,
Beny D Setyawan

-----Original Message-----
From: alain.briant at bt.com [mailto:alain.briant at bt.com]
Sent: Friday, June 20, 2008 7:37 PM
To: benyds at gmail.com; juniper-nsp at puck.nether.net
Subject: RE: [j-nsp] Best practice to manage log information

Hi Beny

I believe the best Start is the default syslog config of JUNOS:

system {
    syslog {
        user * {
            any emergency;
        }
        file messages {
            any notice;
            authorization info;
        }
        file interactive-commands {
            interactive-commands any;
        }
    }
}

After that you can add, as new target, the syslog server and add some more
traces but carefully.

You must keep in mind that for local logs (on the hard disk) if you see your
log files rotating too fast (I mean if your files with the default size and
number do not cover more than One week of time) they will be useless.

Regarding the trace-options you must be careful also with some "flag all"
statements that are writing on the disk a hudge amount of data.

We've had sometimes some M series hanged because of some heavy traces.

First thing so is to have a quick look at your log files:
"Show log ?" 
If you see some of them that are rotating too fast, start removing some
traces leading that.

Hope this help

Alain


-----Message d'origine-----
De : juniper-nsp-bounces at puck.nether.net
[mailto:juniper-nsp-bounces at puck.nether.net] De la part de Beny D Setyawan
Envoyé : vendredi 20 juin 2008 13:30 À : juniper-nsp at puck.nether.net Objet :
[j-nsp] Best practice to manage log information

Hi List,

 

Somehow my m-series hang and need to reboot. JTAC suspected that this is due
to harddisk busy, since there were so many log that write-erase to the
harddisk and suggested to reduce that process. Does anyone has information
what is the best practice on how to manage syslog severity on the Juniper
router? Which log should be send to syslog server and should be save also in
the router itself.

The goal is how to make the router healhty by reduce log of changing any
information on the router from harddisk on the routing-engine perspective.
But in the other hand we need the log information for the NMS.

 

Thanks & Rgds,

Beny D Setyawan

 

 

_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp